Transition rST to MyST?

[consideRatio]

I've never really become comfortable with rST, but in the past Markdown wasn't a good option because it lacked the ability to reference other files etc in a good way. This has changed with MyST though.

In the JupyterHub org, we have a few projects that have successfully transitioned MyST markdown with Sphinx instead of rST with Sphinx, and it has been a great success in my mind. For me, it has become easier to maintain docs in those projects and I believe it helps our Jupyter ecosystem contributors contribute to docs a bit easier as most of anyone working with Jupyter knows Markdown already.

Would you be interested in piloting the adoption of MyST (with Sphinx) instead of rST (with Sphinx) in the docs for this project?

Related

• JupyterHub Helm chart use of MyST
• MyST Markdown reference

[martindurant]

I think most/all of the other dask projects use rst, so maybe you want to ask more generally at dask/community. I personally am not a great fan of rst, so if this does the job, it may just be what I've been wanting.

[TomAugspurger]

Agreed that it's worth mentioning in dask/community, but I don't think that should block a subproject from adopting myst. Things like the dask-sphinx-them will still work just fine, regardless of whether the source is in rst or markdown.

My experience with MyST has been very positive.

[consideRatio]

I think it can make a lot of sense to acquire some input on a organizational level as well, so I opened dask/community#178. There is value to be consistent across the org, and piloting the use of MyST in this project would break such consistency.

[jcrist]

I generally agree with @jacobtomlinson 's comment here: dask/community#178 (comment) - in particular I also don't see MyST worth switching to for existing work, but may be worth using for new docs.

I personally like rST, but I have also been writing it for years and have memorized all the common syntax I use. I can understand that other devs may be more familiar with markdown, and may find MyST easier to write.

That said, part of the reason something like MyST exists is that the extra directives of reST are kind of necessary for producing nice docs. And since rST is the standard for Python projects (and rST is still needed for writing docstrings), I'd expect devs to be more familiar with the rST form of these directives. If we switched, those familiar with rST would have to relearn the directive syntax in MyST, which adds a mild disincentive in my mind to switching.

Besides directives, for standard prose passages the main syntax issues that people have (IMO) with rST are:

• links
• images
• inline code (need two backticks instead of one)

Other syntax (like headers, bullets, etc...) are the same in markdown, or clear enough (like headers) that people seem to pick them up well enough.

While the syntax for links and images may be foreign to those that already know markdown, I'd argue that:

• Links are common enough in docstrings, and users writing those will already need to know rST syntax.
• Images are relatively rare regardless, looking up the image syntax when you need it doesn't seem too onerous IMO.

In summary - I think since rST is pretty standard across python projects, many devs already know it, it's still required for docstrings, and MyST would still require learning its custom syntax for directives - that switching to MyST (which may be nice) isn't worth it for an existing project like dask-gateway. This isn't a hard blocker from me, but I'd need to see a really compelling argument for me to want to make the switch here.

[consideRatio]

Thanks for chiming in @jcrist!

I consider the headings to be a bit troublesome still, I keep failing to remember what characters to use to represent the different headers.

And since rST is the standard for Python projects (and rST is still needed for writing docstrings),

@choldgraf do you know about this? I've assumed this was configurable depending on what kind of parser you have, and that there were some parser that would allow for this to be done with markdown?

If this isn't the case, then I'm also a lot less positive to transforming a project to use MyST, I've been assuming we can go 100% Markdown at this point.

that switching to MyST (which may be nice) isn't worth it for an existing project like dask-gateway.

I think this situation is more complicated than a single worth it or not. For me, I'd be very happy to transition a project to MyST from rST if I were to be an active maintainer in the project because, but for someone else it may not feel worth it. Of course no individual makes a contribution to make such transition, it requires review effort in the end as well.

I consider the key question to answer would be if its worth the review effort for the reviewer if someone like me who appreciates MyST so much considers it worth the time investment to do the transformation work.

---

I conclude that indeed MyST parsing of docstrings isn't supported yet.

executablebooks/MyST-Parser#228

[choldgraf]

In case others are curious, I did a quick poll of some folks asking about their thoughts on rST last year or so: https://predictablynoisy.com/posts/2020/2020-01-22-rst-thoughts/

Yep docstrings are not supported via Markdown. Sphinx unfortunately hard-codes some assumptions about rST in things like autodoc (if I recall correctly, see sphinx-doc/sphinx#8018 (comment) for example). Though you can still include them via the eval-rst directive:

```{eval-rst}
.. autosummary:: mymodule
```

and so-on.

[choldgraf]

If you're interested in going with "partial support" as a test-case, I might recommend choosing a single sub-section of the docs and converting that entirely to markdown (and enforce that in the future). I think this will require less context-switching than if you did a piecemeal transition here and there

[jcrist]

Thanks Chris and Erik. While I think MyST is interesting, I just don't see transitioning as worth it currently. Perhaps we could try it out in a new dask subproject(?), but I'd be against converting any of the docs here currently - the context switching between syntaxes within a repo (or even cross repos since major dask projects like dask/dask & dask/distributed also use rST) would definitely trip me up (and I'd guess I wouldn't be the only one).

[martindurant]

As another point against MyST, it seems that IDEs don't tend to provide all the helpers that are available for pure MD or RST.

[choldgraf]

All of those rationales make sense to me - I think ultimately it comes down to "Do you want to write rST?" for many people the answer is "oh god no, never" and for that crowd I think that MyST is a good option. But if you are happy writing rST and you expect future contributors to also be happy writing rST, then I think it makes sense 👍

@martindurant (just a note that there is a vscode extension for myst! https://marketplace.visualstudio.com/items?itemName=ExecutableBookProject.myst-highlight)

[consideRatio]

Thanks for chiming in everyone ❤️!

For me the criteria to put in work on this is that there was a consensus that its wanted which I perceive there not to be, due to that is it okay for me to close this topic as resolved?

[jcrist]

Yeah, I'd say so (at least for Dask-Gateway). Perhaps if a new dask subproject is created we can safely try out MyST there without causing development churn on other subprojects. Thanks Erik!