Add new guide about Jupyter in Sphinx

[astrojuanlu]

Still work in progress, but I already decided what sections to cover and created a local project to attach screenshots. I am not done with the tricky bits, but at least I now understood (and hopefully conveyed) the relationship between nbsphinx and MyST-NB for processing .ipynb notebooks. Will add a few extra things for completeness.

[astrojuanlu]

Rendered version: https://docs--8283.org.readthedocs.build/en/8283/guides/jupyter.html

[astrojuanlu]

This is now ready for review. I decided to remove the "Live code execution on HTML documentation" because it's not strictly related to Jupyter notebooks, and the current content is dense enough already.

[astrojuanlu]

And by the way, this is the project I used to test everything and generate the screenshots: https://github.com/astrojuanlu/jupyter-on-sphinx

[ericholscher]

@astrojuanlu probably better to use https://github.com/readthedocs/test-builds, which is what that repo is for.

[ericholscher]

This is a good start, but it feels a bit disjoint. Without having a lot of knowledge of Jupyter, I didn't really get much understanding of the value of these tasks or what my goals with them would be. I feel like we need a lot more context and explanation of why this is useful.

I'd also really like us to have an opinion more. It seems like Myst NB is simpler and probably the better approach? Why does Myst-NB exist is nbsphinx does the exact same thing? Should I use Jupyterlab or something else for editing? I feel like we could be providing a lot more guidance to people here, or at least making some assumptions to clarify the copy and workflow.

[astrojuanlu]

Thanks @ericholscher for the review! Point well taken, this is mostly useful for people that already know Jupyter and they know what they want. It also contains some examples, even though I was trying to avoid sounding too opinionated. Will rework it a bit so it contains more general context and guidance, and more explicit recommendations.

[astrojuanlu]

Did a handful of changes, the guide is now longer but hopefully much better.

Pinging also @choldgraf, @chrisjsewell, and @mgeier since I used your input for some parts of it 🚀

[astrojuanlu]

Also, I took the liberty of adding one extra commit to address #8291, let me know if it's too early for it.

[choldgraf]

A few comments and suggested edits from me! In general this is a nice start that does a good job of covering the basics and several options! I like how you are following up sections with links to more in-depth documentation 👍

[choldgraf]

I'm leaning towards just not telling people what they "should" do - it feels like that might become semi-arbitrary and opens up the tutorial to debate between the two projects. It seems cleaner to just focus on the "feature matrix" and UI/UX of each tool, rather than to give opinionated recommendations. Does that make sense?

[astrojuanlu]

I agree that doing specific recommendations might be contentious. However, the two projects are very similar, and I (as a user) was very confused about the differences between them, so if we don't offer at least a small amount of guidance, I think we might leave our users confused or looking for answers. I was very careful here to weigh the pros and cons, and @chrisjsewell corrected me in several places where I was wrong about MyST-NB. What do you think @ericholscher ?

[ericholscher]

I guess I still don't understand the differences, or why they both exist. If there is a short explanation of when one would be better, we should include that. If there are basically just 2 libraries that do the same thing, we should mention that, and clarify when it's better to use one. I don't know if we can do much more than that.

[astrojuanlu]

That's already done in the document (there's a background section explaining the technical differences between the two). About the "why", I don't think this is the place to clarify that -- the interested readers could head over spatialaudio/nbsphinx#420 for some historical discussion :) But the two projects have evolved since then.

[choldgraf]

to be honest this one feels a little bit out-of-place given the tutorial's topic. I feel like "generate a gallery from multiple pages" is a generic use-case that is not notebook-specific (though it is a cool feature of nbsphinx). I almost feel like it deserves its own section (that covers sphinx-gallery primarily, and links to nbsphinx to talk about notebook-specific functionality) rather than to talk about galleries in the context of notebooks.

[astrojuanlu]

I'm not sure. While "generating galleries from notebooks" is something that several projects are doing or seeking to do, "generating galleries from multiple pages" sounds like too abstract of a use case to have a guide dedicated to it.

For me as a user, creating galleries is 40 % of the reason why notebooks are cool, since thumbnails can be automatically generated and notebooks are an excellent choice for creating how-to guides. About sphinx-gallery, I tried using it in a personal project that was using notebooks for examples already poliastro/poliastro#828 but the workflow ended up being more complex, so we abandoned this approach.

[choldgraf]

I'll leave this one to you - I think it comes down to whether "creating a gallery of notebooks" is a "core part of the notebook experience". In my opinion, it isn't, but I don't feel strongly about it, so if others believe strongly that it is, then I'll defer

[ericholscher]

I had the same feedback -- I think we probably want a different guide on galleries/examples, and to link from that from this page. #8283 (comment)

[astrojuanlu]

Fair enough, perhaps I was being too stubborn - I agree to make another guide about example galleries. However, given that I would need to learn sphinx-gallery and investigate if there are other alternatives, do you folks think we can do it in another pull request? That way we can get this closer to being published.

[astrojuanlu]

Thanks @choldgraf for your comments 🙏🏽 I did another round of changes. Apart from some outstanding questions about scope or intent, I think this is it. cc @readthedocs/core

[chrisjsewell]

Thanks @astrojuanlu one question though; where are these guides in the docs lol?!

I tried https://docs--8283.org.readthedocs.build/en/8283/ and could not see them, and even searched for Jupyter and the result was empty

[ericholscher]

@chrisjsewell unfortunately you need to go to https://docs.readthedocs.io/en/latest/guides/index.html. We should fix this tho, and probably just have a high-level guides section that includes them all on the front page.

Edit: #8300 will fix part of this

[humitos]

Looks good to me! I left several comments and suggestions that may be addressed before merging.

[astrojuanlu]

Done with the last round of changes 💪🏽

[humitos]

LGTM!

[astrojuanlu]

Thanks everyone for the comments, I now enabled intersphinx for all projects and @humitos stamped his seal of approval, so I will merge. We can improve it, but I think it is already a valuable resource I wish I had 5 years ago. 🚀

[choldgraf]

🚀

[humitos]

@humitos stamped his seal of approval

The power of the 🟢 , 💪🏼