Install MyST CLI using tools Pythonistas are familiar with (e.g. conda)

[rowanc1]

It is possible to install node dependencies through Conda, including activating the correct version of nodejs. @stevejpurves and I tried this a few months back and couldn't figure it out.

If anyone from the community has experience with this, we would absolutely love the help!! I believe that having this in a more familiar package manager would certainly help with onboarding Python users.

@choldgraf maybe you have some pointers on who can do this sort of stuff in their sleep?!

[choldgraf]

In my experience - there are many in the conda-forge community who are willing to help maintain packages and who know much more about it than I do :-) lots of folks help maintain other parts of the myst python ecosystem as well. Obviously it gets easier when the project is more well-used and known, so this might be more straightforward over time as we direct more attention to this tool.

Maybe @ocefpaf has thoughts on anybody that knows how to manage javascripty things in the conda-forge world?

Another option is that I think we could actually make mystjs installable with python as well, I know that the sphinx-theme-builder uses nodeenv to bundle its own version of node, and that might be an option (ref pyproject.toml)...cc @pradyunsg if he has any thoughts there

[ocefpaf]

Maybe @ocefpaf has thoughts on anybody that knows how to manage javascripty things in the conda-forge world?

I don't know the first thing about JS but we do package node on conda-forge and folks use it to later install JS stuff. Maybe @bollwyvl did some recipes with that? Sorry bollwyvl if I'm misremembering.

[pradyunsg]

Is there a reason the CLI was built on the JS toolchain rather than the Python one? But more broadly, we could use nodeenv under the hood of a PyPI package if we really wanted to do that; but it might actually be cleaner to provide a myst-cli on PyPI that uses myst-parser rather than trying to adapt a npm package to a PyPI package (it works for pyright though, so it's not strictly out of the picture).

[bollwyvl]

Some thoughts on packaging and distributing npm confections as useful CLI tools:

• use ncc single-file distributions
  • nobody wants the 1000s of node_modules files crapped all over their env, much less their path length, space, and case (in)sensitivity on windows
• try to get stuff to install with --ignore-optionals, --ignore-scripts, and anything else that reduces the surface area of drive-by npm modulejacking, etc.
• capture the versions (and full licenses) of the mountain of third-party dependencies that this will inevitably be vendored
• make simple wrappers (e.g. .bat, .sh, etc) rather than requiring nodejs path/to/thing.js
• establish a policy of building (and testing) against the oldest active node LTS
• probably consider alternate, less-insecure js runtimes like deno and bun

as to the python-node-sandwich:

• if really wanting to do this with python, make a wrapper script that's good at dealing with nodejs %PATH% hell
  • but i really just can't recommend a python package installing a "magic" nodejs environment
    • especially one pinned down to a hard version, which seems to be en vogue
• if particularly crazy, it's possible (especially with self-contained scripts) to do run against a v8-in-python like pyminiracer or pyduktape2
  • both appear semi-dead, and have lingering licensing issues because v8

[rowanc1]

This is fantastic -- thank you @bollwyvl!! We are already getting down to a single js file with esbuild, I will look into ncc as well (not sure what the tradeoffs are). Right now the entire CLI is a single file, that can just run via node myst.js -v etc.

For "make [a] simple wrapper", I am still pretty fuzzy here:
is this what would be packaged up and distributed through pypi or conda (along with licenses)? Do you know any examples in the [conda?] ecosystem that: (1) ensure a version of node is installed (maybe you are saying this is optional / we shouldn't); (2) wrap up that single JS file; and (3) provide access without requiring the node cli.js?

Any pointers on that final step would be really helpful - @stevejpurves is also a bit more versed in the python packaging world, and might be able to ask better questions!

@pradyunsg there is some more context about the JS tools here:

• Major upcoming functionality for mystjs executablebooks/meta#838
  A bit of a different focus than the current python tools, more on single articles, JATS, PDF, running natively in the browser, etc. Aiming for good compatibility through the (almost ready) MEP process.

[bollwyvl]

If distributing through PyPI (a well-understood source for conda-forge), it might look something like this with some entry_point cruft like this (stirring in whatever build backend you want).

Then the conda-forge package (which would get automated PRs after PyPI releases) would look something like this. This one just says node, though i'd, again, suggest pinning to even-numbered LTS nodes, or having a single feedstock that built thing and thing-base that doesn't rely on node at all... one should opt into beta testing the odd-numbered releases.

Another tack altogether is targeting WASM, which would be portable, as node/deno/bun and the browser would run it natively, while python (and many other languages) could use one of the various bridge backends such as wasmer, etc. There is nascent tooling for doing this from JS with e.g. QuickJS, but it does open the can of worms of porting to rust, which has robust wasm, node, and python toolchains.

[rowanc1]

This has now been completed!
https://mystmd.org/guide/quickstart#installing-the-myst-markdown-cli

The CLI is available as mystmd across all package managers! 🚀

Thanks all for your help. :)