🤖 Make name & display_name required properties for KernelSpec

[agoose77]

The nbformat spec requires that name and display_name are set in a kernelspec object. Presently, we don't require either of these fields.

This is a problem when e.g. invoking jupytext on a notebook that only has name:

jupytext input.md --output output.md --from md:myst --to md:myst

which fails with:

[jupytext] Reading input.md in format md:myst
Traceback (most recent call last):
  File "/tmp/tmp.jZemAlJPwg/.venv/bin/jupytext", line 8, in <module>
    sys.exit(jupytext())
             ^^^^^^^^^^
  File "/tmp/tmp.jZemAlJPwg/.venv/lib/python3.11/site-packages/jupytext/cli.py", line 497, in jupytext
    exit_code += jupytext_single_file(nb_file, args, log)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp.jZemAlJPwg/.venv/lib/python3.11/site-packages/jupytext/cli.py", line 561, in jupytext_single_file
    notebook = read(nb_file, fmt=fmt, config=config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp.jZemAlJPwg/.venv/lib/python3.11/site-packages/jupytext/jupytext.py", line 432, in read
    return read(stream, as_version=as_version, fmt=fmt, config=config, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp.jZemAlJPwg/.venv/lib/python3.11/site-packages/jupytext/jupytext.py", line 441, in read
    return reads(fp.read(), fmt, config=config, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp.jZemAlJPwg/.venv/lib/python3.11/site-packages/jupytext/jupytext.py", line 390, in reads
    notebook = reader.reads(text, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp.jZemAlJPwg/.venv/lib/python3.11/site-packages/jupytext/jupytext.py", line 103, in reads
    return myst_to_notebook(s)
           ^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp.jZemAlJPwg/.venv/lib/python3.11/site-packages/jupytext/myst.py", line 295, in myst_to_notebook
    notebook = nbf_version.new_notebook(**kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp.jZemAlJPwg/.venv/lib/python3.11/site-packages/nbformat/v4/nbbase.py", line 170, in new_notebook
    validate(nb)
  File "/tmp/tmp.jZemAlJPwg/.venv/lib/python3.11/site-packages/nbformat/v4/nbbase.py", line 41, in validate
    return validate_orig(node, ref=ref, version=nbformat)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/tmp/tmp.jZemAlJPwg/.venv/lib/python3.11/site-packages/nbformat/validator.py", line 509, in validate
    raise error
nbformat.validator.NotebookValidationError: 'display_name' is a required property

Failed validating 'required' in notebook['properties']['metadata']['properties']['kernelspec']:

On instance['metadata']['kernelspec']:
{'name': 'python3'}

This PR makes name and display_name "soft" required; default values will be chosen if they're missing, but warnings will also be issued.

In future, we can eventually just make them required.

This PR also changes our execution hash key strategy. Now, it includes the kernelspec name! It's highly unlikely to be a common bug, but users could e.g. switch from one Python kernel to another, and the cache would remain unchanged. This would definitely be a problem if different kernels had different environments.

As a result of this change, existing caches will be invalidated. But, I think that's OK given how young execution is (I don't think we advertise it as stable yet).

[changeset-bot]

🦋 Changeset detected

Latest commit: 9688ccd

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 4 packages

Name	Type
myst-frontmatter	Minor
myst-common	Minor
myst-config	Minor
myst-spec-ext	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[rowanc1]

Looks good, haven't tested. Let me know if you want me to do that, but the changes look straight forward.

[rowanc1]

I suppose all hashes will change. That is fine though.

[agoose77]

Yes, I made a note of that in the description. It's unfortunate, and we could add some backwards-compat logic, but I think it's early enough in the project's history to just break it.

[rowanc1]

This should never be triggered in the current CLI workflow, right? Because we always have a valid object coming out of frontmatter validation?

[agoose77]

If you don't define frontmatter.kernelspec, you don't get a kernelspec object:

mystmd/packages/myst-frontmatter/src/page/types.ts

Line 18 in 4a5d3b7

kernelspec?: KernelSpec;

[agoose77]

@rowanc1 are you happy to merge this in now?