Guess params by introspecting the _parameters_ cell

[fcollonval]

This is a reboot of #158

It adds a new option --help-notebook to create a new cli usage:

papermill --help-notebook input_notebook.ipynb

Due to that I had to remove the required statement on OUTPUT_PATH. But the test is still done within the papermill function to keep the behavior if --help-notebook is not set.

Fixes #225 => in particular the default parameters resulting of the code introspection are added as a dictionary in metadata nb.metadata['papermill']['default_parameters'].

For now this add the ability for Python. I'm not familiar to the other languages to build the proper regex.

Note: I use a regex instead of ast for Python for two reasons: ast removes all comments but I use them to get a help comment. ast parsing for non-python language won't be available. So it seems better to use regex for all languages.

A follow up could be to address #55, but there is a important issue on how to fairly evaluate the variable type for non-python language.

[codecov]

Codecov Report

Merging #531 into main will increase coverage by 0.52%.
The diff coverage is 98.49%.

@@            Coverage Diff             @@
##             main     #531      +/-   ##
==========================================
+ Coverage   91.77%   92.30%   +0.52%     
==========================================
  Files          14       16       +2     
  Lines        1289     1403     +114     
==========================================
+ Hits         1183     1295     +112     
- Misses        106      108       +2     

[MSeal]

Thanks for posting the PR -- I just got back and will get a chance to go through it in the next day or two

[MSeal]

Thanks for building this out. There's some comments to address and probably a little bit of rearrangement needed to avoid code paths when executing but I think the core implementation idea is solid in approach.

[MSeal]

Really nice actually -- it even extracts the comment on the end of foo = bar # a comment!

[MSeal]

The other way to do this for Python is to use the ast module (e.g. https://www.mattlayman.com/blog/2018/decipher-python-ast/) and extracting the assignments programmatically. However this won't work cross-language and could even hiccup with ipython specific capabilities with python kernels (e.g. !run-something). So I think regex is the KISS way to go.

[fcollonval]

That is exactly my note in the PR description 😉

Note: I use a regex instead of ast for Python for two reasons: ast removes all comments but I use them to get a help comment. ast parsing for non-python language won't be available. So it seems better to use regex for all languages.

[MSeal]

Yep just re-enforcing the idea

[MSeal]

Probably want more examples (maybe by supplying the parameter cell directly to the inspect call in a different test parameterization) with a variety of code arrangements to prove your comment compressing code all works as expected.

[fcollonval]

This is actually done in papermill/tests/test_translators.py:test_inspect_python (at line 112)

[MSeal]

Thanks for the improvement and the quick turn around on the review!