Add templating for Jupyter notebooks.

[wisygig]

Motivation

Add templates for jupyter notebooks. Examples are available in the README.md file.

API changes

None.

Implementation Notes

Anything that could be helpful to the reviewer. Especially if it not obvious from looking at the diffs.

Functional Tests

see nb_util/tests/test_jupyter_template.py.

[rdvelazquez]

FYI - I got this to work on my computer on my own dummy example. When I first ran it I got the following error:

jupyter_template.py in <module>()
     17     'code': nbformat.v4.new_code_cell,
     18     'markdown': nbformat.v4.new_markdown_cell,
---> 19     'raw': nbformat.v4.new_raw_cell,
     20 }
     21 

AttributeError: module 'nbformat.v4' has no attribute 'new_raw_cell'

When I commented out the 'raw': nbformat.v4.new_raw_cell, line in jupyter_template.py it worked fine....

I may use this for some other applications soon and I'll let you know how that works out.

@wisygig and @dcgoss let me know if you want any review of the code in this PR.

Nice work Jason!

[wisygig]

@rdvelazquez Interesting! Can you run a pip list | grep 'nbformat' in your terminal?

[rdvelazquez]

when I run pip list | grep 'nbformat' I get a fairly long output. It starts with:

grep is not recognized as an internal or external command, operable program or batch file

Here's a snippet of the output for reference (I couldn't really make sense of it):

pip\_vendor\colorama\ansitowin32.py", line 169, in write_and_convert
    self.write_plain_text(text, cursor, len(text))
  File "C:\Users\rvelazquez\AppData\Local\Continuum\Anaconda3\lib\site-packages\
pip\_vendor\colorama\ansitowin32.py", line 175, in write_plain_text
    self.wrapped.flush()
OSError: [Errno 22] Invalid argument
Call stack:
  File "C:\Users\rvelazquez\AppData\Local\Continuum\Anaconda3\Scripts\pip-script
.py", line 5, in <module>
    sys.exit(pip.main())
  File "C:\Users\rvelazquez\AppData\Local\Continuum\Anaconda3\lib\site-packages\
pip\__init__.py", line 249, in main
    return command.main(cmd_args)
  File "C:\Users\rvelazquez\AppData\Local\Continuum\Anaconda3\lib\site-packages\
pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "C:\Users\rvelazquez\AppData\Local\Continuum\Anaconda3\lib\site-packages\
pip\commands\list.py", line 164, in run
    self.output_package_listing(packages, options)

The output ends with:

Message: 'zict (0.1.2)'
Arguments: ()
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='cp12
52'>
OSError: [Errno 22] Invalid argument

I have version 4.2.0 of nbformat installed (both when I list via conda and via pip).

[wisygig]

Ah, that explains it! I was working with nbformat 4.3.0. It should be in the requirements.txt file.

[rdvelazquez]

That was it. I updated to nbformat 4.3.0, uncommented the 'raw': nbformat.v4.new_raw_cell line and it ran just fine. Thanks @wisygig!

[dcgoss]

Hey! I’ve been on vacation since last Friday, so I’ll take a look at this when I return on Monday.

[dcgoss]

Wow, this is an amazing package! @wisygig you should really consider making this into a standalone package, something seems wrong about keeping it embedded into ml-workers.
Can it substitute an array value in for a key?

[rdvelazquez]

I'll second the standalone package comment!

[wisygig]

@dcgoss Yes, but only as a string. For example, {'value': "['gene1', 'gene2', 'gene3']"}.

[dhimmel]

@wisygig thanks for this cool PR. It's definitely a useful utility. I agree with @dcgoss and @rdvelazquez that it makes sense as a package.

We have two options:

1. merge this PR and you can make this a package in the future
2. you make a standalone package now that we add as a dependency.

@dcgoss mentioned that we need some additional changes ml_task_runner.py for this to become operable. I want to make sure this PR goes somewhere, since it contains lot's of great work!