Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Command-line interface: jupyter execute #165

Merged
merged 26 commits into from
Nov 2, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 57 additions & 2 deletions docs/client.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,9 @@ Executing notebooks can be very helpful, for example, to run all notebooks
in Python library in one step, or as a way to automate the data analysis in
projects involving more than one notebook.

Executing notebooks using the Python API interface
--------------------------------------------------
Using the Python API interface
------------------------------

This section will illustrate the Python API interface.

Example
Expand Down Expand Up @@ -166,3 +167,57 @@ a UI.

If you can't view widget results after execution, you may need to select
:menuselection:`Trust Notebook` under the :menuselection:`File` menu.

Using a command-line interface
------------------------------

This section will illustrate how to run notebooks from your terminal. It supports the most basic use case. For more sophisticated execution options, consider the `papermill <https://pypi.org/project/papermill/>`_ library.

This library's command line tool is available by running `jupyter execute`. It expects notebooks as input arguments and accepts optional flags to modify the default behavior.

Running a notebook is this easy.::

jupyter execute notebook.ipynb

You can pass more than one notebook as well.::

jupyter execute notebook.ipynb notebook2.ipynb

By default, notebook errors will be raised and printed into the terminal. You can suppress them by passing the ``--allow-errors`` flag.::

jupyter execute notebook.ipynb --allow-errors

Other options allow you to modify the timeout length and dictate the kernel in use. A full set of options is available via the help command.::

jupyter execute --help

An application used to execute notebook files (*.ipynb)

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
<cmd> --help-all

--allow-errors
Errors are ignored and execution is continued until the end of the notebook.
Equivalent to: [--NbClientApp.allow_errors=True]
--timeout=<Int>
The time to wait (in seconds) for output from executions. If a cell
execution takes longer, a TimeoutError is raised. ``-1`` will disable the
timeout.
Default: None
Equivalent to: [--NbClientApp.timeout]
--startup_timeout=<Int>
The time to wait (in seconds) for the kernel to start. If kernel startup
takes longer, a RuntimeError is raised.
Default: 60
Equivalent to: [--NbClientApp.startup_timeout]
--kernel_name=<Unicode>
Name of kernel to use to execute the cells. If not set, use the kernel_spec
embedded in the notebook.
Default: ''
Equivalent to: [--NbClientApp.kernel_name]

To see all available configurables, use `--help-all`.
14 changes: 6 additions & 8 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,12 +13,12 @@ Welcome to nbclient

---

**NBClient**, a client library for programmatic notebook execution, is a tool for running Jupyter Notebooks in
different execution contexts. NBClient was spun out of `nbconvert <https://nbconvert.readthedocs.io/en/latest/>`_'s
former ``ExecutePreprocessor``.

**NBClient** lets you **execute** notebooks.

A client library for programmatic notebook execution, **NBClient** is a tool for running Jupyter Notebooks in
different execution contexts, including the command line. NBClient was spun out of `nbconvert <https://nbconvert.readthedocs.io/en/latest/>`_'s
former ``ExecutePreprocessor``.

Demo
----

Expand All @@ -30,9 +30,7 @@ To demo **NBClient** interactively, click the Binder link below:
Origins
-------

This library used to be part of `nbconvert <https://nbconvert.readthedocs.io/en/latest/>`_ and was extracted into its own
library for easier updating and importing by downstream libraries and
applications.
This library used to be part of `nbconvert <https://nbconvert.readthedocs.io/en/latest/>`_ and was extracted into its ownlibrary for easier updating and importing by downstream libraries and applications.

Python Version Support
----------------------
Expand Down Expand Up @@ -63,7 +61,7 @@ this documentation section will help you.

.. toctree::
:maxdepth: 3
:caption: Reference
:caption: Table of Contents

reference/index.rst
reference/nbclient.tests.rst
Expand Down
157 changes: 157 additions & 0 deletions nbclient/cli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
import logging
import pathlib
import sys
from textwrap import dedent

import nbformat
from jupyter_core.application import JupyterApp
from traitlets import Bool, Integer, List, Unicode, default
from traitlets.config import catch_config_error

from nbclient import __version__

from .client import NotebookClient

nbclient_aliases = {
'timeout': 'NbClientApp.timeout',
'startup_timeout': 'NbClientApp.startup_timeout',
'kernel_name': 'NbClientApp.kernel_name',
}

nbclient_flags = {
'allow-errors': (
{
'NbClientApp': {
'allow_errors': True,
},
},
"Errors are ignored and execution is continued until the end of the notebook.",
),
}


class NbClientApp(JupyterApp):
"""
An application used to execute notebook files (``*.ipynb``)
"""

version = __version__
name = 'jupyter-execute'
aliases = nbclient_aliases
flags = nbclient_flags

description = Unicode("An application used to execute notebook files (*.ipynb)")
notebooks = List([], help="Path of notebooks to convert").tag(config=True)
timeout: int = Integer(
None,
allow_none=True,
help=dedent(
"""
The time to wait (in seconds) for output from executions.
If a cell execution takes longer, a TimeoutError is raised.
``-1`` will disable the timeout.
"""
),
).tag(config=True)
startup_timeout: int = Integer(
60,
help=dedent(
"""
The time to wait (in seconds) for the kernel to start.
If kernel startup takes longer, a RuntimeError is
raised.
"""
),
).tag(config=True)
allow_errors: bool = Bool(
False,
help=dedent(
"""
When a cell raises an error the default behavior is that
execution is stopped and a `CellExecutionError`
is raised.
If this flag is provided, errors are ignored and execution
is continued until the end of the notebook.
"""
),
).tag(config=True)
skip_cells_with_tag: str = Unicode(
'skip-execution',
help=dedent(
"""
Name of the cell tag to use to denote a cell that should be skipped.
"""
),
).tag(config=True)
kernel_name: str = Unicode(
'',
help=dedent(
"""
Name of kernel to use to execute the cells.
If not set, use the kernel_spec embedded in the notebook.
"""
),
).tag(config=True)

@default('log_level')
def _log_level_default(self):
return logging.INFO

@catch_config_error
def initialize(self, argv=None):
super().initialize(argv)

# Get notebooks to run
self.notebooks = self.get_notebooks()

# If there are none, throw an error
if not self.notebooks:
print("jupyter-execute: error: expected path to notebook")
sys.exit(-1)

# Loop and run them one by one
[self.run_notebook(path) for path in self.notebooks]

def get_notebooks(self):
# If notebooks were provided from the command line, use those
if self.extra_args:
notebooks = self.extra_args
# If not, look to the class attribute
else:
notebooks = self.notebooks

# Return what we got.
return notebooks

def run_notebook(self, notebook_path):
# Log it
self.log.info(f"Executing {notebook_path}")

name = notebook_path.replace(".ipynb", "")

# Get its parent directory so we can add it to the $PATH
path = pathlib.Path(notebook_path).parent.absolute()

# Set the intput file paths
input_path = f"{name}.ipynb"

# Open up the notebook we're going to run
with open(input_path) as f:
nb = nbformat.read(f, as_version=4)

# Configure nbclient to run the notebook
client = NotebookClient(
nb,
timeout=self.timeout,
startup_timeout=self.startup_timeout,
skip_cells_with_tag=self.skip_cells_with_tag,
allow_errors=self.allow_errors,
kernel_name=self.kernel_name,
resources={'metadata': {'path': path}},
)

# Run it
client.execute()


main = NbClientApp.launch_instance
47 changes: 47 additions & 0 deletions nbclient/tests/files/Error.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "d200673b",
"metadata": {},
"outputs": [
{
"ename": "ZeroDivisionError",
"evalue": "division by zero",
"output_type": "error",
"traceback": [
"\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
"\u001b[0;31mZeroDivisionError\u001b[0m Traceback (most recent call last)",
"\u001b[0;32m/tmp/ipykernel_1277493/182040962.py\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0;36m0\u001b[0m\u001b[0;34m/\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
"\u001b[0;31mZeroDivisionError\u001b[0m: division by zero"
]
}
],
"source": [
"0/0"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.5"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
2 changes: 1 addition & 1 deletion nbclient/tests/files/Skip Execution with Cell Tag.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -34,4 +34,4 @@
"metadata": {},
"nbformat": 4,
"nbformat_minor": 1
}
}
5 changes: 5 additions & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,11 @@ def read_reqs(fname):
python_requires=">=3.6.1",
install_requires=requirements,
extras_require=extras_require,
entry_points={
'console_scripts': [
'jupyter-execute = nbclient.cli:main',
],
},
project_urls={
'Documentation': 'https://nbclient.readthedocs.io',
'Funding': 'https://numfocus.org/',
Expand Down