0.5.13: pytest warnings and fails

[kloczek]

I'm trying to package your module as an rpm package. So I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix>
• run pytest with PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>

Two units are failing and there some pytest wearnings. Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-nbclient-0.5.13-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-nbclient-0.5.13-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.12, pytest-6.2.5, py-1.11.0, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/nbclient-0.5.13
collected 91 items

nbclient/tests/test_client.py .......F.....................................F........................................ [ 94%]
nbclient/tests/test_util.py ...ss                                        [100%]

=================================== FAILURES ===================================
______________ test_run_all_notebooks[JupyterWidgets.ipynb-opts7] ______________

input_name = 'JupyterWidgets.ipynb', opts = {'kernel_name': 'python'}

    @pytest.mark.parametrize(
        ["input_name", "opts"],
        [
            ("Other Comms.ipynb", dict(kernel_name="python")),
            ("Clear Output.ipynb", dict(kernel_name="python")),
            ("Empty Cell.ipynb", dict(kernel_name="python")),
            ("Factorials.ipynb", dict(kernel_name="python")),
            ("HelloWorld.ipynb", dict(kernel_name="python")),
            ("Inline Image.ipynb", dict(kernel_name="python")),
            (
                "Interrupt.ipynb",
                dict(kernel_name="python", timeout=1, interrupt_on_timeout=True, allow_errors=True),
            ),
            ("JupyterWidgets.ipynb", dict(kernel_name="python")),
            ("Skip Exceptions with Cell Tags.ipynb", dict(kernel_name="python")),
            ("Skip Exceptions.ipynb", dict(kernel_name="python", allow_errors=True)),
            ("Skip Execution with Cell Tag.ipynb", dict(kernel_name="python")),
            ("SVG.ipynb", dict(kernel_name="python")),
            ("Unicode.ipynb", dict(kernel_name="python")),
            ("UnicodePy3.ipynb", dict(kernel_name="python")),
            ("update-display-id.ipynb", dict(kernel_name="python")),
            ("Check History in Memory.ipynb", dict(kernel_name="python")),
        ],
    )
    def test_run_all_notebooks(input_name, opts):
        """Runs a series of test notebooks and compares them to their actual output"""
        input_file = os.path.join(current_dir, 'files', input_name)
>       input_nb, output_nb = run_notebook(input_file, opts, notebook_resources())

nbclient/tests/test_client.py:296:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
nbclient/tests/test_client.py:93: in run_notebook
    output_nb = executor.execute()
nbclient/util.py:84: in wrapped
    return just_run(coro(*args, **kwargs))
nbclient/util.py:62: in just_run
    return loop.run_until_complete(coro)
/usr/lib64/python3.8/asyncio/base_events.py:616: in run_until_complete
    return future.result()
nbclient/client.py:663: in async_execute
    await self.async_execute_cell(
nbclient/client.py:965: in async_execute_cell
    await self._check_raise_for_error(cell, cell_index, exec_reply)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <nbclient.client.NotebookClient object at 0x7f848c745550>
cell = {'cell_type': 'code', 'metadata': {'execution': {'iopub.status.busy': '2022-03-11T09:38:24.627291Z', 'iopub.execute_in... 'ipywidgets'"]}], 'source': "import ipywidgets\nlabel = ipywidgets.Label('Hello World')\nlabel", 'execution_count': 1}
cell_index = 0
exec_reply = {'buffers': [], 'content': {'ename': 'ModuleNotFoundError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'a19cb5ee-...e, 'engine': 'a19cb5ee-5291-4c63-81d7-d9f42e82ec09', 'started': '2022-03-11T09:38:24.627755Z', 'status': 'error'}, ...}

    async def _check_raise_for_error(
        self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
    ) -> None:

        if exec_reply is None:
            return None

        exec_reply_content = exec_reply['content']
        if exec_reply_content['status'] != 'error':
            return None

        cell_allows_errors = (not self.force_raise_errors) and (
            self.allow_errors
            or exec_reply_content.get('ename') in self.allow_error_names
            or "raises-exception" in cell.metadata.get("tags", [])
        )
        await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
        if not cell_allows_errors:
>           raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E           ------------------
E           import ipywidgets
E           label = ipywidgets.Label('Hello World')
E           label
E           ------------------
E
E           ---------------------------------------------------------------------------
E           ModuleNotFoundError                       Traceback (most recent call last)
E           <IPY-INPUT> in <module>
E           ----> 1 import ipywidgets
E                 2 label = ipywidgets.Label('Hello World')
E                 3 label
E
E           ModuleNotFoundError: No module named 'ipywidgets'
E           ModuleNotFoundError: No module named 'ipywidgets'

nbclient/client.py:862: CellExecutionError
----------------------------- Captured stderr call -----------------------------
[IPKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled
___________________________ TestExecute.test_widgets ___________________________

self = <nbclient.tests.test_client.TestExecute testMethod=test_widgets>

    def test_widgets(self):
        """Runs a test notebook with widgets and checks the widget state is saved."""
        input_file = os.path.join(current_dir, 'files', 'JupyterWidgets.ipynb')
        opts = dict(kernel_name="python")
        res = self.build_resources()
        res['metadata']['path'] = os.path.dirname(input_file)
>       input_nb, output_nb = run_notebook(input_file, opts, res)

nbclient/tests/test_client.py:736:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
nbclient/tests/test_client.py:93: in run_notebook
    output_nb = executor.execute()
nbclient/util.py:84: in wrapped
    return just_run(coro(*args, **kwargs))
nbclient/util.py:62: in just_run
    return loop.run_until_complete(coro)
/usr/lib/python3.8/site-packages/nest_asyncio.py:81: in run_until_complete
    return f.result()
/usr/lib64/python3.8/asyncio/futures.py:178: in result
    raise self._exception
/usr/lib64/python3.8/asyncio/tasks.py:280: in __step
    result = coro.send(None)
nbclient/client.py:663: in async_execute
    await self.async_execute_cell(
nbclient/client.py:965: in async_execute_cell
    await self._check_raise_for_error(cell, cell_index, exec_reply)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <nbclient.client.NotebookClient object at 0x7f848e3dd4c0>
cell = {'cell_type': 'code', 'metadata': {'execution': {'iopub.status.busy': '2022-03-11T09:39:55.012475Z', 'iopub.execute_in... 'ipywidgets'"]}], 'source': "import ipywidgets\nlabel = ipywidgets.Label('Hello World')\nlabel", 'execution_count': 1}
cell_index = 0
exec_reply = {'buffers': [], 'content': {'ename': 'ModuleNotFoundError', 'engine_info': {'engine_id': -1, 'engine_uuid': '70545c82-...e, 'engine': '70545c82-48b8-4fef-a42c-1a4b19589fbd', 'started': '2022-03-11T09:39:55.013193Z', 'status': 'error'}, ...}

    async def _check_raise_for_error(
        self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
    ) -> None:

        if exec_reply is None:
            return None

        exec_reply_content = exec_reply['content']
        if exec_reply_content['status'] != 'error':
            return None

        cell_allows_errors = (not self.force_raise_errors) and (
            self.allow_errors
            or exec_reply_content.get('ename') in self.allow_error_names
            or "raises-exception" in cell.metadata.get("tags", [])
        )
        await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
        if not cell_allows_errors:
>           raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)
E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:
E           ------------------
E           import ipywidgets
E           label = ipywidgets.Label('Hello World')
E           label
E           ------------------
E
E           ---------------------------------------------------------------------------
E           ModuleNotFoundError                       Traceback (most recent call last)
E           <IPY-INPUT> in <module>
E           ----> 1 import ipywidgets
E                 2 label = ipywidgets.Label('Hello World')
E                 3 label
E
E           ModuleNotFoundError: No module named 'ipywidgets'
E           ModuleNotFoundError: No module named 'ipywidgets'

nbclient/client.py:862: CellExecutionError
----------------------------- Captured stderr call -----------------------------
[IPKernelApp] WARNING | debugpy_stream undefined, debugging will not be enabled
=============================== warnings summary ===============================
../../../../../usr/lib/python3.8/site-packages/nbconvert/exporters/html.py:110
  /usr/lib/python3.8/site-packages/nbconvert/exporters/html.py:110: DeprecationWarning: 'contextfilter' is renamed to 'pass_context', the old name will be removed in Jinja 3.1.
    def markdown2html(self, context, source):

nbclient/tests/test_util.py:62
  /home/tkloczko/rpmbuild/BUILD/nbclient-0.5.13/nbclient/tests/test_util.py:62: PytestUnknownMarkWarning: Unknown pytest.mark.asyncio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
    @pytest.mark.asyncio

nbclient/tests/test_util.py:69
  /home/tkloczko/rpmbuild/BUILD/nbclient-0.5.13/nbclient/tests/test_util.py:69: PytestUnknownMarkWarning: Unknown pytest.mark.asyncio - is this a typo?  You can register custom marks to avoid this warning - for details, see https://docs.pytest.org/en/stable/mark.html
    @pytest.mark.asyncio

nbclient/tests/test_client.py: 110 warnings
  /usr/lib/python3.8/site-packages/nbformat/validator.py:355: DeprecationWarning: Passing a schema to Validator.iter_errors is deprecated and will be removed in a future release. Call validator.evolve(schema=new_schema).iter_errors(...) instead.
    for error in errors:

nbclient/tests/test_client.py::TestRunCell::test_cell_hooks
nbclient/tests/test_client.py::TestRunCell::test_display_data_message
nbclient/tests/test_client.py::TestRunCell::test_error_cell_hooks
nbclient/tests/test_client.py::TestRunCell::test_execution_count_message_ignored_on_override
nbclient/tests/test_client.py::TestRunCell::test_widget_comm_buffer_message_single
  /usr/lib64/python3.8/unittest/mock.py:2030: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    setattr(_type, entry, MagicProxy(entry, self))

nbclient/tests/test_client.py::TestRunCell::test_deadline_iopub
  /usr/lib64/python3.8/unittest/mock.py:740: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    return object.__setattr__(self, name, value)

nbclient/tests/test_client.py::TestRunCell::test_eventual_deadline_iopub
  /home/tkloczko/rpmbuild/BUILD/nbclient-0.5.13/nbclient/tests/test_client.py:936: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    message_mock.side_effect = message_seq(list(message_mock.side_effect)[:-1])

nbclient/tests/test_client.py::TestRunCell::test_eventual_deadline_iopub
  /usr/lib64/python3.8/unittest/mock.py:773: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    return object.__setattr__(self, name, value)

nbclient/tests/test_client.py::TestRunCell::test_execution_count_message
  /usr/lib64/python3.8/unittest/mock.py:1063: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    _safe_super(CallableMixin, self).__init__(

nbclient/tests/test_client.py::TestRunCell::test_non_code_cell_hooks
  /usr/lib64/python3.8/unittest/mock.py:1085: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    return self._execute_mock_call(*args, **kwargs)

nbclient/tests/test_util.py::test_run_hook_sync
nbclient/tests/test_util.py::test_run_hook_async
  /usr/lib/python3.8/site-packages/_pytest/python.py:172: PytestUnhandledCoroutineWarning: async def functions are not natively supported and have been skipped.
  You need to install a suitable plugin for your async framework, for example:
    - anyio
    - pytest-asyncio
    - pytest-tornasync
    - pytest-trio
    - pytest-twisted
    warnings.warn(PytestUnhandledCoroutineWarning(msg.format(nodeid)))

-- Docs: https://docs.pytest.org/en/stable/warnings.html
=========================== short test summary info ============================
SKIPPED [2] ../../../../../usr/lib/python3.8/site-packages/_pytest/python.py:173: async def function and no async plugin installed (see warnings)
FAILED nbclient/tests/test_client.py::test_run_all_notebooks[JupyterWidgets.ipynb-opts7]
FAILED nbclient/tests/test_client.py::TestExecute::test_widgets - nbclient.ex...
====== 2 failed, 87 passed, 2 skipped, 125 warnings in 106.51s (0:01:46) =======

[davidbrochart]

Tests require ipywidgets.

[kloczek]

After install ipwidgets and pytest-asyncio there are still some warnings

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-nbclient-0.6.3-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-nbclient-0.6.3-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/nbclient-0.6.3
plugins: asyncio-0.18.2
asyncio: mode=legacy
collected 91 items

nbclient/tests/test_client.py ...................................................................................... [ 94%]
nbclient/tests/test_util.py .....                                        [100%]

=============================== warnings summary ===============================
../../../../../usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:191
  /usr/lib/python3.8/site-packages/pytest_asyncio/plugin.py:191: DeprecationWarning: The 'asyncio_mode' default value will change to 'strict' in future, please explicitly use 'asyncio_mode=strict' or 'asyncio_mode=auto' in pytest configuration file.
    config.issue_config_time_warning(LEGACY_MODE, stacklevel=2)

nbclient/tests/test_client.py::TestRunCell::test_cell_hooks
nbclient/tests/test_client.py::TestRunCell::test_error_cell_hooks
nbclient/tests/test_client.py::TestRunCell::test_execution_count_message_ignored_on_override
nbclient/tests/test_client.py::TestRunCell::test_non_code_cell_hooks
  /usr/lib64/python3.8/unittest/mock.py:2076: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    self.name = name
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

nbclient/tests/test_client.py::TestRunCell::test_deadline_iopub
  /usr/lib64/python3.8/unittest/mock.py:740: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    return object.__setattr__(self, name, value)
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

nbclient/tests/test_client.py::TestRunCell::test_display_data_same_id_message
  /usr/lib64/python3.8/enum.py:339: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    return cls.__new__(cls, value)
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

nbclient/tests/test_client.py::TestRunCell::test_eventual_deadline_iopub
  /home/tkloczko/rpmbuild/BUILD/nbclient-0.6.3/nbclient/tests/test_client.py:967: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    message_mock.side_effect = message_seq(list(message_mock.side_effect)[:-1])
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

nbclient/tests/test_client.py::TestRunCell::test_eventual_deadline_iopub
  /usr/lib64/python3.8/unittest/mock.py:773: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    return object.__setattr__(self, name, value)
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

nbclient/tests/test_client.py::TestRunCell::test_widget_comm_buffer_message_single
  /usr/lib/python3.8/site-packages/traitlets/traitlets.py:1561: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
    if method_name in c.__dict__:
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================== 91 passed, 10 warnings in 84.08s (0:01:24) ==================
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-2d3c13c5-3e7c-4cca-9e3a-d551473c307b
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-2d3c13c5-3e7c-4cca-9e3a-d551473c307b'
  warnings.warn(
sys:1: RuntimeWarning: coroutine 'make_async.<locals>._' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

[kloczek]

Tested. Thank you 😄

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-nbclient-0.6.3-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-nbclient-0.6.3-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra
=========================================================================== test session starts ============================================================================
platform linux -- Python 3.8.13, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/tkloczko/rpmbuild/BUILD/nbclient-0.6.3, configfile: pytest.ini
plugins: asyncio-0.18.2
asyncio: mode=auto
collected 91 items

nbclient/tests/test_client.py ...................................................................................... [ 94%]
nbclient/tests/test_util.py .....                                        [100%]

======================== 91 passed in 83.97s (0:01:23) =========================
/usr/lib/python3.8/site-packages/_pytest/pathlib.py:79: PytestWarning: (rm_rf) error removing /tmp/pytest-of-tkloczko/garbage-38afec6b-341d-4413-8d4f-22b5dc5ec1e0
<class 'OSError'>: [Errno 39] Directory not empty: '/tmp/pytest-of-tkloczko/garbage-38afec6b-341d-4413-8d4f-22b5dc5ec1e0'
  warnings.warn(
Task was destroyed but it is pending!
task: <Task pending name='Task-91' coro=<NotebookClient._async_poll_output_msg() done, defined at /home/tkloczko/rpmbuild/BUILD/nbclient-0.6.3/nbclient/client.py:793> wait_for=<Future cancelled>>