Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] Add nbresuse==0.3.3 (full freeze.py) #904

Merged
merged 2 commits into from
Jun 4, 2020

Conversation

manics
Copy link
Member

@manics manics commented Jun 2, 2020

Adds nbresuse==0.3.3, followed by running freeze.py

JupyterLab automatically displays the metrics provided by nbresuse, it doesn't need a separate extension. Note that as mentioned in jupyterhub/binderhub#1097 (comment) there are alternative JupyterLab front-ends for nbresuse, for instance https://github.com/jtpio/jupyterlab-system-monitor

Closes jupyterhub/binderhub#1097

Screenshots from testing locally with docker run -e MEM_LIMIT=200000000 ...

Notebook:
image

Lab:
image

@manics manics changed the title Add nbresuse==0.3.3 (full freeze.py) [MRG] Add nbresuse==0.3.3 (full freeze.py) Jun 2, 2020
@betatim
Copy link
Member

betatim commented Jun 2, 2020

I think this will be a useful addition. Going to wait and see if someone else has an opinion.


On a meta level it makes me wonder if we should start thinking about "repo2docker" and "repo2docker binderhub style". The latter would be r2d + all the custom extensions and packages we are adding that are useful when running in a BinderHub. Or maybe it isn't needed as most people use repo2docker via a BinderHub any way.

@manics
Copy link
Member Author

manics commented Jun 3, 2020

It looks like Travis is failing on Python 2.7:

+ /srv/conda/envs/kernel/bin/ipython kernel install --prefix /srv/conda/envs/notebook
Traceback (most recent call last):
  File "/srv/conda/envs/kernel/bin/ipython", line 7, in <module>
    from IPython import start_ipython
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/IPython/__init__.py", line 49, in <module>
    from .terminal.embed import embed
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/IPython/terminal/embed.py", line 18, in <module>
    from IPython.terminal.interactiveshell import TerminalInteractiveShell
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/IPython/terminal/interactiveshell.py", line 16, in <module>
    from prompt_toolkit.document import Document
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/prompt_toolkit/__init__.py", line 16, in <module>
    from .interface import CommandLineInterface
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/prompt_toolkit/interface.py", line 19, in <module>
    from .application import Application, AbortAction
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/prompt_toolkit/application.py", line 3, in <module>
    from .buffer import Buffer, AcceptAction
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/prompt_toolkit/buffer.py", line 7, in <module>
    from .auto_suggest import AutoSuggest
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/prompt_toolkit/auto_suggest.py", line 13, in <module>
    from .filters import to_cli_filter
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/prompt_toolkit/filters/__init__.py", line 33, in <module>
    from .base import *
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/prompt_toolkit/filters/base.py", line 5, in <module>
    from prompt_toolkit.utils import test_callable_args
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/prompt_toolkit/utils.py", line 9, in <module>
    from wcwidth import wcwidth
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/wcwidth/__init__.py", line 12, in <module>
    from .wcwidth import ZERO_WIDTH  # noqa
  File "/srv/conda/envs/kernel/lib/python2.7/site-packages/wcwidth/wcwidth.py", line 82, in <module>
    from backports.functools_lru_cache import lru_cache
ImportError: No module named functools_lru_cache
Removing intermediate container e2a8fd2ebbd3
The command '/bin/sh -c bash /tmp/install-miniforge.bash && rm /tmp/install-miniforge.bash /tmp/environment.yml' returned a non-zero code: 1

which may be due to the change to wcwidth:

-  - wcwidth=0.1.8=py_0
+  - wcwidth=0.2.3=pyh9f0ad1d_0

Upstream diff: jquast/wcwidth@0.1.8...0.2.3
How should I deal with this?

@manics
Copy link
Member Author

manics commented Jun 3, 2020

For comparison I've also got a separate branch with a minimal commit that only adds nbresuse and psutil, this passes Travis: manics@f53c291

@betatim
Copy link
Member

betatim commented Jun 3, 2020

Seems like a bug in wcwidth no? Can we pin to a lower version that still works?

Later versions of wcwidth require `backports.functools-lru-cache>=1.2.1; python_version < "3.2"` but the conda-forge wcwidth package was automatically bumped without this new requirement.
@manics
Copy link
Member Author

manics commented Jun 4, 2020

I think it's a bug in the conda-forge wcwidth package (upstream added a new dependency for python<3.2 which is missing from the conda package), I've pinned it for Python 2.7 only.

The remaining CI failure looks unrelated.

@betatim
Copy link
Member

betatim commented Jun 4, 2020

Restarted the CI. It seems sometimes the "does the memory limit work" tests fail. It seems like some of the nodes our tests run on support it and some don't :-/

@betatim betatim merged commit ebf973a into jupyterhub:master Jun 4, 2020
@manics manics deleted the nbresuse-freeze-all branch June 4, 2020 09:09
@bitnik
Copy link
Contributor

bitnik commented Jun 10, 2020

@manics thanks for adding this! may I ask why you didnt add v0.3.4 instead? I mead, did you simply miss it or is there a specific reason? I am just curious :)

@bitnik
Copy link
Contributor

bitnik commented Jun 10, 2020

okay, v0.3.4 has a problem with jupyterlab: jupyter-server/jupyter-resource-usage#36

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

UI to show resource usage (e.g. CPU, memory) if limits are set
3 participants