Running JupyterLab alongside the Cylc UI

[dpmatthews]

This is possible now that the uiserver is a jupyter server extension (#230).
We should:

• document how to do this
  • jupyter: document how to add Cylc to an existing setup cylc-doc#628
• explain how to choose whether you start with JupyterLab or the Cylc UI on launch
  • jupyter: document how to add Cylc to an existing setup cylc-doc#628
• investigate how to support switching between the 2 applications (e.g. adding a link in the Cylc UI)
  • JupyterServer: easier switching between extensions #470
• investigate implications for multi-user support
  • cylc hub: document multi-user config post jupyter server v2 upgrade cylc-doc#625

[jbaksta]

To get started with this, I suppose we did a couple things here at NCAR to facilitate accessing the Cylc-UI through our current JupyterHub deployment.

• The first one was to get the Cylc Extension registered with Jupyter server. This wasn't complicated, dropped a file in $PREFIX/etc/jupyter/jupyter_server_config.d.
• The next was to create a basic front-end extension to JupyterLab for easy navigation to the Cylc-UI.

One thing that I noticed on the setup that I'll have to address at our site is the hub button in the Cylc UI doesn't reference the hub when behind a reverse proxy. I'm sure I'll tweak that in the near future, but but nevertheless, it's there.

I'm also not sure about the issue regarding multiple users, but most of our users will not access other users Cylc workflows.

I think my next thing will be to put the Cyl-UI inside of a JupyterLab Launcher Tab instead of a whole new window. Many of our users will want to watch the GUI while editing their flows in the same space. This would allow such workflow.

[oliver-sanders]

I've created a dedicated issue for the switching of Jupyter Server extensions (e.g. switching between Jupyter Lab and Cylc UI Server) - #470

Jupyter Server can list extensions and extract the root endpoint from them, however, this might be a bit clunky (can't remember) so it might be worth pushing something upstream as these toggles need to be implemented in every server extension separately. We may want to go both ways e.g. Cylc => Lab or Lab => Cylc and there are other extensions user's might want to toggle between so it would be good to come up with a stanardish way of doing this.

[oliver-sanders]

The multi-user support side of things is now covered by - cylc/cylc-doc#625

[oliver-sanders]

The remainder is documentation, opened a cylc-doc issue - cylc/cylc-doc#628

[oliver-sanders]

I think my next thing will be to put the Cyl-UI inside of a JupyterLab Launcher Tab instead of a whole new window. Many of our users will want to watch the GUI while editing their flows in the same space. This would allow such workflow.

Cylc has some endpoints dedicated to particular views to make this easier e.g. /cylc/graph/<workflow-id>. If you've made any progress we would be interested in hearing it.

Long term we would like to consider better ways to blend the Cylc app with Jupyter Lab. In theory, both use the same tab management system and the same server framework, so popping up a tab from one in the other should be possible. At least, breaking down the components of the Cylc app into Lab extensions should be possible. However, there are some substantial barriers to cross so we aren't likely to get onto this in the short term:

• Jupyter Lab uses React, Cylc uses Vue.
• Cylc uses a global data store for efficiency (avoids having multiple overlapping subscriptions). We will need to find a way to reproduce this patten in the Jupyter Lab application.
• Cylc has separate authorisation to Jupyter Lab (i.e. a user might have read permissions to another user's Cylc app, but not to their Jupyter Lab instance).

[oliver-sanders]

Also opened an issue covering the sharing of context between server extensions - #472

[oliver-sanders]

The topics on this issue should now all be covered by specific ones so I'll close this umbrella issue.

If anyone comes across this issue and has more to suggest, feel free reach out to us on Discourse or our developer chat on Element (links in the README).

We're open to contributions if you have any code you'd like to share.