Getting Started Docs for Non-Developers

[psychemedia]

[Reposted from here by request. Note that this is a personal opinion and the original tone which in part described JupyterLab as a "hostile architecture" was a deliberate parody from a user persona along the lines of a someone who is most definitely not-a developer, but has enough understanding of how to read and modify simple pre-existing, found html/js/css to modify simple web based UIs. Please don't take offence! And apologies for this post being so long - I don't have the time to make it shorter...]

---

Context and motivation (not in the original)

The context is added in part because it helps identify the "just gulf" between a seasoned developer and a novice. "Just do X" is often packed with huge amounts of tacit knowledge and assumptions about pre-existing skills and environments that are meaningless to a novice. And whilst various approaches may be useful for a developer building production code, they aren't actually required if you simply want to create a really naive single feature extension for your own local purposes in a constrained environment.

One of the attractive features of the original IPython notebook was that it provided a relatively low barrier to entry for users trying to make their own simple customisations to the UI via simple extensions. The notebook provided a way to introduce not-coders to code, and the UI provided a way for e.g. instructors who might previously have tried to build a simple html'n'javascript application to support their students to have a go at building widgets they could render in a notebook, or customising the notebook UI itself.

The original extensions were relatively simple in terms of design: a simple Python package that registered the extension, which itself often amounted to nothing more than a single simple javascript file.

By inspection of simple pre-existing extensions, the complexity was low enough to allow someone with quite naive python and web development skills (basic HTML, js and css) to figure out how to modify the extension to create their own extension to customise the UI or add a feature into a notebook. Development could be direct: edit a .js file in text editor, run pip install . and then reload their browser to see if it worked, using browser tools and console.log debug statements to fix it. Naive but accessible. You could even make progress by editing files in a GitHub repo using the Githib file editing tools and then user MyBinder to build and test the result.

By contrast, getting started with JupyterLab appears to require an understanding of development environments, typescript, node, npm, toml files, poetry, locks of various sorts, build scripts and build processes, a good understanding of the JupyterLab framework, hooks, promises, asyncs and awaits. A long way from writing a simple js script to make a simple customisation of the html/css UI.

It maybe it doesn't require all or even many of the .toml, .poetry, .ts, .numerous_hidden files that appear in a typical JupyterLab extension repo to actually create a JupyterLab extension. But for a novice, looking at the repo for a way in to get started with their own extension, there is no way to tell. There are too many decisions to make about what file to look in and which file relates to the extension rather than the packaging or the build process. In many cases, a novice may not be able to identify user-editable files from files that are automatically generated. They are not and will never be a developer. That is not their job, They are promoter users of the environment trying to hack a simple customisation to make the environment and or the materials rendered in it more effective for their users.

If have-a-go not-developers are a potential user group for making their own very simple custom extensions, at least as a starting point, it would be really useful if there were an on-ramp for not-developers that followed an incremental, constructive approach to building a minimum viable extension.

Writing tutorials for novices is really hard. They have to be self-contained and not miss out any steps. They have to assume zero experience and zero pre-installed tools. Rather than providing a cookie-cutter that dumps a load of unexplained files onto someone's desktop and a tutorial that tells them to "just" do voodoo then build and install the final extension, the tutorial for a novice user introduces only essential files a line at a time explaining what each file is and why it is necessary. There is no pre-understanding to draw on.

I would try to write such a tutorial myself, but I can't find a way in to making sense of what's going for myself (time poor, not a developer; I don't know a toml from a typescript and rely on View Source from things that run in my browser to make sense of what's going on. If I did spend time learning developer skills, it would probably be motivated by building things for a more general environment such as VS Code. But that isn't going to happen either!;-)

---

From the docs, the getting started is to install a development environment, and I’m already lost. When ever I try to use node it seems to download the internet and the instructions typically say “build the package” without saying what words of incantation I need to type into the command line to actually build the package (why should I just know how to do that?)

The next step is to install a cookiecutter. This doesn’t necessarily help because I have no idea what all the files are for, whether they or necessary, or what changes can be made to each one to perfrom a particular task. I’d rather be interested to a minimally viable set of files one at a time with an explanation of what each one does and containing nothing uncommented that is not essential. (Some “useful but optional” fragments may also be handy so I can uncomment them and try them out to see what they do, but not too many.)

When it comes to trying out some example code, I need to learn a new language, .ts (which is to say, TypeScript). I have no idea what TypeScript is or how to run it.

I also need to import load of things from @ things, whatever they are. If I’m trying to figure out how to do a thing by cribbing code from seeing what files are are loaded to support a working extension in my browser (which I suspect is way harder to do with JupyterLab than it was from classic notebook) I’m not sure if there’s an obvious way to re-engineer the TypeScript code from the Javascript in the browser that does something like what I want to do.

I’m not totally sure what “locate the extension“ means? Is that something I have to do or is it something the code exampe is doing? (I am getting less than rational at this point because I already know that I am at the point of blindly clicking run at things I don’t understand.)

Before we can try out the extension, it needs building. In the classic notebook extensions I could simply install a package, but now I need to build one:

This is a step back to the old, pre-REPL days because there is a level of indirection here: I don’t get to try the code I’ve written, I have to convert it to somethig else. When it doesn’t work, where did I go wrong?

• with the logic?
• with the javascript?
• with the typescript?
• with the build process?
• can the TypeScript be “right” and the javascript “wrong”? I have no idea…

I think things have improved with JupyterLab now that installing an extension doesn’t require an length delay as the JupyterLab environment rebuilds itself (which was a block in itself in earlier days).

Okay, so skimming the docs doesn’t give me a sense that I’d be able to do anything other than follow the steps, click the buttons and create the example extension.

How about checking a repo to see if I can make sense of a pre-existing extension that does something close to what I what?

The classic notebook collapsible headings extension allows you to click on a heading and collapse all the cells beneath it to the next heading of the same or higher level. It works by setting a piece of metadata on the cell containing the heading you want to collapse. A community contributed extension does the same thing, but uses a different tag, "heading_collapsed": "true" rather than "heading_collapsed": true (related issue). Either that or the JupyterLab extension is broken for some other reason.

Here’s what the repo looks like (again, I’m not intending to mock or attack the repo creator, this is just a typical, parodied, example):

Based on nothing at all except my own personal prejudices, I reckon every file halves the number of folk who think they can make sense of what’s going on… (prove me wrong ;-)

Here’s the Jupyter classic notebook extension repo:

So what do I see as the “hostile architecture” elements?

• there is not an immediately obvious direct way to write some javascript, install it into JupyterLab and check the extension works;
• in many example repos, a lot of the files relate to packaging the project; for a novice, it’s not clear what the extension files are, what the exntension files are, and whether all the project files are necessary
  using TypeScript introduces a level of indirection: the user is now developing for JupyterLab, not for the end user environment they can view source from in the browser. (I think this is something I hadn’t articulated to myself before: in classic notebook extensions, you hack the final code; in JupyterLab, you write code in application land, and magic voodoo converts it to things that run in the browser.)
• in developing for JupyterLab, you need to know what bit o f Juyterlab to hook into. There’s a lot of hooks, and it’s not clear how a (non-developer) novice can find the ones they need to hook into, let alone how to use them.

And finally:

• there isn’t a “constructive” tutorial that builds up a minimally viable extension from a blank sheet an explained step at a time.
  As I recall from years and years ago, if you ever see or hear a developer say “just”, or you can add a silent “just” to a statement ((just) build the node thing) you know the explanation is far from complete and is not followable.

Faced with these challenges of having to step up and go and do a developer course, learn about project tools, pick up TypeScript, andd try to familiarise myself with a complex application framework, I would probably opt to learn how to develop a VS Code extension on the grounds that the application is more general, runs by default as an desktop application rather than browser accessed service, has increasingly rich support for Jupyter notebooks, and has a wide range of other extensions to use, and crib from, and that can be easily discovered from the VS Code extensions marketplace.

I think followable is missing term in the reproducibility lexicon, in both a weak sense and a strong sense. In the weak sense, if you follow the instructions, does it work? In a strong sense, if you follow the instructions, does the reader come away feeling that they could create their own extension.

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[krassowski]

I wonder if ipylab (https://github.com/jtpio/ipylab) could be an answer for needs of non-programmers who only know Python and would like to customize JupyterLab behaviour without building a full-blown extension (which indeed requires following the strict rules that were established to enforce good quality and security - but also resulted in higher entry barrier). Any thoughts on this one @psychemedia, @jtpio? I started recommending it some time ago as it seems as the nicest solution for many common problems and has a very clean API.

[krassowski]

Maybe there could be an irlab and ijulab in the future as well?

[psychemedia]

@krassowski I'm not really familiar with ipylab (will take a look); from a quick glance it seems to offer JupyterLab automation / customisation from a notebook script which could be interesting for some use cases.

This also reminds me of JupyterLab workspaces, which I always thought were a hugely attractive feature of JupyterLab (eg in terms of an instructor being able to set up a customised visual workspace for a particular activity rather than expecting a novice to try to figure out how to layout multiple panels in a sensible way, for example) but always seemed lacking in documentation and evangelism/promotion?

[jtpio]

Right, the purpose of ipylab was to make it easier for folks to have control over their UI, in Python. But it can also be used as a limited playground for learning the JupyterLab API and know what is possible to do with it.

has a very clean API.

It's pretty much a subset of JupyterLab API but exposed in Python. Ideally we wouldn't need ipylab but instead have Python bindings to be able to develop proper lab extensions but in Python. Hopefully this would also lower the barrier to entry and fit more into the Jupyter ecosystem.

In the near term future, we should however be able to implement some kind of editor built into JupyterLab, that would use the federated extension mechanism to be able to write extension in plain JavaScript and in JupyterLab, and have them run on the fly. We might need something like #8866 to land first.

---

This is also similar to the idea of having a Theme Editor built into JupyterLab: #3928

For themes it makes even more sense since it should only be CSS, and can be frustrating to go through the whole extension development workflow just for that.

[psychemedia]

@jtpio Ah, sounds good: so a playground on the one hand, and a scripting environment on the other. Just in terms of Duplo style building blocks rather than Lego Technic, have you also seen https://github.com/AaronWatters/jp_proxy_widget which can be used to wrap off-the-shelf js apps in an ipywidget shell (not sure if it still works as JupyterLab / ipywidgets update etc)? Along with https://github.com/jupyter-widgets/jupyterlab-sidecar and https://jupyter-server-proxy.readthedocs.io/en/latest/ this a provides a simple way of getting things into a Jupyter UI context.

[jasongrout]

FYI, @wolfv and I iterated on a project a while ago to develop JLab extensions live in JupyterLab, with the idea of lowering the barrier for customizing JupyterLab and quickly iterating on extension code:

https://github.com/wolfv/jupyterlab-dynext

[jasongrout]

I've updated the jupyterlab-dynext extension for jlab 3 (jupyterlab/jupyterlab-plugin-playground#3), and posted a short screenshot video of it in action at jupyterlab/jupyterlab-plugin-playground#3 (comment).

@psychemedia - I'm curious what you think about how easy this extension makes it to experiment with extensions inside JupyterLab:

[jtpio]

Rather than providing a cookie-cutter that dumps a load of unexplained files onto someone's desktop and a tutorial that tells them to "just" do voodoo then build and install the final extension, the tutorial for a novice user introduces only essential files a line at a time explaining what each file is and why it is necessary

This doesn’t necessarily help because I have no idea what all the files are for

To address these points, maybe the JavaScript cookiecutter could be simplified to only include the minimal amount of files?

And we continue providing two cookiecutters, with slightly different goals:

• TypeScript cookiecutter: full-blown cookiecutter with all the files like it is now (to ensure best practices)
• JavaScript cookiecutter: only the files required to develop the prebuilt extension. The initial script could do the symlinking like when developing notebook extensions. Then ideally editing the JavaScript source would not need a new rebuild of the webpack federated module (to be confirmed)

[jtpio]

Linking to a similar discussion for reference: jupyterlab/extension-cookiecutter-js#36

[psychemedia]

Here's another example (scroll down) of a really simple pattern, for one way communication of data from py state into a js environment that uses the passed data to render an object in a notebook: https://blog.ouseful.info/2021/09/30/a-simple-pattern-for-embedding-third-party-javascript-generated-graphics-in-jupyter-notebools/

It's essentially just saving a templated html page to disk then loading it into an iframe. No dev tools etc required.

(@jasongrout Not had chance to play with dynext yet; will be keen to see how far half-an-hour with ignorance as a starting point gets me...!;-)

[psychemedia]

Looking at JupyerLab docs I realise a significant point of possible confusion relating to scope.

TL:DR for this post: a lot of educational end user customisations don't go any further than tweaking the notebook UI, and maybe introspecting around it. In many cases, the desired effect can be achieved simply by hacking HTML into a markdown cell (at least until notebook updates break div styles in markdown cells which can totally wreck a classroom activity).

Confusion point 2: for educator, think: primary school teacher customising notebooks for use teaching elementary programming to 10 year olds; slightly more advanced would be a secondary school teacher teaching 14 year old kids programming. As part of a National Curriculum. Which means many of the kids aren't interested and the teacher is just trying to make the activities fun.

A lot of the customisations that educators work with focus on modifying the look of a single notebook, even if this just means using the notebook largely as a simple HTML publishing environment with the occasional bit of code.

Extensions in that context have a quite limited meaning eg corresponding to many of the UI tweaks that might be found in ipython-contrib/jupyter_contrib_nbextensions.

Some of those extensions may add toolbar functions, some include config settings (in a yml file) that are handled via the Jupyter-contrib/jupyter_nbextensions_configurator (and that offers a relatively simple interface for accessing updated settings from the configurator in the extension code).

Other extension types include things like jupyter-server-proxy where essentially get to launch a web app via button on the notebook homepage (or JupyterLab launcher) and then view it down a path on the notebook server and there are cribbable examples of those if you want to create your own.

The jupyterlab/extension-examples seem to be more about plumbing examples but it's hard to see how those, or the JupyterLab "notebook" object docs, directly relate to notebook minutiae such as:

• "how do something to a single selected notebook cell, a set of selected notebook cells from a toolbar button?"; or
• "how do I iterate through all cells and update their outer html (eg changing class tags to modify style) when a notebook is opened?"; or
• "how do I access and modify cell content?"; or
• (from Python) "how do I access the content of the previous cell in the notebook (classic notebook had a handy javascript object in the form of Jupyter.notebook(with eg .get_cell_elements() and get_cell() methods) that could be used to do all sorts of notebook related introspection manipulations, often in conjunction with an IPython.notebook.kernel.execute() call to set state back in the Python environment. (I appreciate these may be hacky and "informal", and may have wider secutiry concerns, but in a small, local, trusted environment, they often work well enough to achieve a particular effect).

[jtpio]

FYI @psychemedia the latest release of ipylab should now be compatible with RetroLab: https://github.com/jtpio/ipylab/releases/tag/v0.5.2

And can be tested on Binder: https://mybinder.org/v2/gh/jtpio/ipylab/stable?urlpath=retro/notebooks/examples/commands.ipynb

Not everything will work (for example left and right areas), but this can still provide an example of a Python <-> JS bridge for JupyterLab based applications.