Blockers for using retrolab in data8

[yuvipanda]

JupyterLab retro is a great alternative to the classic notebook, keeping the simple document oriented UI while giving us new features like RTC. We want to get it to a point where it can be used for data8. This issue will track whatever needs to be fixed for that to happen.

• /tree/<path> should redirect to /notebooks/<path> if <path> is a notebook file jupyterlab/retrolab#153
• display(HTML(...)) For local link doesn't download on single click jupyterlab/retrolab#152
• Add retrolab as an option in nbgitpuller.link - depends on /tree/<path> should redirect to /notebooks/<path> if <path> is a notebook file jupyterlab/retrolab#153
• Simple button to switch back to classic notebook jupyterlab/retrolab#157
• Get everything into a 2.x release
• Install release with all these fixes on our hubs
• Figure out how JS for saving notebooks will work (otter does this)
• Wite a blog post about Data 8 using Retro Lab!
• Create a documentation for launching retro lab (articulate the why for this transition)

/cc @Yanay1 and @ericvd-ucb

[yuvipanda]

jupyterlab/retrolab#159 and jupyterlab/retrolab#157 should fix two of these issues. Once merged, I'd want to backport them to the 2.x release so we can deploy them.

[yuvipanda]

@Yanay1 @ericvd-ucb all the blockers we have identified have been fixed. Can we try giving it a shot again?

[ericvd-ucb]

@Yanay1 has left Berkeley to go across the bay
I think this is a question for Data 8 course staff - eg Wagner or DeNero
@yuvipanda is the point that this is future-proofing strategy?
and if so how urgent is it?

[yuvipanda]

@ericvd-ucb I think I'd like us to use this for the next semester. Perhaps we can have a meeting where we go through some of the workflows but with retrolab to see what else needs fixing? I think we've fixed all the reported issues so far.

[balajialg]

@yuvipanda @ericvd-ucb This is awesome! Any specific reasons to start with Data 8? Was this a prior request from Data 8 staff? I wonder whether it would be a viable strategy to begin testing with a smaller class like CMP BIO (Genomics class) and test how the retro lab functionality works before moving it to Data 8? Thoughts?

[yuvipanda]

@balajialg we could do that as well - but I suspect some of those classes might just want to use JupyterLab straight up instead? I think data8 is the big convert here, as it is teaching retrolab's target audience. We could definitely start smaller, but ideally we won't wait one more full semester before starting data8 out. @fperez has been teaching data100 to 1200 students with JupyterLab, and that already shook out a lot of issues with otter and other systems that data8 also uses (JupyterLab and retrolab share most code)

[ericvd-ucb]

So I guess some of the elements would be

• could we "test it out somewhere"
• we could test it out on Data 88 - 75 students ?
• some percent of Data 8 students?
• make up a set of nbgitpuller links
• probably need to convince DeNero of the need to do this
• Most importantly need to talk to Data 8 course staff who does assignment generation

[fperez]

@yuvipanda thanks for the ping! I'm 💯 on this, and happy to try to move things forward. I had a quick look and just opened a couple of issues in the retro repo for things that stood out to me. I think jupyterlab/retrolab#259 should be added to your blockers list at the top, not so sure about the file manager ones: those might be easier to live with for now, as I don't know how critical the FM is to Data 8 workflows.

[yuvipanda]

That's a good list, @ericvd-ucb. I thought we had already convinced David Wagner of this last semester. I think coupling this with real time collab is perhaps an important aspect to highlight.

If you think that starting with a smaller class is the way to go, let's definitely do that.

@fperez the data8 workflow doesn't use the file manager at all, so shouldn't affect it.

[davidwagner]

For Fall 2021, Melissa Wong and Stephanie Djajadi are the leads of the Data 8 staff team who generate assignments.

[ericvd-ucb]

OK well lets get student staff to try it once more with Data 8 materials
@yuvipanda can you direct us to the retrolab nbgitpuller syntax

[Yanay1]

I made the change to retro lab at the beginning of summer (before reverting back) so I have the paths which I guess I forgot to commit:

demos_base = 'http://data8.datahub.berkeley.edu/hub/user-redirect/git-sync?repo=https://github.com/data-8/materials-su21&urlpath=retro/notebooks/materials-su21'
demos_file = '&subPath=lec/%s'

datahub_base = 'http://data8.datahub.berkeley.edu/hub/user-redirect/git-sync?repo=https://github.com/data-8/materials-su21&urlpath=retro/notebooks/materials-su21'
datahub_path = '<a href="' + datahub_base + '/materials/su21/'

These would replace the existing variables in site/content/pages/calendar.ipynb (have to change su21 to new semester)

[yuvipanda]

Thanks a lot, @Yanay1! Does this help, @ericvd-ucb?

[yuvipanda]

I also opened jupyterhub/nbgitpuller#225 to add retrolab support to the link generator.

[ericvd-ucb]

Hi all. Yep I think I can figure out the syntax from @Yanay1 code. I am guessing this is actually referring to Data8 assignment generation scripts

[yuvipanda]

@ericvd-ucb yeah. for a simple string replace, you can replace urlpath=tree/ with urlpath=retro/tree/ in all current nbgitpuller URLs and they will open with retrolab instead.

[yuvipanda]

@ericvd-ucb nbgitpuller.link now supports opening in retrolab

[yuvipanda]

Looks like otter relies on classic notebook javascript execution with display (https://github.com/ucbds-infra/otter-grader/blob/6a33a7519c8015201989250a1d185d4b03638892/otter/check/utils.py#L18) to help save notebooks before submitting them. This is extremely important functionality, but has already been a bit flaky (been debugging issues with problems saving with @davidwagner). It also won't work with retrolab as it is.

We must find a way to allow python code to ask for the current notebook to be saved. This is a blocker for retrolab / lab adoption I think. I think a more robust solution will also fix the current issues around failure to save.

@jtpio any thoughts on how we can address this?