Python in the browser as a solution to pickling

[lidavidm]

Pickling is a problem for SymPy Live as it can lead to strange errors (not everything is picklable); running Python in the browser could avoid this. There are three options here:

• CPython via Native Client
  • Pros: full CPython
  • Cons: Chrome only, large size
• CPython via Emscripten
  • Pros: full CPython
  • Cons: slower than CPython, large size, seems to hang/crash browsers (when I tried)
• PyPy via PyPy.js
  • Pros: relatively small (only 32MB JavaScript)
  • Cons: relatively new, buggy (import random leads to an error)

I'm leaving this here mostly to record ideas/options.

[asmeurer]

Another solution is to just use something other than the App Engine that supports a persistent session.

[asmeurer]

Just played with PyPy.js. It will probably also be extremely slow. It's only faster for certain things, and then only if the jit is warmed up. It is also missing a lot of the standard library (e.g., I couldn't import random).

[stefnotch]

It is possible to run Sympy in the browser using Pyodide. I've already attempted that for one of my projects and it works reasonably well.

Another option would be improving Sympy until it can reasonably be executed in the browser using Brython
sympy/sympy#18889

[ivanistheone]

Yes, I've tested sympy in the pyodide shell and it works fine.

Typing import sympy leads to a O(10MB) async download of mpmath and sympy:

After that every command I tested runs fine.

Overall this could be a way forward to remove or reduce the need for a backend for sympy live shell (and docs shell), for devices with good internet connection (where 10MB download is not an issue) and modern browsers (capable of running wasm).

[dynamicwebpaige]

Just tested SymPy with Pyodide, and experienced similar performance / latency to the SymPy Live implementation. It would be awesome to see Pyodide-enabled code samples in SymPy documentation.

[stefnotch]

I agree, this is definitely worth pursuing further! I suppose the absolutely simplest option would be serving a single, static .html file, which loads Pyodide from the CDN. However, that wouldn't be properly integrated into the rest of the project. Maybe someone who has more knowledge here could chip in on what the best way of going forward would be?

[asmeurer]

Moving SymPy Live and SymPy Gamma to Pyodide would be great if it works, and would save us a lot of money.

[ivanistheone]

I'm interested in contributing to and helping out any effort towards a pyodide-backed Gamma and Live.

I've been following this thread and this pull request on the jupyter-book thiebe project, which I hope can be helpful as a "source of inspiration." JupyterLite is another code base worth watching [see exectute method].

[stefnotch]

So, taking another look at the code, if I understand it correctly

• This code along with Sympy has to be executed in Pyodide on start
• The "evaluate" call here has to be replaced with a call to Pyodide
• The "autocomplete" call here should probably also be replaced with an appropriate call to Pyodide

Though, I suppose "replaced" might be the wrong word to use, since it might be worthwhile to keep the current App Engine backend around for now.

[jtpio]

Hi all,

Just landed here from jupyter-book/thebe#465.

sympy seems to be working fine with Pyodide. I haven't tried everything yet but so far it looks good.

Actually we have an example in the JupyterLite notebooks, if you want to try it out in the browser:

https://jupyterlite.readthedocs.io/en/latest/_static/retro/notebooks/?path=pyolite.ipynb

It also renders nicely just like with a regular Jupyter Notebook or with JupyterLab:

As well as in the code consoles:

https://jupyterlite.readthedocs.io/en/latest/_static/retro/consoles/?path=/console-1-018f1666-a731-40f8-90dc-5196e5f6c23b

[asmeurer]

I would expect if we switch to pyiodide the entire backend should be refactored. Right now it tries to persist variables to fake a session, since the App Engine doesn't persist a real session. But with pyiodide we can have a real persistent session on the client's computer.

What I would love to see is a community supported Sphinx extension that lets you run examples in pyiodide, that is not tied to SymPy. That way we can have the same functionality we currently have in our docs, but without the burden of having to maintain it ourselves, which doesn't work so well given that most SymPy developers are not web developers. The same thing with the Live shell. We should look at converting SymPy Gamma to use pyiodide as well.

[eagleoflqj]

I'm interested in contributing to and helping out any effort towards a pyodide-backed Gamma and Live.

Please take a look of the port of SymPy Gamma: https://github.com/eagleoflqj/sympy_beta
I plan to directly copy https://pyodide.org/en/stable/console.html to this project as a live shell

[ivanistheone]

@eagleoflqj This is very impressive work on sympy_beta. The Vue-fication makes the code very easy to read and understand. I suppose a similar approach for the live shell would be good, although we'd need a new "interactive shell" component like you said.

One thing to keep in mind is sympy-live shell is used both as a standalone webapp (https://live.sympy.org/) and also as a component in the docs, example include hook, and src1 src2.

As you will see in src1 src2 there is a lot of extra functionality bundled into the current live shell component, which would be really nice to keep if re-implementing in Vue.

There is also some logic in the Python code for evaluating expressions worth looking into
https://github.com/sympy/sympy-live/blob/master/app/handlers.py#L269-L275
https://github.com/sympy/sympy-live/blob/master/app/shell.py#L308-L321 although the pyodide shift will reduce the complexity A LOT since no more need for picklables and appengine session storage.

@eagleoflqj Could you please setup a barebones shell component (e.g. based on https://pyodide.org/en/stable/console.html ) and share it so we can start experimenting with stuff?

First thing I will expriment is how to run python "middleware" on each exec/eval code, because these might be needed for post-processors like PRINTERS and exception handlers. Also probably for debugging too...

[ivanistheone]

@asmeurer said:

What I would love to see is a community supported Sphinx extension that lets you run examples in pyiodide, that is not tied to SymPy.

What you're describing is very close to this: https://github.com/executablebooks/sphinx-thebe if it had a pyodide backend.

example book, use the Live Code option to enable

thebe abstracts away the python execution "backend" and I'm sure the community will get a pyodide backend going soon enough (we just wait for it; POC code already available).

That way we can have the same functionality we currently have in our docs, but without the burden of having to maintain it ourselves, which doesn't work so well given that most SymPy developers are not web developers.

Thebe backed runnable code in the docs would be a good option in terms of low maintenance, but break with the current UI:

• CURRENT of clicking on individual lines to execute them (which I don't think is too intuitive, but ppl might be used to it).
• NEW each code block is it's own "shell"

If we could have something like a thebe-shell component in additinon to the inline code blocks that would be the best, with sympy-live being powered by the same thebe-shell component.

[eagleoflqj]

@eagleoflqj Could you please setup a barebones shell component (e.g. based on https://pyodide.org/en/stable/console.html ) and share it so we can start experimenting with stuff?

I just took the first step to integrate pyodide shell into sympy beta eagleoflqj/sympy_beta@1ee37be
Currently when you switch between views the shell is reloaded, and if your input reaches the bottom it won't automatically scroll to a good view. I'll try to fix them, and maybe improve it by supporting multiple, minimize-able, restart-able terminal. But feel free to try this version.
As I directly used code from pyodide, this component has to be licensed no permissive than MPL-2.0 (and I choose to keep MPL-2.0). I don't know whether sympy live can benefit from it.

[jtpio]

FYI I opened #198 to switch to using JupyterLite for the REPL on https://live.sympy.org:

JupyterLite comes with the Pyodide (currently at version 0.19.0) based Python kernel, and supports for rich output rendering and IPython.