Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support binary data in comm messages via buffers #3974

Closed
seeM opened this issue Jul 11, 2024 · 2 comments
Closed

Support binary data in comm messages via buffers #3974

seeM opened this issue Jul 11, 2024 · 2 comments
Assignees
@seeM
Labels
area: runtimes Issues related to Language Runtimes area: widgets Issues related to Widgets. enhancement New feature or request theme: app builder
Milestone
2024.09.0 Pre-Rel...

Comments

@seeM
Copy link
Contributor

seeM commented Jul 11, 2024

I've always wondered what the buffers key was used for. It's explained really well in this comment: jupyter-widgets/ipywidgets#2378 (comment). In summary:

We use buffers to encode binary data from widgets

Here's a simple example that (I think) requires buffers to work:

import ipywidgets
with open('image.png', 'rb') as f: image = f.read()  # you'll need an actual image file
display(ipywidgets.Image(value=image, format='png'))
@seeM seeM added enhancement New feature or request area: runtimes Issues related to Language Runtimes labels Jul 11, 2024
@seeM seeM changed the title Supports binary data in comm messages via buffers Support binary data in comm messages via buffers Jul 11, 2024
@juliasilge juliasilge added this to the Release Candidate milestone Jul 15, 2024
@seeM seeM mentioned this issue Jul 30, 2024
Merged
@seeM seeM self-assigned this Jul 30, 2024
@seeM
Copy link
Contributor Author

seeM commented Jul 30, 2024 •
edited
Loading

This is needed for the following widget libraries, so this is a pre-requisite for #1924.

  • bqplot
  • ipycanvas
  • ipydatagrid
  • ipympl (i.e. Matplotlib's widget backend)
  • pythreejs

@seeM seeM added theme: app builder area: widgets Issues related to Widgets. labels Jul 30, 2024
seeM added a commit that referenced this issue Jul 30, 2024
@seeM
Fully interactive IPyWidgets (#4172)
ce4c6b5 
This PR adds support for fully interactive IPyWidgets, addressing #3276.

For example, create and display a widget:

```python
import ipywidgets
w = ipywidgets.IntSlider()
display(w)
```

Interact with the widget in the UI, then check its value in the console:

```python
w.value
```

Set the value in the console:

```python
w.value = 13
```

Note the updated UI.

---

We've already been using VSCode notebook renderers to render interactive
plots in the Plots pane. This PR adds a notebook renderer for
IPyWidgets, and routes messages between the renderer (widget frontend)
and the runtime (widget backend) via the updated Positron IPyWidgets
service.

### Unit Tests

The new implementation of the IPyWidgets service is unit tested. This PR
adds a few mocks to make that possible. Most useful are
`TestLanguageRuntimeSession ` and `TestRuntimeClientInstance`, although
they're only implemented as far as needed for this PR.

I used the convention of starting Positron test suites with `Positron -
`. There's also a new script to run only Positron's unit tests excluding
VSCode's:

```sh
./scripts/test-positron.sh
```

Note that this script is not included in our CI since we already run
`./scripts/test.sh` which is a superset.

### QA Notes

These widgets/libraries should work:

* All of `ipywidgets` except `Output` and `Image`
* `ipytree`
* `ipyleaflet`
* `anywidget`
* `drawdata`
* `pythreejs`

These widgets/libraries will not yet work:

* `bqplot`
* `ipycanvas`
* `ipydatagrid`
* `ipympl`

Most of the remaining widgets/libraries require us to support `buffers`
in the Jupyter messaging protocol (#3974) which I intend to work on
next.
@seeM seeM mentioned this issue Aug 1, 2024
Merged
seeM added a commit that referenced this issue Aug 5, 2024
@seeM
Support binary data in comm messages via buffers (#4208)
c8e3bb9 
Addresses #3974 and
#4173.

This PR plumbs `buffers` from jupyter-adapter messages through to the
main thread and specifically to our IPyWidgets notebook renderer. This
unlocks a bunch of widget types (thanks to @isabelizimm for these
examples):

#### `ipywidgets.Image`

```python
import ipywidgets
with open('image.png', 'rb') as f: image = f.read()  # you'll need an actual image path
display(ipywidgets.Image(value=image, format='png'))
```

#### `ipympl` (matplotlib's interactive backend)

Note: Save doesn't work yet. We'll need to handle the `clicked-data-url`
webview message in our `NotebookOutputWebview`.

```python
%matplotlib ipympl
import matplotlib.pyplot as plt
import numpy as np

fig, ax = plt.subplots()
x = np.linspace(0, 2*np.pi, 100)
y = np.sin(3*x)
ax.plot(x, y)
```

#### `ipydatagrid`

```python
import pandas as pd
from ipydatagrid import DataGrid
data = pd.DataFrame({"A": [1, 2, 3], "B": [4, 5, 6]}, index=["One", "Two", "Three"])
w = DataGrid(data, selection_mode="cell", editable=True)
display(w)
# Now change the value of a cell, then run:
w.data
# It should reflect the updated value.
# The next line should update the value in the widget frontend:
w.set_cell_value("A", "One", 3)
```

#### `bqplot`

There appears to be a console warning here, we can address that in a
follow-up.

```py
import bqplot.pyplot as bplt
import numpy as np

x = np.linspace(-10, 10, 100)
y = np.sin(x)
axes_opts = {"x": {"label": "X"}, "y": {"label": "Y"}}

fig = bplt.figure(title="Line Chart")
line = bplt.plot(
    x=x, y=y, axes_options=axes_opts
)
bplt.show()
```
@testlabauto
Copy link
Contributor

Verified Fixed

Positron Version(s) : 2024.08.0-24
OS Version          : OSX

Test scenario(s)

Works in console:

import ipywidgets
with open('/Users/christophermead/Desktop/cmeadPhoto.jpg', 'rb') as f: image = f.read() # you'll need an actual image file
display(ipywidgets.Image(value=image, format='png'))

Link(s) to TestRail test cases run or created:
N/A

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 24, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@seeM seeM

Labels
area: runtimes Issues related to Language Runtimes area: widgets Issues related to Widgets. enhancement New feature or request theme: app builder
Projects
None yet
Milestone
2024.09.0 Pre-Release
Development

No branches or pull requests
3 participants
@seeM @juliasilge @testlabauto