Feature/automated panelite tests

[MarcSkovMadsen]

Going through the notebooks of Panelite to identify which notebooks which to piplite.install what. And which notebooks cannot run without errors has been a pain.

I would like to automate the process of testing the Panelite notebooks for errors. This is an attempt.

• Open Panelite (either online or locally)
• Navigate to notebook
• Run all cells
• Determine if an exception is raised
• [x ] Retry up to 3 times if "Run All" never finishes. But fail if an exception is raised.
• Minimize flakyness of tests
• Maximize speed of tests
• Enable to test new build of Panelite (how do I build panelite in Panel?)
• Functionality to loop through list of relevant notebooks.
• Make sure 100% of tests pass

Works like below (2x speed)

panelite-tests_PKe2RSMg.mp4

[codecov]

Codecov Report

Merging #4655 (ff9ac4b) into main (873fcb6) will decrease coverage by 9.95%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #4655      +/-   ##
==========================================
- Coverage   83.23%   73.28%   -9.95%     
==========================================
  Files         266      266              
  Lines       37423    37423              
==========================================
- Hits        31148    27427    -3721     
- Misses       6275     9996    +3721     

Flag	Coverage Δ
ui-tests	?
unitexamples-tests	73.28% <ø> (-0.01%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 49 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[MarcSkovMadsen]

In order to be able to test the main branch or something close I need the panel and bokeh wheel files for Panelite.

Where do I find the ones I can use for testing the main branch or the latest beta or release candidate @philippjfr ?

Note: pip install jupyterlite should be mentioned in the above guide.

[MarcSkovMadsen]

I've tried building Panelite/ Jupyterlite. But unfortunately it fails. I've reported the issue here jupyterlite/jupyterlite#1055.

[MarcSkovMadsen]

Ok a fresh virtual environment solved my issue and I can build Panelite.

But when I serve it using python -m http.server it does not show any notebook files ??

[philippjfr]

This is great btw but should probably be part of the regular UI test suite (or run as a separate job) rather than being a script.

[MarcSkovMadsen]

Lets get it working and then we can move it around.

But it take some time to run, so I 'm not sure you want to run it always.

[MarcSkovMadsen]

I'm now adding the dependencies for the new notebooks. I can see that you (@philippjfr) are very often reading .parquet files. This is great for data science. But Its not great for Panelite as you cannot read parquet files via pandas.read_parquet 😄

So all those notebooks won't work.

See apache/arrow#34996

[MarcSkovMadsen]

FYI. @philippjfr . I've updated the Panelite file generation script to list the notebooks without dependencies defined.

This way the dictionary of notebooks/ dependencies will be easy to maintain.

[maximlt]

Seems like fastparquet is available on pyodide pyodide/pyodide#3590

[MarcSkovMadsen]

fastparquet seems to work.

I can't seem to figure out if pyodide-http is needed. I read somewhere its now a part of Pyodide. I also think I've read somewhere its built into Panelite. But for the gallery/featured/nyc_deckgl.ipynb notebook I've had to import it specifically for it to work ???? Without it I get the error

[philippjfr]

Note that I've completely overhauled the gallery and may be re-adding specific examples I delete back to how_to examples. I'll also add a script to turn the how-to markdown files into notebooks. So there's a lot of churn that will have to happen here, sorry about that.

[MarcSkovMadsen]

I've added an info note to the Panelite notebooks. This improves the user experience significantly as you know what to expect.

[philippjfr]

I've added an info note to the Panelite notebooks. This improves the user experience significantly as you know what to expect.

Yep, that's great!

[philippjfr]

I'm going to merge. I must admit though I haven't actually run your tests. Will work on that in another PR.