How to handle performance showcasing in the future

[HealthyPear]

UPDATE

• created https://gitlab.cta-observatory.org/mperesano/protopipe-results
• PR Add new script for benchmarking #157 + PR Add book template material #174
• PR Update documentation for release 0.5.0 #175 removes performance showcasing from documentation

Current situation

Right now performance (up to DL3) is stored under protopipe/docs/contribute/benchmarks with DL3 being uploaded here (only poster plot though)

Of course, with more than 1 analysis this becomes not ideal...

What to expect around v0.5.0

I am also testing a nice way to show all benchmarking using Jupyter Book for which I will most likely open a PR soon (just to allow for a first raw solution).

In general, I would say that it won't be ideal to store all the benchmarking material (the executed notebooks) under the documentation in the future (I will most likely do it in v0.5.0, but I hope in the future we choose a different solution).

A possible solution for the future

A possible solution would be to open a repo only for the storage of performances (maybe already on under the DPPS Gitlab project?).

This repo would have a structure like so:

- protopipe version
 - Production
  - Site
   -  NSB level
    - Zenith angle
     - Azimuth direction
      - source type

Inside each folder, there would be

• the material required to showcase an HTML version of the book (a folder containing the benchmarking notebooks and the book material)
• all the necessary to retrieve/reproduce the results (a Docker recipe, an analysis metadata file produced by the interface or similar).

This repository would have it's own "documentation" in which we link to the different books/analyses with external links to get the data or reproduce it (actually after looking a bit more into it, I think it would make sense to make the repo itself a book and each part is 1 analysis...but we'll see).

[kosack]

Sounds reasonable! Let me know if you would like me to create a repo for it