-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add automated benchmarking #377
Conversation
52eda59
to
712df95
Compare
712df95
to
e904eeb
Compare
1767d8f
to
d3ffd87
Compare
f20584d
to
1ebee1d
Compare
71328a0
to
006b8b9
Compare
.asv/results/cirun-peytondmurray--versioned-hdf5-ffd0030/52aca7e9-virtualenv-py3.11.json
Show resolved
Hide resolved
@peytondmurray I think it would be incredibly useful to be able to rerun asv on a controlled environment (GCP) on a PR before it gets merged and compare the results. Do I understand correctly however that this PR can exclusively run in master? Would it be a sensible followup to add the functionality to do a comparative benchmark on an open PR (maybe triggered by a github PR tag)? |
Sure, this is definitely possible - especially if you're fine with just a table of benchmark results as output. I can follow up in another PR with a bot that posts benchmarks in a comment, if that makes the most sense. |
Here's the latest version, I merged this PR into my own fork's I think the graphs aren't displaying correctly because there's only one data point (the latest commit comparison), but it could be that |
@peytondmurray it's failing in master:
https://github.com/deshaw/versioned-hdf5/actions/runs/11476804181/job/31937575352 |
You're installing libhdf5 from conda and h5py from pip. Could you try moving h5py to conda too? |
Yep, will do now. |
This PR adds automated benchmarking on GCP via Cirun:
.cirun.yml
configuration. Seems to be working well already 🎉master
. When the docs build, the asv results are rendered into static HTML. A link to these is added to the front page of the docsdelete_version
benchmark; it was taking way too longCloses #343.
This was tested with a pure python version of
build_data_dict
, as the current one is broken on CI and my local machine: https://github.com/peytondmurray/versioned-hdf5/actions/runs/11391825172