PERF: make vbench runnable in python 3

[jreback]

I believe this is possible ATM.

[jorisvandenbossche]

I think we should also consider in looking into asv (airspeed velocity), if it would be possible to replace our current vbench suite to that (issue for that is: #8361).

Airspeed velocity is actively developed and maintained, other projects like scipy did also start using it, and it has some nice extra features (eg memory benchmarking, interactive web frontend). So maybe it is more worth the effort looking into that, than trying to improve vbench.

Apparently @qwhelan did already some work on this: airspeed-velocity/asv#194
@qwhelan, maybe you can share some thoughts about this?

[rockg]

I agree. I figured I would see how much work it would be to get vbench working in Python3 and then go from there. It was a few simple changes on the vbench side and definitely the harder part is going to be getting the benchmarks compatible. However, this would probably need to be done anyways for asv so I figured it was worth proceeding with vbench. The suite can be run in Python3 now with my vbench changes (note that I didn't do a full test of plotting, but nothing there seemed to stand out as an issue).

[qwhelan]

@jorisvandenbossche Definitely have some comments but I'll have to get back to you guys this weekend. They've added a lot of support for vbench features I thought were significant but I haven't had a chance in the last few weeks to see what's fully/partially/not at all supported.

[qwhelan]

@jorisvandenbossche I have a short script that converts vbench Benchmarks into asv ones with a little bit of ast use. There's a bit of information being lost (at least comments and start dates) but I don't see any issues that would prevent us from having things in parallel for a bit. I'm still working through some compilation issues that I'm fairly sure are OSX-specific, but I do have a subset of our vbench cases running under asv via my translation script. If I don't run into any issues, I'll upload a cleaned-up version tomorrow night.

[jorisvandenbossche]

@qwhelan And what about some of the other things you noted: is it possible to keep the benchmarks in the pandas repo or does it have to be a separate repo? And are the similar abilities to our test_perf.sh script?

[jorisvandenbossche]

And BTW, thanks a lot for working on this!

[qwhelan]

@jorisvandenbossche They've changed it so everything can be stored in the same repo. Also, the asv run and asv compare commands should be able to mostly replace test_perf.py, though there may be some specific commands missing. Notably, it doesn't have the daily sampling vbench does - it will just uniformly slice the commit range based on the number of commits.

[jorisvandenbossche]

good to hear! Just open a PR with what you have, that is the easiest to try out/give feedback

[jorisvandenbossche]

Closing as it is probably not worth to put more work in vbench if we are going to use asv (see #9715)