control for thermal throttling and number of threads

[xhub]

Very nice work on building the datasets and the comparison.

I saw no mention of the control w.r.t the CPU thermal throttling. Unless very efficiently cooled, modern CPU cannot run at max speed for long. This can considerably skew timings. Did you consider this when running the benchmark?

Also, some solver, like gurobi, can make use of multiple threads to speed up computations. I saw no mention of controlling this variable in the repository.

Keep up the good work.

[stephane-caron]

I saw no mention of the control w.r.t the CPU thermal throttling. Unless very efficiently cooled, modern CPU cannot run at max speed for long. This can considerably skew timings. Did you consider this when running the benchmark?

Great point! Currently the benchmark does not monitor this. I drafted a feature request here #88 labeled as https://github.com/qpsolvers/qpsolvers_benchmark/labels/good%20first%20issue (i.e. good way to start contributing to the project, does not require understanding the full pipeline). I don't have the bandwidth to work on this in the short term but I'd be glad to help review and merge a PR for this.

Also, some solver, like gurobi, can make use of multiple threads to speed up computations. I saw no mention of controlling this variable in the repository.

Hmm absolutely, solvers may do things under the hood. The approach of the benchmark, especially in the default settings, is to take solvers as users will get them. (Whenever non-default settings are used they are documented in the results report.) So, if by default Gurobi spins K threads every time it is called, then the benchmark will let it do this.

If it makes sense for some applications we could totally add a single_thread settings controlling for this variable. Again, I'd be happy to help review/merge a PR for this 👍

[xhub]

Thanks for the detailed reply. Sadly, I also have very limited bandwidth.