optuna is failing

[crusaderky]

tests/workflows/test_pytorch_optuna.py::test_hpo is failing with an obscure CommClosedError / CancelledError:

https://github.com/coiled/benchmarks/actions/runs/8362787642/job/22894238513

as this test only runs by enabling it explicitly, I'm unsure when it started failing.
I've recently upgraded to optuna 3.6.0 but the issue was already showing in 3.5.0.

FYI @jrbourbeau

[jrbourbeau]

I'm not sure exactly what's going on here, but things are failing here

benchmarks/tests/conftest.py

Line 618 in cc6cf17

client.register_worker_plugin(request.param["worker_plugin"])

during test setup. It's not clear to me that this is related to optuna -- seems more like a cluster deployment issue (again, I've only looked into this a little)