es_grad.py parallelisation with ray

[ghost]

Hello Alois,

This PR uses ray in es_grad.py to evaluate and update in parallel the actors.
It results in a great speed-up (almost x num_rl_actors).
To use it, you just need to pip install ray in your python environment first.

From my tests, I observed an unchanged performance however note that I had to update first the critic before updating the actors to enable parallelization which is a small deviation compared to your version.

Hope it'll help :)

[apourchot]

Hi Thomas, seems great, thank you ! I think it would be best to add those modifications in a new file, so as to keep the old behavior used in the paper intact. Could you perhaps create a new "es_grad_parallel", or something similar ?