[Tune] Support initial parameters for SkOpt search algorithm

[markgoodhead]

What do these changes do?

Similar to the recent change to HyperOpt (##3944) this implements both:

1. The ability to pass in initial parameter suggestion(s) to be run through Tune first, before using the Optimiser's suggestions. This is for when you already know good parameters and want the Optimiser to be aware of these when it makes future parameter suggestions.
2. The same as 1. but if you already know the reward value for those parameters you can pass these in as well to avoid having to re-run the experiments. In the future it would be nice for Tune to potentially support this functionality directly by loading previously run Tune experiments and initialising the Optimiser with these (kind of like a top level checkpointing functionality) but this feature allows users to do this manually for now.

[richardliaw]

cc @adizim, can you review this PR? It is based off of your previous contribution to the SkOpt search algorithm component.

[AmplabJenkins]

Can one of the admins verify this patch?

[adizim]

I left a few suggestions. I see you have tested the example case with points_to_evaluate and evaluated_rewards set. Similarly, can you see that the example without evaluated_rewards set works, to cover all cases?

[richardliaw]

BTW @markgoodhead, thanks for this contribution!

In the future it would be nice for Tune to potentially support this functionality directly by loading previously run Tune experiments and initialising the Optimiser with these (kind of like a top level checkpointing functionality) but this feature allows users to do this manually for now.

BTW, this should work soon, and the relevant PRs should be #4234 and #3709.

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12793/
Test PASSed.

[adizim]

I wrote up my suggestions in #4345

[markgoodhead]

I wrote up my suggestions in #4345

Hi @adizim - thanks for the review! I'm a little confused as to what branch you'd like me to implement the changes off as #4345 has been closed and I also see you made a PR into my branch (which contains a lot of other changes besides your tweaks so I think muddies the PR too much) - what way would you like me to proceed from here?

EDIT: Never mind I worked it out that the other changes are just due to me being very behind master - I've merged your branch and will do the other changes you suggest 😄

EDIT2: OK actually it looks like there's some other changes in there that's unrelated to this PR @adizim - mainly in python/ray/rllib/tests/test_lstm.py - are you working on these changes elsewhere that need to be merged into master first? Are you happy for them to go in via my branch?

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12846/
Test FAILed.

[markgoodhead]

@richardliaw @adizim I've made all the requested changes after merging @adizim 's branch. This PR now has some random RLLib changes in so not sure what you want to do about that but otherwise it's good to go.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12847/
Test FAILed.

[adizim]

@markgoodhead Apologies for the extraneous changes, I did not mean to include them, I hope they didn't cause too much confusion. @richardliaw was able to revert them just recently.

@markgoodhead The changes look good to me, though it would be helpful to keep the error messages you wrote. Raising TypeErrors with the error messages should be the correct approach. You can also include the error messages in the assertions as they stand now.

[adizim]

@markgoodhead Nevermind, @richardliaw took care of the error messages

[richardliaw]

jenkins retest this please

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/83/
Test PASSed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12870/
Test FAILed.

[richardliaw]

jenkins retest this please

[AmplabJenkins]

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-Perf-Integration-PRB/84/
Test PASSed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12880/
Test FAILed.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12895/
Test FAILed.

[markgoodhead]

Why do the tests keep failing?

[richardliaw]

I think one of the new tests hang very often and there’s a PR to fix it, should be merged soon. I’ll just run the Skopt test locally and merge if passing though.

…

On Fri, Mar 15, 2019 at 3:02 PM markgoodhead ***@***.***> wrote: Why do the tests keep failing? — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub <#4341 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEUc5bNra3A4JL_B7mt9NaxmGZuZ1JRbks5vXBh3gaJpZM4brZ7O> .

[richardliaw]

This runs locally.

[AmplabJenkins]

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/12917/
Test FAILed.