[RLlib] Add lr_schedule support to SimpleQ and PG.

[mgerstgrasser]

Why are these changes needed?

SimpleQ and PG currently don't support a learning rate schedule. SimpleQ does list a lr_schedule key in its default config, but will simply ignore it, which could lead to user confusion. PG does not accept a lr_schedule key, but there is no reason why it shouldn't support a learning rate schedule.

Related issue number

Closes #28266

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[ArturNiederfahrenhorst]

Thanks for this PR! I'm adding some testing and minor stuff and then we can have a review and merge :)

[mgerstgrasser]

Thanks for this PR! I'm adding some testing and minor stuff and then we can have a review and merge :)

Oh thank you! Anything I can do to help?

[ArturNiederfahrenhorst]

Not for now, thanks! @sven1977 will have to review next.

[sven1977]

Hey @mgerstgrasser , thanks for the PR! This looks great.
One test case seems to be failing: rllib/tests/test_nested_action_spaces.py

I'm getting this error here:

[mgerstgrasser]

Hey @mgerstgrasser , thanks for the PR! This looks great. One test case seems to be failing: rllib/tests/test_nested_action_spaces.py

I'm getting this error here:

@sven1977 Should be fixed now, thanks for flagging this. Issue was in the test - it's passing a PG config into BC. I'm just removing the lr_schedule from the config now in that test.