[rllib] Use compress observations where replay buffers and image obs are used in tuned examples

[avnishn]

Many of our tuned examples using replay and atari wouldn't be able to run on clusters or single node machines because they'll oom without enabling this compression.

Signed-off-by: avnish [email protected]

Why are these changes needed?

Related issue number

Checks

• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[gjoliver]

nit nit nit
can we write the comments more formally, like:
Compress observations to make sure the size of the ReplayBuffer fits most computers.
something like that? not really good at wording, but you get the idea.
thanks.

[avnishn]

could I change
"we should set this because few machines would be able to contain the replay buffers in memory otherwise" to

"we should set compress_observations to True because few machines would be able to contain the replay buffers in memory otherwise"

is that descriptive enough?

[avnishn]

I addressed the comments, but during today's experiments realized that only pong examples need obs compression so I turned off the compression in the non pong examples that I changed before.

[avnishn]

just needed a newline

[gjoliver]

this is good trick.

[gjoliver]

@richardliaw please merge