[RLlib] QuickStart doesn't work

[tudorjnu]

What happened + What you expected to happen

The QuickStart code doesn't work, throwing the following error:

ValueError: Env must be of one of the following supported types: BaseEnv, gym.Env, MultiAgentEnv, Vec
torEnv, RemoteBaseEnv, ExternalMultiAgentEnv, ExternalEnv, but instead is of type <class '__main__.ParrotEnv'>.

Versions / Dependencies

name: rllib
channels:
  - defaults
dependencies:
  - _libgcc_mutex=0.1=main
  - _openmp_mutex=5.1=1_gnu
  - ca-certificates=2022.10.11=h06a4308_0
  - certifi=2022.9.24=py38h06a4308_0
  - ld_impl_linux-64=2.38=h1181459_1
  - libffi=3.4.2=h6a678d5_6
  - libgcc-ng=11.2.0=h1234567_1
  - libgomp=11.2.0=h1234567_1
  - libstdcxx-ng=11.2.0=h1234567_1
  - ncurses=6.3=h5eee18b_3
  - openssl=1.1.1s=h7f8727e_0
  - pip=22.3.1=py38h06a4308_0
  - python=3.8.15=h7a1cb2a_2
  - readline=8.2=h5eee18b_0
  - setuptools=65.5.0=py38h06a4308_0
  - sqlite=3.40.0=h5082296_0
  - tk=8.6.12=h1ccaba5_0
  - wheel=0.37.1=pyhd3eb1b0_0
  - xz=5.2.8=h5eee18b_0
  - zlib=1.2.13=h5eee18b_0
  - pip:
      - absl-py==1.3.0
      - aiosignal==1.3.1
      - ale-py==0.7.5
      - astunparse==1.6.3
      - atari-py==0.2.9
      - attrs==22.1.0
      - autorom==0.4.2
      - autorom-accept-rom-license==0.5.0
      - cachetools==5.2.0
      - charset-normalizer==2.1.1
      - click==8.1.3
      - cloudpickle==2.2.0
      - commonmark==0.9.1
      - contourpy==1.0.6
      - cycler==0.11.0
      - distlib==0.3.6
      - dm-control==1.0.9
      - dm-env==1.6
      - dm-tree==0.1.8
      - filelock==3.8.2
      - flatbuffers==22.12.6
      - fonttools==4.38.0
      - frozenlist==1.3.3
      - gast==0.4.0
      - glfw==2.5.5
      - google-auth==2.15.0
      - google-auth-oauthlib==0.4.6
      - google-pasta==0.2.0
      - grpcio==1.51.1
      - gym==0.23.1
      - gym-notices==0.0.8
      - gymnasium==0.27.0
      - gymnasium-notices==0.0.1
      - h5py==3.7.0
      - idna==3.4
      - imageio==2.23.0
      - importlib-metadata==5.2.0
      - importlib-resources==5.10.1
      - jax-jumpy==0.2.0
      - jsonschema==4.17.3
      - keras==2.11.0
      - kiwisolver==1.4.4
      - labmaze==1.0.6
      - libclang==14.0.6
      - libtorrent==2.0.7
      - lxml==4.9.2
      - lz4==4.0.2
      - markdown==3.4.1
      - markupsafe==2.1.1
      - matplotlib==3.6.2
      - msgpack==1.0.4
      - mujoco==2.3.1.post1
      - networkx==2.8.8
      - numpy==1.24.0
      - nvidia-cublas-cu11==11.10.3.66
      - nvidia-cuda-nvrtc-cu11==11.7.99
      - nvidia-cuda-runtime-cu11==11.7.99
      - nvidia-cudnn-cu11==8.5.0.96
      - oauthlib==3.2.2
      - opt-einsum==3.3.0
      - packaging==22.0
      - pandas==1.5.2
      - pillow==9.3.0
      - pkgutil-resolve-name==1.3.10
      - platformdirs==2.6.0
      - protobuf==3.19.6
      - pyasn1==0.4.8
      - pyasn1-modules==0.2.8
      - pygments==2.13.0
      - pyopengl==3.1.6
      - pyparsing==3.0.9
      - pyrsistent==0.19.2
      - python-dateutil==2.8.2
      - pytz==2022.7
      - pywavelets==1.4.1
      - pyyaml==6.0
      - ray==2.2.0
      - requests==2.28.1
      - requests-oauthlib==1.3.1
      - rich==12.6.0
      - rsa==4.9
      - scikit-image==0.19.3
      - scipy==1.9.3
      - shimmy==0.2.0
      - six==1.16.0
      - tabulate==0.9.0
      - tensorboard==2.11.0
      - tensorboard-data-server==0.6.1
      - tensorboard-plugin-wit==1.8.1
      - tensorboardx==2.5.1
      - tensorflow==2.11.0
      - tensorflow-estimator==2.11.0
      - tensorflow-io-gcs-filesystem==0.29.0
      - termcolor==2.1.1
      - tifffile==2022.10.10
      - torch==1.13.1
      - tqdm==4.64.1
      - typer==0.7.0
      - typing-extensions==4.4.0
      - urllib3==1.26.13
      - virtualenv==20.17.1
      - werkzeug==2.2.2
      - wrapt==1.14.1
      - zipp==3.11.0

Reproduction script

The code is on the main page.

Issue Severity

Low: It annoys or frustrates me.

[ArturNiederfahrenhorst]

Hi @tudorjnu ,

I don't see a script called "QuickStart", are you talking about this one? It run's fine one my end.

[TeamDman]

Possibly related to gymnasium support which hasn't had a full release yet #29697
but nightly builds will prob work

[tudorjnu]

Hello! Sorry for the late reply. Yes, it was the script from the github rllib page. The ParrotEnv script. I found out that the normal installation also seems to throw a bunch of numpy errors and fail which seems to be the fault of the numpy version. I downgraded the numpy version to numpy<1.24 and it seems to work.

Thank you and Happy New Year! 🥳

[ArturNiederfahrenhorst]

Hi @tudorjnu ,

Your description says "The code is on the main page." and you added "it was the script from the github rllib page".
That makes me believe it was probably this script? If so, this one runs fine on my end on master.
Your error does not look like it, but since you mention numpy, could it be related to #31240?

[tudorjnu]

Hello @ArturNiederfahrenhorst!

Yap, that is the script I am referring to. And it is probably related to that issue indeed. Other than that, it seems that the environment runs well.

I installed the environment using the following command within a conda environment:

 pip install -U "ray[rllib] @ https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp39-cp39-manylinux2014_x86_64.whl"

[ArturNiederfahrenhorst]

Hi @tudorjnu ,

You installed a nightly version and the script you executed was released together with Ray 2.2.0.
You can find scripts that relate to the master branch (which is released in nightlies) on the master documentation pages.
Thanks for opening the issue though 🙂

[yonghwanbot]

I got the above error if I registered the env via,
"
from ray.tune.registry import register_env
def env_creator(env_config):
return MyEnv(...) # return an env instance
register_env("my_env", env_creator)
"
Your code has multiple rollout workers, but didn't register the env in the way that the official document described. I am not sure if it still can do parallel processing (e.g. using multiple GPUs simultaneously) properly.