[Core] Issue with installation of ray

[gomrinal]

What happened + What you expected to happen

Error

2023-06-28 05:02:18,247 ERROR services.py:1207 -- Failed to start the dashboard , return code 1
2023-06-28 05:02:18,248 ERROR services.py:1232 -- Error should be written to 'dashboard.log' or 'dashboard.err'. We are printing the last 20 lines for you. See 'https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure' to find where the log file is.
2023-06-28 05:02:18,268 ERROR services.py:1276 -- 
The last 20 lines of /tmp/ray/session_2023-06-28_05-02-16_322649_92398/logs/dashboard.log (it contains the error message from the dashboard): 
    modules = self._load_modules(self._modules_to_load)
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/site-packages/ray/dashboard/head.py", line 207, in _load_modules
    head_cls_list = dashboard_utils.get_all_modules(DashboardHeadModule)
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/site-packages/ray/dashboard/utils.py", line 121, in get_all_modules
    importlib.import_module(name)
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/site-packages/ray/dashboard/modules/snapshot/snapshot_head.py", line 40, in <module>
    class RayActivityResponse(BaseModel, extra=Extra.allow):
  File "pydantic/main.py", line 324, in pydantic.main.ModelMetaclass.__new__
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/abc.py", line 85, in __new__
    cls = super().__new__(mcls, name, bases, namespace, **kwargs)
TypeError: __init_subclass__() takes no keyword arguments
2023-06-28 05:02:18,457 INFO worker.py:1636 -- Started a local Ray instance.
(raylet) [2023-06-28 05:02:19,537 E 92409 10348061] (raylet) agent_manager.cc:135: The raylet exited immediately because the Ray agent failed. The raylet fate shares with the agent. This can happen because the Ray agent was unexpectedly killed or failed. Agent can fail when
(raylet) - The version of `grpcio` doesn't follow Ray's requirement. Agent can segfault with the incorrect `grpcio` version. Check the grpcio version `pip freeze | grep grpcio`.
(raylet) - The agent failed to start because of unexpected error or port conflict. Read the log `cat /tmp/ray/session_latest/logs/dashboard_agent.log`. You can find the log file structure here https://docs.ray.io/en/master/ray-observability/ray-logging.html#logging-directory-structure.
(raylet) - The agent is killed by the OS (e.g., out of memory).
Traceback (most recent call last):
  File "main.py", line 4, in <module>
    dataset = ray.data.read_csv("s3://anonymous@air-example-data/breast_cancer.csv")
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/site-packages/ray/data/read_api.py", line 1015, in read_csv
    return read_datasource(
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/site-packages/ray/_private/auto_init_hook.py", line 18, in auto_init_wrapper
    return fn(*args, **kwargs)
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/site-packages/ray/data/read_api.py", line 380, in read_datasource
    requested_parallelism, min_safe_parallelism, read_tasks = ray.get(
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/site-packages/ray/_private/auto_init_hook.py", line 18, in auto_init_wrapper
    return fn(*args, **kwargs)
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/site-packages/ray/_private/client_mode_hook.py", line 103, in wrapper
    return func(*args, **kwargs)
  File "/Users/mrizzlybear/opt/anaconda3/lib/python3.8/site-packages/ray/_private/worker.py", line 2542, in get
    raise value
ray.exceptions.LocalRayletDiedError: The task's local raylet died. Check raylet.out for more information.

Versions / Dependencies

pip install -U "ray[air]"

The below Ray AIR tutorial was written with the following libraries.

Consider running the following to ensure that the code below runs properly:

pip install -U pandas>=1.3.5
pip install -U torch>=1.12
pip install -U numpy>=1.19.5
pip install -U tensorflow>=2.6.2
pip install -U pyarrow>=6.0.1

Reproduction script

import ray

# Load data.
dataset = ray.data.read_csv("s3://anonymous@air-example-data/breast_cancer.csv")

# Split data into train and validation.
train_dataset, valid_dataset = dataset.train_test_split(test_size=0.3)

# Create a test dataset by dropping the target column.
test_dataset = valid_dataset.drop_columns(cols=["target"])

Issue Severity

High: It blocks me from completing my task.

[rkooo567]

It's a known issue that happens due to a new pydantic release. The immediate workaround is to use pydantic < 2.0. There's also a working PR that will be merged soon (which also will be available from ray 2.6) #37055

[cadedaniel]

@gomrinal does the workaround unblock you ?

[rkooo567]

It is duplicate of #37019. We have an active PR to fix the root cause. Closing as duplicate