Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[core] Better management of GCS client in CoreWorker. #35684

Open
fishbone opened this issue May 23, 2023 · 1 comment
Open

[core] Better management of GCS client in CoreWorker. #35684

fishbone opened this issue May 23, 2023 · 1 comment
Labels
core Issues that should be addressed in Ray Core core-worker enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks

Comments

@fishbone
Copy link
Contributor

What happened + What you expected to happen

GcsClient is not managed well in CoreWorker which make the management of GCS connection hard. Right now, we have GcsClient in Pubsub/CoreWorker/Py. Although after migration to cpp based GCS client, the creation logic is randomly distributed in the cluster.

A hot fix is to create a singleton globally and reuse it #35624

This works in some way, but it doesn't gives us the flexibility to shutdown the GCS client channel.

A better way is to create the channel in a centralized way and pass this arounds to other endpoints.

Ideally, all GCS based service should be initialized in CoreWorker and just be passed to python with some cython API.

When CoreWorker is shutdown, we should just shutdown everything.

Related issue: #35681

Versions / Dependencies

master

Reproduction script

In code.

Issue Severity

None
@fishbone fishbone added P1 Issue that should be fixed within a few weeks core Issues that should be addressed in Ray Core labels May 23, 2023
@fishbone fishbone mentioned this issue May 23, 2023
Merged
8 tasks
@jovany-wang
Copy link
Contributor

Goooood

@scv119 scv119 added the enhancement Request for new feature and/or capability label Jun 13, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
core Issues that should be addressed in Ray Core core-worker enhancement Request for new feature and/or capability P1 Issue that should be fixed within a few weeks
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@scv119 @jjyao @jovany-wang @fishbone