Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix Client Table #2880

Merged
merged 7 commits into from
Sep 17, 2018
Merged

Conversation

richardliaw
Copy link
Contributor

@richardliaw richardliaw commented Sep 15, 2018

Hides append-only log in the Python implementation.

Should partially address #2852.

TODO:

  • Add tests for this

@@ -522,8 +524,9 @@ def client_table(self):
client.ResourcesTotalCapacity(i)
for i in range(client.ResourcesTotalLabelLength())
}
node_info.append({
"ClientID": ray.utils.binary_to_hex(client.ClientId()),
client_id = ray.utils.binary_to_hex(client.ClientId())

This comment was marked as resolved.

})
return node_info
}
return list(node_info.values())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a list is probably the right structure to return, though I'm not sure. A map from client_id to info doesn't make sense because client_ids are not really meaningful to users. A map from node_ip_address to list of clients could make sense, though in the vast majority of cases each list will contain a single element.

I think this is fine for now. This should work in the setting where we have multiple raylets per node, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, this works in the setting where we have multiple raylets per node (although it doesn't work when the raylets per node are removed ;) #2878)

I think the client_id -> info mapping is actually fine, but I'm also OK with leaving as is. In calling client_table, I don't expect a list of node_ip_addresses; I expect some data structure that has len() == num clients.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8224/
Test FAILed.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8231/
Test FAILed.

@richardliaw
Copy link
Contributor Author

OK I added a light test for this using subprocess to start ray; this may only work in xray? and may also require https://pypi.org/project/pytest-timeout/ to be installed on Travis.

Is there a better way to achieve what I'm trying to do?

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8235/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8236/
Test PASSed.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8248/
Test PASSed.



@ray.remote(num_cpus=1)
def cpu_task(seconds):
time.sleep(seconds)


def _get_raylet_pid(raylet_socket):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @robertnishihara, is there a better way of getting the Raylet PID?

Copy link
Collaborator

@robertnishihara robertnishihara left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, pending tests passing.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8258/
Test FAILed.

@robertnishihara
Copy link
Collaborator

jenkins, retest this please

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/8265/
Test PASSed.

@robertnishihara robertnishihara merged commit 899e458 into ray-project:master Sep 17, 2018
@robertnishihara robertnishihara deleted the client_table_fix branch September 17, 2018 19:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants