Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[autoscaler] Updating a file in file mounts causes all worker nodes to get restarted #3860

Merged
merged 1 commit into from
Jan 28, 2019
Merged

[autoscaler] Updating a file in file mounts causes all worker nodes to get restarted #3860

merged 1 commit into from
Jan 28, 2019

Conversation

ericl
Copy link
Contributor

@ericl ericl commented Jan 25, 2019 •
edited
Loading

What do these changes do?

It seems like if you update a file on the head node that was part of file mounts, all worker nodes get restarted.

This is particularly bad if you try importing a file in the mounts, which may create *.pyc files and trigger worker restarts.

Related issue number

Closes #3859

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11160/
Test FAILed.

@ericl
Copy link
Contributor Author

ericl commented Jan 26, 2019

jenkins retest this please

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Ray-PRB/11200/
Test PASSed.

@ericl ericl added the tests-ok The tagger certifies test failures are unrelated and assumes personal liability. label Jan 27, 2019
@ericl
Copy link
Contributor Author

ericl commented Jan 27, 2019

FYI @hartikainen I recall you reported something similar before, and we couldn't figure it out

@ericl ericl mentioned this pull request Jan 28, 2019
Merged
@ericl ericl merged commit c75038b into ray-project:master Jan 28, 2019
@richardliaw richardliaw mentioned this pull request Jan 30, 2019
Merged
@stephanie-wang stephanie-wang mentioned this pull request Feb 5, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@richardliaw richardliaw richardliaw approved these changes

Assignees

@richardliaw richardliaw

Labels
tests-ok The tagger certifies test failures are unrelated and assumes personal liability.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@ericl @AmplabJenkins @richardliaw