[Sample YAML] Bump ray version in pod security YAML to 2.4.0 #1160
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: Archit Kulkarni <[email protected]>
kevin85421
approved these changes
Jun 12, 2023
There was a problem hiding this comment.
We are using Ray 2.2.0 here due to a dependency issue related to protobuf. Please refer to issue #873 for more details. The issue was resolved by the Ray community by manually updating the Docker images for Ray 2.2.0. However, Docker images for Ray 2.3.0 do not fix this issue manually. It looks like this issue was resolved by Ray 2.4.0.
architkulkarni
added a commit
to architkulkarni/kuberay
that referenced
this pull request
Jun 12, 2023
…ject#1160) The existing sample YAML was pinned to Ray 2.2.0. Running the test locally failed with 2023-06-12:14:21:56,768 INFO [utils.py:163] Execute command: kubectl logs -n=pod-security -l ray.io/node-type=head --tail=-1 Error from server (BadRequest): container "ray-head" in pod "raycluster-pod-security-head-hg67c" is waiting to start: ContainerCreating ERROR ====================================================================== ERROR: test_ray_cluster_with_security_context (__main__.PodSecurityTestCase) Create a RayCluster with securityContext config under restricted mode. ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/test_security.py", line 98, in test_ray_cluster_with_security_context ray_cluster_add_event.trigger() File "/Users/archit/kuberay/tests/framework/prototype.py", line 165, in trigger self.wait() File "/Users/archit/kuberay/tests/framework/prototype.py", line 277, in wait show_cluster_info(self.namespace) File "/Users/archit/kuberay/tests/framework/prototype.py", line 90, in show_cluster_info shell_subprocess_run(f'kubectl logs -n={cr_namespace} -l ray.io/node-type=head --tail=-1') File "/Users/archit/kuberay/tests/framework/utils.py", line 164, in shell_subprocess_run return subprocess.run(command, shell = True, check = check).returncode File "/Users/archit/anaconda3/envs/ray-py38/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'kubectl logs -n=pod-security -l ray.io/node-type=head --tail=-1' returned non-zero exit status 1. ---------------------------------------------------------------------- Ran 2 tests in 1189.059s FAILED (errors=1) It's possible this is just a race condition in the test, but it should be updated to Ray 2.4.0 regardless. I tested it locally with Ray 2.4.0 and it passes. ---------------------------------------------------------------------- Ran 2 tests in 900.921s OK This PR will also be cherry-picked to the 0.5.2 release branch. Signed-off-by: Archit Kulkarni <[email protected]>
4 tasks
architkulkarni
added a commit
that referenced
this pull request
Jun 13, 2023
…1161) The existing sample YAML was pinned to Ray 2.2.0. Running the test locally failed with 2023-06-12:14:21:56,768 INFO [utils.py:163] Execute command: kubectl logs -n=pod-security -l ray.io/node-type=head --tail=-1 Error from server (BadRequest): container "ray-head" in pod "raycluster-pod-security-head-hg67c" is waiting to start: ContainerCreating ERROR ====================================================================== ERROR: test_ray_cluster_with_security_context (__main__.PodSecurityTestCase) Create a RayCluster with securityContext config under restricted mode. ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/test_security.py", line 98, in test_ray_cluster_with_security_context ray_cluster_add_event.trigger() File "/Users/archit/kuberay/tests/framework/prototype.py", line 165, in trigger self.wait() File "/Users/archit/kuberay/tests/framework/prototype.py", line 277, in wait show_cluster_info(self.namespace) File "/Users/archit/kuberay/tests/framework/prototype.py", line 90, in show_cluster_info shell_subprocess_run(f'kubectl logs -n={cr_namespace} -l ray.io/node-type=head --tail=-1') File "/Users/archit/kuberay/tests/framework/utils.py", line 164, in shell_subprocess_run return subprocess.run(command, shell = True, check = check).returncode File "/Users/archit/anaconda3/envs/ray-py38/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'kubectl logs -n=pod-security -l ray.io/node-type=head --tail=-1' returned non-zero exit status 1. ---------------------------------------------------------------------- Ran 2 tests in 1189.059s FAILED (errors=1) It's possible this is just a race condition in the test, but it should be updated to Ray 2.4.0 regardless. I tested it locally with Ray 2.4.0 and it passes. ---------------------------------------------------------------------- Ran 2 tests in 900.921s OK This PR will also be cherry-picked to the 0.5.2 release branch. Signed-off-by: Archit Kulkarni <[email protected]>
lowang-bh
pushed a commit
to lowang-bh/kuberay
that referenced
this pull request
Sep 24, 2023
…ject#1160) The existing sample YAML was pinned to Ray 2.2.0. Running the test locally failed with 2023-06-12:14:21:56,768 INFO [utils.py:163] Execute command: kubectl logs -n=pod-security -l ray.io/node-type=head --tail=-1 Error from server (BadRequest): container "ray-head" in pod "raycluster-pod-security-head-hg67c" is waiting to start: ContainerCreating ERROR ====================================================================== ERROR: test_ray_cluster_with_security_context (__main__.PodSecurityTestCase) Create a RayCluster with securityContext config under restricted mode. ---------------------------------------------------------------------- Traceback (most recent call last): File "tests/test_security.py", line 98, in test_ray_cluster_with_security_context ray_cluster_add_event.trigger() File "/Users/archit/kuberay/tests/framework/prototype.py", line 165, in trigger self.wait() File "/Users/archit/kuberay/tests/framework/prototype.py", line 277, in wait show_cluster_info(self.namespace) File "/Users/archit/kuberay/tests/framework/prototype.py", line 90, in show_cluster_info shell_subprocess_run(f'kubectl logs -n={cr_namespace} -l ray.io/node-type=head --tail=-1') File "/Users/archit/kuberay/tests/framework/utils.py", line 164, in shell_subprocess_run return subprocess.run(command, shell = True, check = check).returncode File "/Users/archit/anaconda3/envs/ray-py38/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'kubectl logs -n=pod-security -l ray.io/node-type=head --tail=-1' returned non-zero exit status 1. ---------------------------------------------------------------------- Ran 2 tests in 1189.059s FAILED (errors=1) It's possible this is just a race condition in the test, but it should be updated to Ray 2.4.0 regardless. I tested it locally with Ray 2.4.0 and it passes. ---------------------------------------------------------------------- Ran 2 tests in 900.921s OK This PR will also be cherry-picked to the 0.5.2 release branch. Signed-off-by: Archit Kulkarni <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
The existing sample YAML was pinned to Ray 2.2.0. Running the test locally failed with
It's possible this is just a race condition in the test, but it should be updated to Ray 2.4.0 regardless. I tested it locally with Ray 2.4.0 and it passes.
This PR will also be cherry-picked to the 0.5.2 release branch.
Related issue number
Checks