Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] label rayNodeType is useless #698

Merged
merged 6 commits into from
Nov 9, 2022
Merged

[Bug] label rayNodeType is useless #698

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a6ca7bb
sample YAMLs
kevin85421 Nov 8, 2022
85a457a
pass go unit tests
kevin85421 Nov 8, 2022
76aadae
prometheus
kevin85421 Nov 8, 2022
13443f5
compatibility test & config test
kevin85421 Nov 8, 2022
e150a65
helm chart
kevin85421 Nov 8, 2022
16b690c
update service
kevin85421 Nov 9, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion config/prometheus/podMonitor.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,6 @@ spec:
- ray-system
selector:
matchLabels:
rayNodeType: worker
ray.io/node-type: worker
ray.io/is-ray-node: "yes"
podMetricsEndpoints:
Expand Down
1 change: 0 additions & 1 deletion docs/guidance/observability.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,6 @@ spec:
matchLabels:
ray.io/is-ray-node: "yes"
ray.io/node-type: worker
rayNodeType: worker
```

Since we are not selecting a Kubernetes service but pods, our `matchLabels` now define a set of labels that is common on all Ray workers.
Expand Down
3 changes: 0 additions & 3 deletions helm-chart/ray-cluster/templates/raycluster-cluster.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@ spec:
annotations: {{- toYaml .Values.head.annotations | nindent 10 }}
labels:
groupName: {{ .Values.head.groupName }}
rayNodeType: {{ .Values.head.type }}
rayCluster: {{ include "ray-cluster.fullname" . }}
{{ include "ray-cluster.labels" . | indent 10 }}

Expand Down Expand Up @@ -108,7 +107,6 @@ spec:
metadata:
annotations: {{- toYaml $values.annotations | nindent 10 }}
labels:
rayNodeType: {{ $values.type }}
groupName: {{ $groupName }}
rayCluster: {{ include "ray-cluster.fullname" $ }}
{{ include "ray-cluster.labels" $ | indent 10 }}
Expand Down Expand Up @@ -158,7 +156,6 @@ spec:
metadata:
annotations: {{- toYaml .Values.worker.annotations | nindent 10 }}
labels:
rayNodeType: {{ .Values.worker.type }}
groupName: {{ .Values.worker.groupName }}
rayCluster: {{ include "ray-cluster.fullname" . }}
{{ include "ray-cluster.labels" . | indent 10 }}
Expand Down
8 changes: 3 additions & 5 deletions ray-operator/apis/ray/v1alpha1/rayjob_types_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,9 +38,8 @@ var expectedRayJob = RayJob{
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{
"rayCluster": "raycluster-sample",
"rayNodeType": "head",
"groupName": "headgroup",
"rayCluster": "raycluster-sample",
"groupName": "headgroup",
},
Annotations: map[string]string{
"key": "value",
Expand Down Expand Up @@ -166,8 +165,7 @@ var testRayJobJSON = `{
"creationTimestamp": null,
"labels": {
"groupName": "headgroup",
"rayCluster": "raycluster-sample",
"rayNodeType": "head"
"rayCluster": "raycluster-sample"
},
"annotations": {
"key": "value"
Expand Down
8 changes: 3 additions & 5 deletions ray-operator/apis/ray/v1alpha1/rayservice_types_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,8 @@ var myRayService = &RayService{
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{
"rayCluster": "raycluster-sample",
"rayNodeType": "head",
"groupName": "headgroup",
"rayCluster": "raycluster-sample",
"groupName": "headgroup",
},
Annotations: map[string]string{
"key": "value",
Expand Down Expand Up @@ -258,8 +257,7 @@ var expected = `{
"creationTimestamp":null,
"labels":{
"groupName":"headgroup",
"rayCluster":"raycluster-sample",
"rayNodeType":"head"
"rayCluster":"raycluster-sample"
},
"annotations":{
"key":"value"
Expand Down
3 changes: 0 additions & 3 deletions ray-operator/config/samples/ray-cluster.autoscaler.large.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,9 +56,6 @@ spec:
headGroupSpec:
# Kubernetes Service Type, valid values are 'ClusterIP', 'NodePort' and 'LoadBalancer'
serviceType: ClusterIP
# logical group name, for this called head-group, also can be functional
# pod type head or worker
# rayNodeType: head # Not needed since it is under the headgroup
# the following params are used to complete the ray start: ray start --head --block --port=6379 ...
rayStartParams:
# Flag "no-monitor" will be automatically set when autoscaling is enabled.
Expand Down
3 changes: 0 additions & 3 deletions ray-operator/config/samples/ray-cluster.autoscaler.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,9 +47,6 @@ spec:
headGroupSpec:
# Kubernetes Service Type, valid values are 'ClusterIP', 'NodePort' and 'LoadBalancer'
serviceType: ClusterIP
# logical group name, for this called head-group, also can be functional
# pod type head or worker
# rayNodeType: head # Not needed since it is under the headgroup
# the following params are used to complete the ray start: ray start --head --block ...
rayStartParams:
# Flag "no-monitor" will be automatically set when autoscaling is enabled.
Expand Down
2 changes: 0 additions & 2 deletions ray-operator/config/samples/ray-cluster.complete.large.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-sample # will be injected if missing
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
# annotations for pod
annotations:
Expand Down Expand Up @@ -94,7 +93,6 @@ spec:
metadata:
labels:
rayCluster: raycluster-complete # will be injected if missing
rayNodeType: worker # will be injected if missing
groupName: small-group # will be injected if missing
spec:
containers:
Expand Down
2 changes: 0 additions & 2 deletions ray-operator/config/samples/ray-cluster.complete.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-sample # will be injected if missing
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
# annotations for pod
annotations:
Expand Down Expand Up @@ -87,7 +86,6 @@ spec:
metadata:
labels:
rayCluster: raycluster-complete # will be injected if missing
rayNodeType: worker # will be injected if missing
groupName: small-group # will be injected if missing
spec:
containers:
Expand Down
2 changes: 0 additions & 2 deletions ray-operator/config/samples/ray-cluster.external-redis.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-external-redis # will be injected if missing
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
spec:
containers:
Expand Down Expand Up @@ -135,7 +134,6 @@ spec:
metadata:
labels:
rayCluster: raycluster-external-redis # will be injected if missing
rayNodeType: worker # will be injected if missing
groupName: small-group # will be injected if missing
# annotations for pod
annotations:
Expand Down
4 changes: 0 additions & 4 deletions ray-operator/config/samples/ray-cluster.getting-started.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,6 @@ spec:
serviceType: ClusterIP
# the pod replicas in this group typed head (assuming there could be more than 1 in the future)
replicas: 1
# logical group name, for this called head-group, also can be functional
# pod type head or worker
# rayNodeType: head # Not needed since it is under the headgroup
# the following params are used to complete the ray start: ray start --head --block --redis-port=6379 ...
rayStartParams:
port: '6379' # should match headService targetPort
Expand All @@ -37,7 +34,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-sample # will be injected if missing
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
# annotations for pod
annotations:
Expand Down
6 changes: 0 additions & 6 deletions ray-operator/config/samples/ray-cluster.heterogeneous.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,9 +44,6 @@ spec:
serviceType: ClusterIP
# the pod replicas in this group typed head (assuming there could be more than 1 in the future)
replicas: 1
# logical group name, for this called head-group, also can be functional
# pod type head or worker
# rayNodeType: head # Not needed since it is under the headgroup
# the following params are used to complete the ray start: ray start --head --block --redis-port=6379 ...
rayStartParams:
port: '6379'
Expand All @@ -63,7 +60,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-heterogeneous # will be injected if missing
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
# annotations for pod
annotations:
Expand Down Expand Up @@ -125,7 +121,6 @@ spec:
metadata:
labels:
rayCluster: raycluster-heterogeneous # will be injected if missing
rayNodeType: worker # will be injected if missing
groupName: small-group # will be injected if missing
# annotations for pod
annotations:
Expand Down Expand Up @@ -200,7 +195,6 @@ spec:
metadata:
labels:
rayCluster: raycluster-heterogeneous # will be injected if missing
rayNodeType: worker # will be injected if missing
# annotations for pod
annotations:
key: value
Expand Down
4 changes: 0 additions & 4 deletions ray-operator/config/samples/ray-cluster.mini.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,6 @@ spec:
serviceType: ClusterIP
# the pod replicas in this group typed head (assuming there could be more than 1 in the future)
replicas: 1
# logical group name, for this called head-group, also can be functional
# pod type head or worker
# rayNodeType: head # Not needed since it is under the headgroup
# the following params are used to complete the ray start: ray start --head --block --redis-port=6379 ...
rayStartParams:
port: '6379' # should match container port named gcs-server
Expand All @@ -38,7 +35,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-sample # will be injected if missing
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
# annotations for pod
annotations:
Expand Down
4 changes: 0 additions & 4 deletions ray-operator/config/samples/ray_v1alpha1_rayjob.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,6 @@ spec:
serviceType: ClusterIP
# the pod replicas in this group typed head (assuming there could be more than 1 in the future)
replicas: 1
# logical group name, for this called head-group, also can be functional
# pod type head or worker
# rayNodeType: head # Not needed since it is under the headgroup
# the following params are used to complete the ray start: ray start --head --block --redis-port=6379 ...
rayStartParams:
port: '6379' # should match container port named gcs-server
Expand All @@ -40,7 +37,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-sample # will be injected if missing
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
# annotations for pod
annotations:
Expand Down
6 changes: 1 addition & 5 deletions ray-operator/config/samples/ray_v1alpha1_rayservice.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ spec:
serveConfig:
importPath: fruit.deployment_graph
runtimeEnv: |
working_dir: "https://github.com/ray-project/test_dag/archive/c620251044717ace0a4c19d766d43c5099af8a77.zip"
working_dir: "https://github.com/ray-project/test_dag/archive/41d09119cbdf8450599f993f51318e9e27c59098.zip"
deployments:
- name: MangoStand
numReplicas: 1
Expand Down Expand Up @@ -50,9 +50,6 @@ spec:
serviceType: ClusterIP
# the pod replicas in this group typed head (assuming there could be more than 1 in the future)
replicas: 1
# logical group name, for this called head-group, also can be functional
# pod type head or worker
# rayNodeType: head # Not needed since it is under the headgroup
# the following params are used to complete the ray start: ray start --head --block --redis-port=6379 ...
rayStartParams:
port: '6379' # should match container port named gcs-server
Expand All @@ -70,7 +67,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-sample # will be injected if missing
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
# annotations for pod
annotations:
Expand Down
5 changes: 2 additions & 3 deletions ray-operator/controllers/ray/rayjob_controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,9 +65,8 @@ var _ = Context("Inside the default namespace", func() {
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{
"rayCluster": "raycluster-sample",
"rayNodeType": "head",
"groupName": "headgroup",
"rayCluster": "raycluster-sample",
"groupName": "headgroup",
},
Annotations: map[string]string{
"key": "value",
Expand Down
5 changes: 2 additions & 3 deletions ray-operator/controllers/ray/rayservice_controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,9 +102,8 @@ var _ = Context("Inside the default namespace", func() {
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: map[string]string{
"rayCluster": "raycluster-sample",
"rayNodeType": "head",
"groupName": "headgroup",
"rayCluster": "raycluster-sample",
"groupName": "headgroup",
},
Annotations: map[string]string{
"key": "value",
Expand Down
4 changes: 2 additions & 2 deletions tests/compatibility-test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@ def test_ray_serve(self):

# KubeRay only allows at most 1 head pod per RayCluster instance at the same time. In addition,
# if we have 0 head pods at this moment, it indicates that the head pod crashes unexpectedly.
headpods = utils.get_pod(k8s_api, namespace='default', label_selector='rayNodeType=head')
headpods = utils.get_pod(k8s_api, namespace='default', label_selector='ray.io/node-type=head')
assert(len(headpods.items) == 1)
old_head_pod = headpods.items[0]
old_head_pod_name = old_head_pod.metadata.name
Expand Down Expand Up @@ -240,7 +240,7 @@ def test_detached_actor(self):

# KubeRay only allows at most 1 head pod per RayCluster instance at the same time. In addition,
# if we have 0 head pods at this moment, it indicates that the head pod crashes unexpectedly.
headpods = utils.get_pod(k8s_api, namespace='default', label_selector='rayNodeType=head')
headpods = utils.get_pod(k8s_api, namespace='default', label_selector='ray.io/node-type=head')
assert(len(headpods.items) == 1)
old_head_pod = headpods.items[0]
old_head_pod_name = old_head_pod.metadata.name
Expand Down
5 changes: 0 additions & 5 deletions tests/config/ray-cluster.mini.yaml.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,6 @@ spec:
serviceType: ClusterIP
# the pod replicas in this group typed head (assuming there could be more than 1 in the future)
replicas: 1
# logical group name, for this called head-group, also can be functional
# pod type head or worker
# rayNodeType: head # Not needed since it is under the headgroup
# the following params are used to complete the ray start: ray start --head --block --redis-port=6379 ...
rayStartParams:
port: '6379' # should match headService targetPort
Expand All @@ -34,7 +31,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-compatibility-test
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
# annotations for pod
annotations:
Expand Down Expand Up @@ -70,7 +66,6 @@ spec:
metadata:
labels:
rayCluster: raycluster-compatibility-test
rayNodeType: worker # will be injected if missing
groupName: small-group # will be injected if missing
# annotations for pod
annotations:
Expand Down
2 changes: 0 additions & 2 deletions tests/config/ray-cluster.ray-ft.yaml.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,6 @@ spec:
metadata:
labels:
rayCluster: raycluster-compatibility-test
rayNodeType: head
groupName: headgroup # will be injected if missing
spec:
containers:
Expand Down Expand Up @@ -131,7 +130,6 @@ spec:
metadata:
labels:
rayCluster: raycluster-compatibility-test
rayNodeType: worker # will be injected if missing
groupName: small-group # will be injected if missing
# annotations for pod
annotations:
Expand Down
4 changes: 0 additions & 4 deletions tests/config/ray-service-cluster-update.yaml.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,6 @@ spec:
serviceType: ClusterIP
# the pod replicas in this group typed head (assuming there could be more than 1 in the future)
replicas: 1
# logical group name, for this called head-group, also can be functional
# pod type head or worker
# rayNodeType: head # Not needed since it is under the headgroup
# the following params are used to complete the ray start: ray start --head --block --redis-port=6379 ...
rayStartParams:
port: '6379' # should match container port named gcs-server
Expand All @@ -67,7 +64,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-compatibility-test # will be injected if missing
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
# annotations for pod
annotations:
Expand Down
4 changes: 0 additions & 4 deletions tests/config/ray-service-serve-update.yaml.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,6 @@ spec:
serviceType: ClusterIP
# the pod replicas in this group typed head (assuming there could be more than 1 in the future)
replicas: 1
# logical group name, for this called head-group, also can be functional
# pod type head or worker
# rayNodeType: head # Not needed since it is under the headgroup
# the following params are used to complete the ray start: ray start --head --block --redis-port=6379 ...
rayStartParams:
port: '6379' # should match container port named gcs-server
Expand All @@ -67,7 +64,6 @@ spec:
# custom labels. NOTE: do not define custom labels start with `raycluster.`, they may be used in controller.
# Refer to https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
rayCluster: raycluster-compatibility-test # will be injected if missing
rayNodeType: head # will be injected if missing, must be head or wroker
groupName: headgroup # will be injected if missing
# annotations for pod
annotations:
Expand Down
Loading