Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] add runAsNonRoot filed #195

Merged
merged 5 commits into from
Jul 27, 2023
Merged

[Feature] add runAsNonRoot filed #195

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e1a02fc
[Feature] add runAsNonRoot filed
yandongxiao Jul 26, 2023
ba9abda
update comment on RunAsNonRoot field
yandongxiao Jul 26, 2023
3b128e3
remove fsGroup field
yandongxiao Jul 26, 2023
7d1c2f8
add FSGroup field in pod securityContext
yandongxiao Jul 27, 2023
aa3aa04
Update pkg/k8sutils/templates/pod/spec.go
yandongxiao Jul 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions config/crd/bases/starrocks.com_starrocksclusters.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1168,6 +1168,11 @@ spec:
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
runAsNonRoot:
description: 'RunAsGroup is used to determine whether to run starrocks
as a normal user. If RunAsGroup is true, operator will set RunAsGroup
and RunAsGroup to 1000 in securityContext. default: nil'
type: boolean
schedulerName:
description: SchedulerName is the name of the kubernetes scheduler
that will be used to schedule the pods.
Expand Down Expand Up @@ -3112,6 +3117,11 @@ spec:
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
runAsNonRoot:
description: 'RunAsGroup is used to determine whether to run starrocks
as a normal user. If RunAsGroup is true, operator will set RunAsGroup
and RunAsGroup to 1000 in securityContext. default: nil'
type: boolean
schedulerName:
description: SchedulerName is the name of the kubernetes scheduler
that will be used to schedule the pods.
Expand Down Expand Up @@ -4382,6 +4392,11 @@ spec:
it defaults to Limits if that is explicitly specified, otherwise
to an implementation-defined value. More info: https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/'
type: object
runAsNonRoot:
description: 'RunAsGroup is used to determine whether to run starrocks
as a normal user. If RunAsGroup is true, operator will set RunAsGroup
and RunAsGroup to 1000 in securityContext. default: nil'
type: boolean
schedulerName:
description: SchedulerName is the name of the kubernetes scheduler
that will be used to schedule the pods.
Expand Down
6 changes: 6 additions & 0 deletions deploy/starrocks.com_starrocksclusters.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -549,6 +549,8 @@ spec:
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
type: object
runAsNonRoot:
type: boolean
schedulerName:
type: string
secrets:
Expand Down Expand Up @@ -1467,6 +1469,8 @@ spec:
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
type: object
runAsNonRoot:
type: boolean
schedulerName:
type: string
secrets:
Expand Down Expand Up @@ -2058,6 +2062,8 @@ spec:
pattern: ^(\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))(([KMGTPE]i)|[numkMGTPE]|([eE](\+|-)?(([0-9]+(\.[0-9]*)?)|(\.[0-9]+))))?$
x-kubernetes-int-or-string: true
type: object
runAsNonRoot:
type: boolean
schedulerName:
type: string
secrets:
Expand Down
52 changes: 52 additions & 0 deletions doc/api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1374,6 +1374,19 @@ string
</tr>
<tr>
<td>
<code>runAsNonRoot</code><br/>
<em>
bool
</em>
</td>
<td>
<p>RunAsGroup is used to determine whether to run starrocks as a normal user.
If RunAsGroup is true, operator will set RunAsGroup and RunAsGroup to 1000 in securityContext.
default: nil</p>
</td>
</tr>
<tr>
<td>
<code>fsGroup</code><br/>
<em>
int64
Expand Down Expand Up @@ -2037,6 +2050,19 @@ string
</tr>
<tr>
<td>
<code>runAsNonRoot</code><br/>
<em>
bool
</em>
</td>
<td>
<p>RunAsGroup is used to determine whether to run starrocks as a normal user.
If RunAsGroup is true, operator will set RunAsGroup and RunAsGroup to 1000 in securityContext.
default: nil</p>
</td>
</tr>
<tr>
<td>
<code>fsGroup</code><br/>
<em>
int64
Expand Down Expand Up @@ -2486,6 +2512,19 @@ string
</tr>
<tr>
<td>
<code>runAsNonRoot</code><br/>
<em>
bool
</em>
</td>
<td>
<p>RunAsGroup is used to determine whether to run starrocks as a normal user.
If RunAsGroup is true, operator will set RunAsGroup and RunAsGroup to 1000 in securityContext.
default: nil</p>
</td>
</tr>
<tr>
<td>
<code>fsGroup</code><br/>
<em>
int64
Expand Down Expand Up @@ -2884,6 +2923,19 @@ string
</tr>
<tr>
<td>
<code>runAsNonRoot</code><br/>
<em>
bool
</em>
</td>
<td>
<p>RunAsGroup is used to determine whether to run starrocks as a normal user.
If RunAsGroup is true, operator will set RunAsGroup and RunAsGroup to 1000 in securityContext.
default: nil</p>
</td>
</tr>
<tr>
<td>
<code>fsGroup</code><br/>
<em>
int64
Expand Down
3 changes: 3 additions & 0 deletions helm-charts/charts/kube-starrocks/templates/starrocks/starrockscluster.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ spec:
serviceAccount: {{ .Values.starrocksFESpec.serviceAccount }}
{{- end }}
fsGroup: {{ .Values.starrocksFESpec.fsGroup }}
runAsNonRoot: {{ .Values.starrocksFESpec.runAsNonRoot }}
{{- if .Values.starrocksFESpec.nodeSelector }}
nodeSelector:
{{ toYaml .Values.starrocksFESpec.nodeSelector | indent 6 }}
Expand Down Expand Up @@ -162,6 +163,7 @@ spec:
serviceAccount: {{ .Values.starrocksBeSpec.serviceAccount }}
{{- end }}
fsGroup: {{ .Values.starrocksBeSpec.fsGroup }}
runAsNonRoot: {{ .Values.starrocksBeSpec.runAsNonRoot }}
{{- if .Values.starrocksBeSpec.podLabels }}
podLabels:
{{toYaml .Values.starrocksBeSpec.podLabels | indent 6 }}
Expand Down Expand Up @@ -250,6 +252,7 @@ spec:
serviceAccount: {{ .Values.starrocksCnSpec.serviceAccount }}
{{- end }}
fsGroup: {{ .Values.starrocksCnSpec.fsGroup }}
runAsNonRoot: {{ .Values.starrocksCnSpec.runAsNonRoot }}
{{- if .Values.starrocksCnSpec.podLabels }}
podLabels:
{{toYaml .Values.starrocksCnSpec.podLabels | indent 6 }}
Expand Down
9 changes: 9 additions & 0 deletions helm-charts/charts/kube-starrocks/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,9 @@ starrocksFESpec:
tag: 3.0-latest
# add annotations for fe pods. For example, if you want to config monitor for datadog, you can config the annotations.
annotations: {}
# If runAsNonRoot is true, the container is run as non-root user.
# The userId will be set to 1000, and the groupID will be set to 1000.
runAsNonRoot: false
# A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change
# the ownership of that volume to be owned by the pod.
# Ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods
Expand Down Expand Up @@ -191,6 +194,9 @@ starrocksCnSpec:
serviceAccount: ""
# add annotations for cn pods. example, if you want to config monitor for datadog, you can config the annotations.
annotations: {}
# If runAsNonRoot is true, the container is run as non-root user.
# The userId will be set to 1000, and the groupID will be set to 1000.
runAsNonRoot: false
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this configuration overwrite fsGroup? or we won't use fsGroup any more?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, they both are used to control different part in security context

# A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change
# the ownership of that volume to be owned by the pod.
# Ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods
Expand Down Expand Up @@ -319,6 +325,9 @@ starrocksBeSpec:
serviceAccount: ""
# add annotations for be pods. example, if you want to config monitor for datadog, you can config the annotations.
annotations: {}
# If runAsNonRoot is true, the container is run as non-root user.
# The userId will be set to 1000, and the groupID will be set to 1000.
runAsNonRoot: false
# A special supplemental group that applies to all containers in a pod. Some volume types allow the Kubelet to change
# the ownership of that volume to be owned by the pod.
# Ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#configure-volume-permission-and-ownership-change-policy-for-pods
Expand Down
17 changes: 17 additions & 0 deletions pkg/apis/starrocks/v1/component_type.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ type SpecInterface interface {
GetHostAliases() []corev1.HostAlias
GetSchedulerName() string
GetFsGroup() *int64
GetRunAsNonRoot() (*int64, *int64)
GetAnnotations() map[string]string
}

Expand Down Expand Up @@ -84,6 +85,11 @@ type StarRocksComponentSpec struct {
// serviceAccount for access cloud service.
ServiceAccount string `json:"serviceAccount,omitempty"`

// RunAsGroup is used to determine whether to run starrocks as a normal user.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fix the comment, it is RunAsNonRoot, not RunAsGroup

// If RunAsGroup is true, operator will set RunAsGroup and RunAsGroup to 1000 in securityContext.
// default: nil
RunAsNonRoot *bool `json:"runAsNonRoot,omitempty"`

// A special supplemental group that applies to all containers in a pod.
// Some volume types allow the Kubelet to change the ownership of that volume
// to be owned by the pod:
Expand Down Expand Up @@ -385,3 +391,14 @@ func (spec *StarRocksComponentSpec) GetSchedulerName() string {
func (spec *StarRocksComponentSpec) GetFsGroup() *int64 {
return spec.FsGroup
}

func (spec *StarRocksComponentSpec) GetRunAsNonRoot() (*int64, *int64) {
runAsNonRoot := spec.RunAsNonRoot
if runAsNonRoot == nil || *runAsNonRoot == false {
return nil, nil
}

userId := int64(1000)
groupId := int64(1000)
return &userId, &groupId
}
5 changes: 5 additions & 0 deletions pkg/apis/starrocks/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 11 additions & 4 deletions pkg/k8sutils/templates/pod/spec.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -425,13 +425,20 @@ func PodSecurityContext(spec v1.SpecInterface) *corev1.PodSecurityContext {
return nil
}

func ContainerSecurityContext() *corev1.SecurityContext {
func ContainerSecurityContext(spec v1.SpecInterface) *corev1.SecurityContext {
userId, groupId := spec.GetRunAsNonRoot()

var runAsNonRoot *bool
if userId != nil && *userId != 0 {
b := true
runAsNonRoot = &b
}
return &corev1.SecurityContext{
RunAsUser: userId,
RunAsGroup: groupId,
RunAsNonRoot: runAsNonRoot,
AllowPrivilegeEscalation: func() *bool { b := false; return &b }(),
// starrocks will create pid file, eg.g /opt/starrocks/fe/bin/fe.pid, so set it to false
ReadOnlyRootFilesystem: func() *bool { b := false; return &b }(),
SeccompProfile: &corev1.SeccompProfile{
Type: corev1.SeccompProfileTypeUnconfined,
},
}
}
2 changes: 1 addition & 1 deletion pkg/sub_controller/be/be_pod.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ func (be *BeController) buildPodTemplate(src *srapi.StarRocksCluster, config map
LivenessProbe: pod.LivenessProbe(rutils.GetPort(config, rutils.WEBSERVER_PORT), pod.HEALTH_API_PATH),
ReadinessProbe: pod.ReadinessProbe(rutils.GetPort(config, rutils.WEBSERVER_PORT), pod.HEALTH_API_PATH),
Lifecycle: pod.LifeCycle("/opt/starrocks/be_prestop.sh"),
SecurityContext: pod.ContainerSecurityContext(),
SecurityContext: pod.ContainerSecurityContext(beSpec),
}
if beSpec.ConfigMapInfo.ConfigMapName != "" && beSpec.ConfigMapInfo.ResolveKey != "" {
beContainer.Env = append(beContainer.Env, corev1.EnvVar{
Expand Down
2 changes: 1 addition & 1 deletion pkg/sub_controller/cn/cn_pod.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ func (cc *CnController) buildPodTemplate(src *srapi.StarRocksCluster, config map
LivenessProbe: pod.LivenessProbe(rutils.GetPort(config, rutils.WEBSERVER_PORT), pod.HEALTH_API_PATH),
ReadinessProbe: pod.ReadinessProbe(rutils.GetPort(config, rutils.WEBSERVER_PORT), pod.HEALTH_API_PATH),
Lifecycle: pod.LifeCycle("/opt/starrocks/cn_prestop.sh"),
SecurityContext: pod.ContainerSecurityContext(),
SecurityContext: pod.ContainerSecurityContext(cnSpec),
}

if cnSpec.ConfigMapInfo.ConfigMapName != "" && cnSpec.ConfigMapInfo.ResolveKey != "" {
Expand Down
2 changes: 1 addition & 1 deletion pkg/sub_controller/fe/fe_pod.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,7 +89,7 @@ func (fc *FeController) buildPodTemplate(src *srapi.StarRocksCluster, config map
LivenessProbe: pod.LivenessProbe(rutils.GetPort(config, rutils.HTTP_PORT), pod.HEALTH_API_PATH),
ReadinessProbe: pod.ReadinessProbe(rutils.GetPort(config, rutils.HTTP_PORT), pod.HEALTH_API_PATH),
Lifecycle: pod.LifeCycle("/opt/starrocks/fe_prestop.sh"),
SecurityContext: pod.ContainerSecurityContext(),
SecurityContext: pod.ContainerSecurityContext(feSpec),
}

if feSpec.ConfigMapInfo.ConfigMapName != "" && feSpec.ConfigMapInfo.ResolveKey != "" {
Expand Down
Loading