Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch to distroless consul-dataplane image with Envoy 1.24 #1676

Merged
merged 23 commits into from
Nov 10, 2022
Merged

Switch to distroless consul-dataplane image with Envoy 1.24 #1676

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
9a24f80
Pass proxy service ID as a path
boxofrad Nov 2, 2022
09e1f25
Do not run consul-dataplane in a shell
boxofrad Nov 2, 2022
e815b5e
fix silly addresses quoting problem
boxofrad Nov 3, 2022
333a246
use my debug distroless build
boxofrad Nov 3, 2022
a1ff509
Merge remote-tracking branch 'origin/main' into boxofrad/distroless
Nov 8, 2022
48471cf
switch to pglass dataplane distroless image
Nov 7, 2022
601da15
Use -proxy-service-id-path for gateways
Nov 8, 2022
b9105f5
Bump consul-server-connection-manager to 0.1.0
Nov 8, 2022
be6c453
Cleanup consul servers in server-acl-init test
Nov 8, 2022
18c999b
update changelog. switch to hashicorppreview/consul-dataplane image
Nov 8, 2022
332be21
Merge remote-tracking branch 'origin/main' into boxofrad/distroless
Nov 8, 2022
05ea688
Remove shell usage for consul-dataplane in gateways
Nov 9, 2022
5e95a02
Remove global.imageConsulDataplane setting from metrics test
Nov 9, 2022
cf9ddda
Test with ghcr.io/pglass/consul-dataplane:distroless
Nov 9, 2022
df3e4f6
Merge remote-tracking branch 'origin/main' into boxofrad/distroless
Nov 9, 2022
ead35d7
Fix bats tests for gateways / consul-dataplane
Nov 9, 2022
fafc958
Fix lint
Nov 10, 2022
1af9104
Merge remote-tracking branch 'origin/main' into boxofrad/distroless
Nov 10, 2022
950f45d
Reformat consul-dataplane command/args for gateways. Fix some tests
Nov 10, 2022
c6bb41e
Merge remote-tracking branch 'origin/main' into boxofrad/distroless
Nov 10, 2022
7db34d6
Unquote -telemetry-prom-scrap-path
Nov 10, 2022
6b55e70
Merge remote-tracking branch 'origin/main' into boxofrad/distroless
Nov 10, 2022
823c89e
Switch back to hashicorppreview/consul-dataplane:1.0-dev
Nov 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## UNRELEASED

BREAKING_CHANGES:
BREAKING CHANGES:
pglass marked this conversation as resolved.
Show resolved Hide resolved
* CLI:
* Change default behavior of `consul-k8s install` to perform the installation when no answer is provided to the prompt. [[GH-1673](https://github.com/hashicorp/consul-k8s/pull/1673)]
* Helm:
Expand All @@ -21,7 +21,7 @@ BREAKING_CHANGES:
* Require `meshGateway.enabled` when peering is enabled. [[GH-1683](https://github.com/hashicorp/consul-k8s/pull/1683)]

FEATURES:
* Consul-dataplane:
* Consul Dataplane:
* Support merged metrics with consul-dataplane. [[GH-1635](https://github.com/hashicorp/consul-k8s/pull/1635)]
* Support transparent proxying when using consul-dataplane. [[GH-1625](https://github.com/hashicorp/consul-k8s/pull/1478),[GH-1632](https://github.com/hashicorp/consul-k8s/pull/1632)]
* Enable sync-catalog to only talk to Consul servers. [[GH-1659](https://github.com/hashicorp/consul-k8s/pull/1659)]
Expand All @@ -42,6 +42,8 @@ IMPROVEMENTS:
* API Gateway: Create PodSecurityPolicy for controller when `global.enablePodSecurityPolicies=true`. [[GH-1656](https://github.com/hashicorp/consul-k8s/pull/1656)]
* API Gateway: Create PodSecurityPolicy and allow controller to bind it to ServiceAccounts that it creates for Gateway Deployments when `global.enablePodSecurityPolicies=true`. [[GH-1672](https://github.com/hashicorp/consul-k8s/pull/1672)]
* Deploy `expose-servers` service only when Admin Partitions(ENT) is enabled. [[GH-1683](https://github.com/hashicorp/consul-k8s/pull/1683)]
* Use a distroless image for `consul-dataplane`. [[GH-1676](https://github.com/hashicorp/consul-k8s/pull/1676)]
* The Envoy version is now 1.24.0 for `consul-dataplane`. [[GH-1676](https://github.com/hashicorp/consul-k8s/pull/1676)]

BUG FIXES:
* Peering
Expand Down
5 changes: 2 additions & 3 deletions acceptance/tests/metrics/metrics_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,9 +100,8 @@ func TestAppMetrics(t *testing.T) {
ns := ctx.KubectlOptions(t).Namespace

helmValues := map[string]string{
"global.datacenter": "dc1",
"global.metrics.enabled": "true",

"global.datacenter": "dc1",
"global.metrics.enabled": "true",
"connectInject.enabled": "true",
"connectInject.metrics.defaultEnableMerging": "true",
}
Expand Down
113 changes: 59 additions & 54 deletions charts/consul/templates/ingress-gateways-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,9 @@ spec:
resources: {{ toYaml (default $defaults.resources .resources) | nindent 10 }}
{{- end }}
volumeMounts:
- name: consul-service
mountPath: /consul/service
readOnly: true
{{- if and $root.Values.global.tls.enabled (not (and $root.Values.externalServers.enabled $root.Values.externalServers.useSystemRoots)) }}
- name: consul-ca-cert
mountPath: /consul/tls/ca
Expand All @@ -245,65 +248,67 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: DP_ENVOY_READY_BIND_ADDRESS
valueFrom:
fieldRef:
fieldPath: status.podIP
- name: DP_CREDENTIAL_LOGIN_META1
value: pod=$(NAMESPACE)/$(POD_NAME)
- name: DP_CREDENTIAL_LOGIN_META2
value: component=ingress-gateway
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because there is no shell in the distroless image, we can't read environment variables within the command. Instead, we set env vars that consul-dataplane understands.

Depends on hashicorp/consul-dataplane#47

- name: DP_SERVICE_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
command:
- /bin/sh
- -ec
- |
consul-dataplane \
-envoy-ready-bind-address=$POD_IP \
-envoy-ready-bind-port=21000 \
{{- if $root.Values.externalServers.enabled }}
-addresses={{ $root.Values.externalServers.hosts | first | quote }} \
{{- else }}
-addresses="{{ template "consul.fullname" $root }}-server.{{ $root.Release.Namespace }}.svc" \
{{- end }}
{{- if $root.Values.externalServers.enabled }}
-grpc-port={{ $root.Values.externalServers.grpcPort }} \
{{- else }}
-grpc-port=8502 \
{{- end }}
-proxy-service-id=$POD_NAME \
-service-node-name=$DP_SERVICE_NODE_NAME \
{{- if $root.Values.global.enableConsulNamespaces }}
-service-namespace={{ (default $defaults.consulNamespace .consulNamespace) }} \
{{- end }}
{{- if and $root.Values.global.tls.enabled }}
{{- if (not (and $root.Values.externalServers.enabled $root.Values.externalServers.useSystemRoots)) }}
-ca-certs=/consul/tls/ca/tls.crt \
{{- end }}
{{- if and $root.Values.externalServers.enabled $root.Values.externalServers.tlsServerName }}
-tls-server-name={{ $root.Values.externalServers.tlsServerName }} \
{{- else if $root.Values.global.cloud.enabled }}
-tls-server-name=server.{{ $root.Values.global.datacenter}}.{{ $root.Values.global.domain}} \
{{- end }}
{{- else }}
-tls-disabled \
{{- end }}
{{- if $root.Values.global.acls.manageSystemACLs }}
-credential-type=login \
-login-bearer-token-path=/var/run/secrets/kubernetes.io/serviceaccount/token \
-login-meta=component=ingress-gateway \
-login-meta=pod=${NAMESPACE}/${POD_NAME} \
-login-auth-method={{ template "consul.fullname" $root }}-k8s-component-auth-method \
{{- if $root.Values.global.adminPartitions.enabled }}
-login-partition={{ $root.Values.global.adminPartitions.name }} \
{{- end }}
{{- end }}
{{- if $root.Values.global.adminPartitions.enabled }}
-service-partition={{ $root.Values.global.adminPartitions.name }} \
{{- end }}
-log-level={{ default $root.Values.global.logLevel }} \
-log-json={{ $root.Values.global.logJSON }} \
{{- if (and $root.Values.global.metrics.enabled $root.Values.global.metrics.enableGatewayMetrics) }}
-telemetry-prom-scrape-path="/metrics"
{{- end }}
{{- if and $root.Values.externalServers.enabled $root.Values.externalServers.skipServerWatch }}
-server-watch-disabled=true
{{- end }}
- consul-dataplane
args:
- -envoy-ready-bind-port=21000
{{- if $root.Values.externalServers.enabled }}
- -addresses={{ $root.Values.externalServers.hosts | first }}
{{- else }}
- -addresses={{ template "consul.fullname" $root }}-server.{{ $root.Release.Namespace }}.svc
{{- end }}
{{- if $root.Values.externalServers.enabled }}
- -grpc-port={{ $root.Values.externalServers.grpcPort }}
{{- else }}
- -grpc-port=8502
{{- end }}
- -proxy-service-id-path=/consul/service/proxy-id
{{- if $root.Values.global.enableConsulNamespaces }}
- -service-namespace={{ (default $defaults.consulNamespace .consulNamespace) }}
{{- end }}
{{- if and $root.Values.global.tls.enabled }}
{{- if (not (and $root.Values.externalServers.enabled $root.Values.externalServers.useSystemRoots)) }}
- -ca-certs=/consul/tls/ca/tls.crt
{{- end }}
{{- if and $root.Values.externalServers.enabled $root.Values.externalServers.tlsServerName }}
- -tls-server-name={{ $root.Values.externalServers.tlsServerName }}
{{- else if $root.Values.global.cloud.enabled }}
- -tls-server-name=server.{{ $root.Values.global.datacenter}}.{{ $root.Values.global.domain}}
{{- end }}
{{- else }}
- -tls-disabled
{{- end }}
{{- if $root.Values.global.acls.manageSystemACLs }}
- -credential-type=login
- -login-bearer-token-path=/var/run/secrets/kubernetes.io/serviceaccount/token
- -login-auth-method={{ template "consul.fullname" $root }}-k8s-component-auth-method
{{- if $root.Values.global.adminPartitions.enabled }}
- -login-partition={{ $root.Values.global.adminPartitions.name }}
{{- end }}
{{- end }}
{{- if $root.Values.global.adminPartitions.enabled }}
- -service-partition={{ $root.Values.global.adminPartitions.name }}
{{- end }}
- -log-level={{ default $root.Values.global.logLevel }}
- -log-json={{ $root.Values.global.logJSON }}
{{- if (and $root.Values.global.metrics.enabled $root.Values.global.metrics.enableGatewayMetrics) }}
- -telemetry-prom-scrape-path=/metrics
{{- end }}
{{- if and $root.Values.externalServers.enabled $root.Values.externalServers.skipServerWatch }}
- -server-watch-disabled=true
{{- end }}
livenessProbe:
tcpSocket:
port: 21000
Expand Down
107 changes: 53 additions & 54 deletions charts/consul/templates/mesh-gateway-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,65 +195,64 @@ spec:
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: DP_CREDENTIAL_LOGIN_META1
value: pod=$(NAMESPACE)/$(POD_NAME)
- name: DP_CREDENTIAL_LOGIN_META2
value: component=mesh-gateway
- name: DP_SERVICE_NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
command:
- /bin/sh
- -ec
- |
consul-dataplane \
{{- if .Values.externalServers.enabled }}
-addresses={{ .Values.externalServers.hosts | first | quote }} \
{{- else }}
-addresses="{{ template "consul.fullname" . }}-server.{{ .Release.Namespace }}.svc" \
{{- end }}
{{- if .Values.externalServers.enabled }}
-grpc-port={{ .Values.externalServers.grpcPort }} \
{{- else }}
-grpc-port=8502 \
{{- end }}
-proxy-service-id=$POD_NAME \
-service-node-name=$DP_SERVICE_NODE_NAME \
{{- if .Values.global.tls.enabled }}
{{- if (not (and .Values.externalServers.enabled .Values.externalServers.useSystemRoots)) }}
-ca-certs=/consul/tls/ca/tls.crt \
{{- end }}
{{- if and .Values.externalServers.enabled .Values.externalServers.tlsServerName }}
-tls-server-name={{.Values.externalServers.tlsServerName }} \
{{- else if .Values.global.cloud.enabled }}
-tls-server-name=server.{{ .Values.global.datacenter}}.{{ .Values.global.domain}} \
{{- end }}
{{- else }}
-tls-disabled \
{{- end }}
{{- if .Values.global.acls.manageSystemACLs }}
-credential-type=login \
-login-bearer-token-path=/var/run/secrets/kubernetes.io/serviceaccount/token \
-login-meta=component=mesh-gateway \
-login-meta=pod=${NAMESPACE}/${POD_NAME} \
{{- if and .Values.global.federation.enabled .Values.global.federation.primaryDatacenter }}
-login-auth-method={{ template "consul.fullname" . }}-k8s-component-auth-method-{{ .Values.global.datacenter }} \
-login-datacenter={{ .Values.global.federation.primaryDatacenter }} \
{{- else }}
-login-auth-method={{ template "consul.fullname" . }}-k8s-component-auth-method \
{{- end }}
{{- if .Values.global.adminPartitions.enabled }}
-login-partition={{ .Values.global.adminPartitions.name }} \
{{- end }}
{{- end }}
{{- if .Values.global.adminPartitions.enabled }}
-service-partition={{ .Values.global.adminPartitions.name }} \
{{- end }}
-log-level={{ default .Values.global.logLevel }} \
-log-json={{ .Values.global.logJSON }} \
{{- if (and .Values.global.metrics.enabled .Values.global.metrics.enableGatewayMetrics) }}
-telemetry-prom-scrape-path="/metrics"
{{- end }}
{{- if and .Values.externalServers.enabled .Values.externalServers.skipServerWatch }}
-server-watch-disabled=true
{{- end }}
- consul-dataplane
args:
{{- if .Values.externalServers.enabled }}
- -addresses={{ .Values.externalServers.hosts | first }}
{{- else }}
- -addresses={{ template "consul.fullname" . }}-server.{{ .Release.Namespace }}.svc
{{- end }}
{{- if .Values.externalServers.enabled }}
- -grpc-port={{ .Values.externalServers.grpcPort }}
{{- else }}
- -grpc-port=8502
{{- end }}
- -proxy-service-id-path=/consul/service/proxy-id
{{- if .Values.global.tls.enabled }}
{{- if (not (and .Values.externalServers.enabled .Values.externalServers.useSystemRoots)) }}
- -ca-certs=/consul/tls/ca/tls.crt
{{- end }}
{{- if and .Values.externalServers.enabled .Values.externalServers.tlsServerName }}
- -tls-server-name={{.Values.externalServers.tlsServerName }}
{{- else if .Values.global.cloud.enabled }}
- -tls-server-name=server.{{ .Values.global.datacenter}}.{{ .Values.global.domain}}
{{- end }}
{{- else }}
- -tls-disabled
{{- end }}
{{- if .Values.global.acls.manageSystemACLs }}
- -credential-type=login
- -login-bearer-token-path=/var/run/secrets/kubernetes.io/serviceaccount/token
{{- if and .Values.global.federation.enabled .Values.global.federation.primaryDatacenter }}
- -login-auth-method={{ template "consul.fullname" . }}-k8s-component-auth-method-{{ .Values.global.datacenter }}
- -login-datacenter={{ .Values.global.federation.primaryDatacenter }}
{{- else }}
- -login-auth-method={{ template "consul.fullname" . }}-k8s-component-auth-method
{{- end }}
{{- if .Values.global.adminPartitions.enabled }}
- -login-partition={{ .Values.global.adminPartitions.name }}
{{- end }}
{{- end }}
{{- if .Values.global.adminPartitions.enabled }}
- -service-partition={{ .Values.global.adminPartitions.name }}
{{- end }}
- -log-level={{ default .Values.global.logLevel }}
- -log-json={{ .Values.global.logJSON }}
{{- if (and .Values.global.metrics.enabled .Values.global.metrics.enableGatewayMetrics) }}
- -telemetry-prom-scrape-path=/metrics
{{- end }}
{{- if and .Values.externalServers.enabled .Values.externalServers.skipServerWatch }}
- -server-watch-disabled=true
{{- end }}
livenessProbe:
tcpSocket:
port: {{ .Values.meshGateway.containerPort }}
Expand Down
Loading