Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make service mirror controller per target cluster #4710

Merged
merged 15 commits into from
Jul 23, 2020
Merged
2 changes: 2 additions & 0 deletions bin/helm-build
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ bindir=$( cd "${BASH_SOURCE[0]%/*}" && pwd )
rootdir=$( cd "$bindir"/.. && pwd )

"$bindir"/helm lint "$rootdir"/charts/linkerd2-multicluster
"$bindir"/helm lint "$rootdir"/charts/linkerd2-multicluster-link
"$bindir"/helm lint "$rootdir"/charts/partials
"$bindir"/helm dep up "$rootdir"/charts/linkerd2-cni
"$bindir"/helm lint "$rootdir"/charts/linkerd2-cni
Expand Down Expand Up @@ -50,6 +51,7 @@ if [ "$1" = package ]; then
"$bindir"/helm --version "$version" --app-version "$tag" -d "$rootdir"/target/helm package "$rootdir"/charts/linkerd2
"$bindir"/helm --version "$version" --app-version "$tag" -d "$rootdir"/target/helm package "$rootdir"/charts/linkerd2-cni
"$bindir"/helm --version "$version" --app-version "$tag" -d "$rootdir"/target/helm package "$rootdir"/charts/linkerd2-multicluster
"$bindir"/helm --version "$version" --app-version "$tag" -d "$rootdir"/target/helm package "$rootdir"/charts/linkerd2-multicluster-link
mv "$rootdir"/target/helm/index-pre.yaml "$rootdir"/target/helm/index-pre-"$version".yaml
"$bindir"/helm repo index --url "https://helm.linkerd.io/$repo/" --merge "$rootdir"/target/helm/index-pre-"$version".yaml "$rootdir"/target/helm

Expand Down
22 changes: 22 additions & 0 deletions charts/linkerd2-multicluster-link/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
OWNERS
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*~
# Various IDEs
.project
.idea/
*.tmproj
7 changes: 7 additions & 0 deletions charts/linkerd2-multicluster-link/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
apiVersion: v1
appVersion: edge-XX.X.X
description: A helm chart containing the resources to enable mirroring of services from a remote cluster
kubeVersion: ">=1.13.0-0"
icon: https://linkerd.io/images/logo-only-200h.png
name: "linkerd2-multicluster-link"
version: 0.1.0
40 changes: 40 additions & 0 deletions charts/linkerd2-multicluster-link/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@

# Linkerd2-multicluster-link Helm Chart

Linkerd is a *service mesh*, designed to give platform-wide observability,
reliability, and security without requiring configuration or code changes. This
chart provides the components needed to enable communication between clusters.

## Configuration

The following table lists the configurable parameters of the
linkerd2-multicluster chart and their default values.

| Parameter | Description | Default |
|---------------------------------|---------------------------------------------------------------------------------------------|----------------------------------------------|
|`controllerComponentLabel` | Control plane label. Do not edit |`linkerd.io/control-plane-component` |
|`controllerImage` | Docker image for the Service mirror component (uses the Linkerd controller image) |`gcr.io/linkerd-io/controller` |
|`controllerImageVersion` | Tag for the Service Mirror container Docker image |`latest version` |
|`createdByAnnotation` | Annotation label for the proxy create. Do not edit. |`linkerd.io/created-by` |
|`gateway` | If the gateway component should be installed |`true` |
|`gatewayLocalProbePath` | The path that will be used by the local liveness checks to ensure the gateway is alive |`/health-local` |
|`gatewayLocalProbePort` | The port that will be used by the local liveness checks to ensure the gateway is alive |`8888` |
|`gatewayName` | The name of the gateway that will be installed |`linkerd-gateway` |
|`gatewayNginxImage` | The Nginx image |`nginx` |
|`gatewayNginxImageVersion` | The version of the Nginx image |`1.17` |
|`gatewayPort` | The port on which all the gateway will accept incoming traffic |`4143` |
|`gatewayProbePath` | The path that will be used by remote clusters for determining whether the gateway is alive |`/health` |
|`gatewayProbePort` | The port used for liveliness probing |`4181` |
|`gatewayProbeSeconds` | The interval (in seconds) between liveness probes |`3` |
|`identityTrustDomain` | Trust domain used for identity of the existing linkerd installation |`cluster.local` |
|`installNamespace` | If the namespace should be installed |`true` |
|`linkerdNamespace` | The namespace of the existing Linkerd installation |`linkerd` |
|`linkerdVersion` | Control plane version | latest version |
|`namespace` | Service Mirror component namespace |`linkerd-multicluster` |
|`proxyOutboundPort` | The port on which the proxy accepts outbound traffic |`4140` |
|`remoteMirrorServiceAccountName` | The name of the service account used to allow remote clusters to mirror local services |`linkerd-service-mirror-remote-access-default`|
|`remoteMirrorServiceAccount` | If the remote mirror service account should be installed |`true` |
|`serviceMirror` | If the service mirror component should be installed |`true` |
|`logLevel` | Log level for the Multicluster components |`info` |
|`serviceMirrorRetryLimit` | Number of times update from the remote cluster is allowed to be requeued (retried) |`3` |
|`serviceMirrorUID` | User id under which the Service Mirror shall be ran |`2103` |
14 changes: 14 additions & 0 deletions charts/linkerd2-multicluster-link/templates/gateway-mirror.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
apiVersion: v1
kind: Service
metadata:
name: probe-gateway-{{.Values.targetClusterName}}
namespace: {{.Values.namespace}}
labels:
mirror.linkerd.io/mirrored-gateway: "true"
mirror.linkerd.io/cluster-name: {{.Values.targetClusterName}}
spec:
ports:
- name: mc-probe
port: {{.Values.gatewayProbePort}}
protocol: TCP
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
{{if .Values.serviceMirror -}}
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: linkerd-service-mirror-access-local-resources
name: linkerd-service-mirror-access-local-resources-{{.Values.targetClusterName}}
labels:
{{.Values.controllerComponentLabel}}: linkerd-service-mirror
mirror.linkerd.io/cluster-name: {{.Values.targetClusterName}}
rules:
- apiGroups: [""]
resources: ["endpoints", "services"]
Expand All @@ -17,85 +17,96 @@ rules:
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: linkerd-service-mirror-access-local-resources
name: linkerd-service-mirror-access-local-resources-{{.Values.targetClusterName}}
labels:
{{.Values.controllerComponentLabel}}: linkerd-service-mirror
mirror.linkerd.io/cluster-name: {{.Values.targetClusterName}}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: linkerd-service-mirror-access-local-resources
name: linkerd-service-mirror-access-local-resources-{{.Values.targetClusterName}}
subjects:
- kind: ServiceAccount
name: linkerd-service-mirror
name: linkerd-service-mirror-{{.Values.targetClusterName}}
namespace: {{.Values.namespace}}
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: linkerd-service-mirror-read-remote-creds
name: linkerd-service-mirror-read-remote-creds-{{.Values.targetClusterName}}
namespace: {{.Values.namespace}}
labels:
{{.Values.controllerComponentLabel}}: linkerd-service-mirror
mirror.linkerd.io/cluster-name: {{.Values.targetClusterName}}
rules:
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["cluster-credentials-{{.Values.targetClusterName}}"]
verbs: ["list", "get", "watch"]
- apiGroups: ["multicluster.linkerd.io"]
resources: ["links"]
verbs: ["list", "get", "watch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: linkerd-service-mirror-read-remote-creds
name: linkerd-service-mirror-read-remote-creds-{{.Values.targetClusterName}}
namespace: {{.Values.namespace}}
labels:
{{.Values.controllerComponentLabel}}: linkerd-service-mirror
mirror.linkerd.io/cluster-name: {{.Values.targetClusterName}}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: linkerd-service-mirror-read-remote-creds
name: linkerd-service-mirror-read-remote-creds-{{.Values.targetClusterName}}
subjects:
- kind: ServiceAccount
name: linkerd-service-mirror
name: linkerd-service-mirror-{{.Values.targetClusterName}}
namespace: {{.Values.namespace}}
---
kind: ServiceAccount
apiVersion: v1
metadata:
name: linkerd-service-mirror
name: linkerd-service-mirror-{{.Values.targetClusterName}}
namespace: {{.Values.namespace}}
labels:
{{.Values.controllerComponentLabel}}: linkerd-service-mirror
mirror.linkerd.io/cluster-name: {{.Values.targetClusterName}}
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
{{.Values.controllerComponentLabel}}: linkerd-service-mirror
name: linkerd-service-mirror
mirror.linkerd.io/cluster-name: {{.Values.targetClusterName}}
name: linkerd-service-mirror-{{.Values.targetClusterName}}
namespace: {{.Values.namespace}}
spec:
replicas: 1
selector:
matchLabels:
{{.Values.controllerComponentLabel}}: linkerd-service-mirror
mirror.linkerd.io/cluster-name: {{.Values.targetClusterName}}
template:
metadata:
annotations:
linkerd.io/inject: enabled
labels:
{{.Values.controllerComponentLabel}}: linkerd-service-mirror
mirror.linkerd.io/cluster-name: {{.Values.targetClusterName}}
spec:
containers:
- args:
- service-mirror
- -log-level={{.Values.logLevel}}
- -event-requeue-limit={{.Values.serviceMirrorRetryLimit}}
- -namespace={{.Values.namespace}}
- {{.Values.targetClusterName}}
image: {{.Values.controllerImage}}:{{.Values.controllerImageVersion}}
name: service-mirror
securityContext:
runAsUser: {{.Values.serviceMirrorUID}}
ports:
- containerPort: 9999
name: admin-http
serviceAccountName: linkerd-service-mirror
{{end -}}
serviceAccountName: linkerd-service-mirror-{{.Values.targetClusterName}}
9 changes: 9 additions & 0 deletions charts/linkerd2-multicluster-link/values.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
controllerComponentLabel: linkerd.io/control-plane-component
controllerImage: gcr.io/linkerd-io/controller
controllerImageVersion: linkerdVersionValue
createdByAnnotation: linkerd.io/created-by
gatewayProbePort: 4181
namespace: linkerd-multicluster
logLevel: info
serviceMirrorRetryLimit: 3
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it is worth getting rid of this retry mechanism alltogether. Frankly in practice I have not seen it pull its weight. I think this was a bit of overengineering on my end. WDYT?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this refactor mostly didn't touch the event processing loop. we can think about removing it, but I don't think that change has to be tied to this refactor.

serviceMirrorUID: 2103
4 changes: 2 additions & 2 deletions charts/linkerd2-multicluster/Chart.yaml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
apiVersion: v1
appVersion: edge-XX.X.X
description: A helm chart containing the resources to enable mirroring of services on remote clusters
description: A helm chart containing the resources to support multicluster linking to remote clusters
kubeVersion: ">=1.13.0-0"
icon: https://linkerd.io/images/logo-only-200h.png
name: "linkerd2-multicluster"
version: 0.1.0
version: 0.1.0
22 changes: 22 additions & 0 deletions charts/linkerd2-multicluster/templates/link-crd.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
###
### Link CRD
###
---
apiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: links.multicluster.linkerd.io
annotations:
{{.Values.createdByAnnotation}}: {{default (printf "linkerd/helm %s" .Values.linkerdVersion) .Values.cliVersion}}
spec:
group: multicluster.linkerd.io
versions:
- name: v1alpha1
served: true
storage: true
scope: Namespaced
names:
plural: links
singular: link
kind: Link
9 changes: 0 additions & 9 deletions charts/linkerd2-multicluster/values.yaml
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
controllerComponentLabel: linkerd.io/control-plane-component
controllerImage: gcr.io/linkerd-io/controller
controllerImageVersion: linkerdVersionValue
createdByAnnotation: linkerd.io/created-by
gateway: true
gatewayLocalProbePath: /health-local
Expand All @@ -12,15 +9,9 @@ gatewayPort: 4143
gatewayProbePath: /health
gatewayProbePort: 4181
gatewayProbeSeconds: 3
identityTrustDomain: cluster.local
installNamespace: true
linkerdNamespace: linkerd
linkerdVersion: linkerdVersionValue
namespace: linkerd-multicluster
proxyOutboundPort: 4140
serviceMirror: true
logLevel: info
serviceMirrorRetryLimit: 3
serviceMirrorUID: 2103
remoteMirrorServiceAccount: true
remoteMirrorServiceAccountName: linkerd-service-mirror-remote-access-default
3 changes: 2 additions & 1 deletion charts/linkerd2/templates/_config.tpl
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@
},
"autoInjectContext": null,
"omitWebhookSideEffects": {{.Values.omitWebhookSideEffects}},
"clusterDomain": "{{.Values.global.clusterDomain}}"
"clusterDomain": "{{.Values.global.clusterDomain}}",
"enableEndpointSlices": "{{.Values.global.enableEndpointSlices}}"
}
{{- end -}}

Expand Down
2 changes: 2 additions & 0 deletions charts/linkerd2/templates/destination-rbac.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,11 @@ rules:
- apiGroups: ["split.smi-spec.io"]
resources: ["trafficsplits"]
verbs: ["list", "get", "watch"]
{{- if .Values.global.enableEndpointSlices }}
- apiGroups: ["discovery.k8s.io"]
resources: ["endpointslices"]
verbs: ["list", "get", "watch"]
{{- end }}
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
Expand Down
5 changes: 5 additions & 0 deletions charts/linkerd2/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ global:

namespace: linkerd

# enables the use of EndpointSlice informers for the destination service;
# enableEndpointSlices should be set to true only if EndpointSlice K8s feature gate is on;
# the feature is still experimental.
enableEndpointSlices: false

identityTrustAnchorsPEM: |

identityTrustDomain: *cluster_domain
Expand Down
5 changes: 1 addition & 4 deletions cli/cmd/check.go
Original file line number Diff line number Diff line change
Expand Up @@ -188,8 +188,7 @@ func configureAndRunChecks(wout io.Writer, werr io.Writer, stage string, options
}
checks = append(checks, healthcheck.LinkerdCNIPluginChecks)
checks = append(checks, healthcheck.LinkerdHAChecks)
checks = append(checks, healthcheck.LinkerdMulticlusterSourceChecks)
checks = append(checks, healthcheck.LinkerdMulticlusterTargetChecks)
checks = append(checks, healthcheck.LinkerdMulticlusterChecks)

checks = append(checks, healthcheck.AddOnCategories...)
}
Expand All @@ -208,8 +207,6 @@ func configureAndRunChecks(wout io.Writer, werr io.Writer, stage string, options
RetryDeadline: time.Now().Add(options.wait),
CNIEnabled: options.cniEnabled,
InstallManifest: installManifest,
SourceCluster: options.multicluster,
TargetCluster: options.multicluster,
})

success := runChecks(wout, werr, hc, options.output)
Expand Down
Loading