Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: support webhook provider in Chart #4032

Merged
merged 19 commits into from
Jan 9, 2024
Merged

feat: support webhook provider in Chart #4032

Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
7e22345
BREAKING CHANGE: 💥 support webhook provider in Chart
mloiseleur Nov 10, 2023
9484b4c
add /healthz to webhook tutorial
mloiseleur Nov 10, 2023
07023d6
keep backward compatibility
mloiseleur Nov 11, 2023
7c093c5
moved images to values
mloiseleur Nov 18, 2023
b793897
Update charts/external-dns/values.yaml
mloiseleur Nov 27, 2023
185f7e2
fixed name for sidecar + doc update + externalVolumeMounts
mloiseleur Nov 27, 2023
9297b9b
add serviceMonitor endpoint, improve webhook provider tutorial and di…
mloiseleur Dec 2, 2023
9d1c0aa
doc: use helm-docs for README
mloiseleur Dec 7, 2023
c2f1aa4
fix rebase error
mloiseleur Dec 7, 2023
ea1e003
Apply suggestions from code review
mloiseleur Dec 8, 2023
b90ff49
introduce external-dns.webhookImage to match current image function
mloiseleur Dec 8, 2023
55156d6
fix port name of probes
mloiseleur Dec 8, 2023
1bdc4d7
update template with webhook provider support
mloiseleur Dec 8, 2023
fc94c26
Apply suggestions from code review
mloiseleur Dec 13, 2023
e423ca2
Update charts/external-dns/templates/deployment.yaml
mloiseleur Dec 18, 2023
bc69c01
Update charts/external-dns/templates/deployment.yaml
mloiseleur Dec 18, 2023
c5e7933
following review on provider.name doc
mloiseleur Jan 4, 2024
3e56c73
Merge branch 'master' into feat/webhook-chart
mloiseleur Jan 9, 2024
9299b59
remove secretConfiguration on webhook
mloiseleur Jan 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions charts/external-dns/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

- Added the option to explicitly enable or disable service account token automounting. ([#3983](https://github.com/kubernetes-sigs/external-dns/pull/3983)) [@gilles-gosuin](https://github.com/gilles-gosuin)
- Added the option to configure revisionHistoryLimit on the K8s Deployment resource. ([#4008](https://github.com/kubernetes-sigs/external-dns/pull/4008)) [@arnisoph](https://github.com/arnisoph)
- Added support for webhook providers, as a sidecar. ([#4032](https://github.com/kubernetes-sigs/external-dns/pull/4032) [@mloiseleur](https://github.com/mloiseleur)

## [v1.13.1] - 2023-09-07

Expand Down
19 changes: 15 additions & 4 deletions charts/external-dns/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ provider with all additional configuration being set via the `extraArgs` value.

| Provider | Supported |
|------------------------|------------|
| `webhook` | ❌ |
| `webhook` | ✅ |

## Namespaced Scoped Installation

Expand Down Expand Up @@ -108,18 +108,29 @@ If `namespaced` is set to `true`, please ensure that `sources` my only contains
| podSecurityContext | object | See _values.yaml_ | [Pod security context](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#podsecuritycontext-v1-core), this supports full customisation. |
| policy | string | `"upsert-only"` | How DNS records are synchronized between sources and providers; available values are `sync` & `upsert-only`. |
| priorityClassName | string | `nil` | Priority class name for the `Pod`. |
| provider.name | string | `"aws"` | _ExternalDNS_ provider name; for the available providers and how to configure them see the [README](https://github.com/kubernetes-sigs/external-dns#deploying-to-a-cluster). |
| provider.name | string | `"aws"` | _ExternalDNS_ provider name. When the provider is `webhook` it will add a sidecar using the `webhook` values. |
| provider.webhook.args | list | `[]` | Extra arguments to provide for the `webhook` container. |
| provider.webhook.env | list | `[]` | [Environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/) for the `webhook` container. |
| provider.webhook.extraVolumeMounts | list | `[]` | Extra [volume mounts](https://kubernetes.io/docs/concepts/storage/volumes/) for the `webhook` container. |
| provider.webhook.image.pullPolicy | string | `"IfNotPresent"` | Image pull policy for the `webhook` container. |
| provider.webhook.image.repository | string | `nil` | Image repository for the `webhook` container. |
| provider.webhook.image.tag | string | `nil` | Image tag for the `webhook` container. |
| provider.webhook.livenessProbe | object | See _values.yaml_ | [Liveness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `external-dns` container. |
| provider.webhook.readinessProbe | object | See _values.yaml_ | [Readiness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `webhook` container. |
| provider.webhook.resources | object | `{}` | [Resources](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) for the `webhook` container. |
| provider.webhook.securityContext | object | See _values.yaml_ | [Pod security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) for the `webhook` container. |
| provider.webhook.serviceMonitor | object | See _values.yaml_ | Optional [Service Monitor](https://prometheus-operator.dev/docs/operator/design/#servicemonitor) configuration for the `webhook` container. |
| rbac.additionalPermissions | list | `[]` | Additional rules to add to the `ClusterRole`. |
| rbac.create | bool | `true` | If `true`, create a `ClusterRole` & `ClusterRoleBinding` with access to the Kubernetes API. |
| readinessProbe | object | See _values.yaml_ | Readiness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `external-dns` container. |
| readinessProbe | object | See _values.yaml_ | [Readiness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `external-dns` container. |
| registry | string | `"txt"` | Specify the registry for storing ownership and labels. Valid values are `txt`, `aws-sd`, `dynamodb` & `noop`. |
| resources | object | `{}` | [Resources](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) for the `external-dns` container. |
| revisionHistoryLimit | int | `nil` | Specify the number of old `ReplicaSets` to retain to allow rollback of the `Deployment``. |
| secretConfiguration.data | object | `{}` | `Secret` data. |
| secretConfiguration.enabled | bool | `false` | If `true`, create a `Secret` to store sensitive provider configuration. |
| secretConfiguration.mountPath | string | `nil` | Mount path for the `Secret`, this can be templated. |
| secretConfiguration.subPath | string | `nil` | Sub-path for mounting the `Secret`, this can be templated. |
| securityContext | object | See _values.yaml_ | [Security context](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#securitycontext-v1-core) for the `external-dns` container. |
| securityContext | object | See _values.yaml_ | [Security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) for the `external-dns` container. |
| service.annotations | object | `{}` | Service annotations. |
| service.port | int | `7979` | Service HTTP port. |
| serviceAccount.annotations | object | `{}` | Annotations to add to the service account. |
Expand Down
2 changes: 1 addition & 1 deletion charts/external-dns/README.md.gotmpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ provider with all additional configuration being set via the `extraArgs` value.

| Provider | Supported |
|------------------------|------------|
| `webhook` | ❌ |
| `webhook` | ✅ |

## Namespaced Scoped Installation

Expand Down
12 changes: 12 additions & 0 deletions charts/external-dns/templates/_helpers.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,15 @@ Provider name, Keeps backward compatibility on provider
{{- .Values.provider.name }}
{{- end }}
{{- end }}

{{/*
The image to use for optional webhook sidecar
*/}}
{{- define "external-dns.webhookImage" -}}
{{- with .image }}
{{- if or (empty .repository) (empty .tag) }}
{{- fail "ERROR: webhook provider needs an image repository and a tag" }}
{{- end }}
{{- printf "%s:%s" .repository .tag }}
{{- end }}
{{- end }}
47 changes: 46 additions & 1 deletion charts/external-dns/templates/deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
{{- $providerName := include "external-dns.providerName" . }}
apiVersion: apps/v1
kind: Deployment
metadata:
Expand Down Expand Up @@ -103,7 +104,7 @@ spec:
{{- range .Values.domainFilters }}
- --domain-filter={{ . }}
{{- end }}
- --provider={{ include "external-dns.providerName" . }}
- --provider={{ $providerName }}
{{- range .Values.extraArgs }}
- {{ tpl . $ }}
{{- end }}
Expand Down Expand Up @@ -132,6 +133,50 @@ spec:
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- if eq $providerName "webhook" }}
{{- with .Values.provider.webhook }}
- name: webhook
image: {{ include "external-dns.webhookImage" . }}
imagePullPolicy: {{ $.Values.image.pullPolicy }}
{{- with .env }}
env:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .args }}
args:
- {{ toYaml . | nindent 12 }}
{{- end }}
ports:
- name: http-webhook
protocol: TCP
containerPort: 8888
mloiseleur marked this conversation as resolved.
Show resolved Hide resolved
- name: http-webhook-metrics
protocol: TCP
containerPort: 8080
livenessProbe:
{{- toYaml .livenessProbe | nindent 12 }}
readinessProbe:
{{- toYaml .readinessProbe | nindent 12 }}
{{- if or $.Values.secretConfiguration.enabled .extraVolumeMounts }}
volumeMounts:
{{- if $.Values.secretConfiguration.enabled }}
- name: secrets
mountPath: {{ tpl $.Values.secretConfiguration.mountPath $ }}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Secrets should not be shared across the main container and sidecar. They should only be readable to containers that need them in order to reduce the risk of compromise. Most of the time (when running a webhook provider) only the webhook sidecar will need secrets, though there are a few obscure sources that could need secrets.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of the time (when running a webhook provider) only the webhook sidecar will need secrets.

Oh That's right !
🤔 Wdyt then about mounting the secret only on external-dns for in-tree provider and only on the sidecar for webhook providers ?
cc @stevehipwell @mrueg

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would be strongly in favour of not supporting secretConfiguration for the webhook sidecar as secrets should preferably be managed outside of the Helm chart and then interacted with via the extraVolumes & extraVolumeMounts values.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cloudfoundry source takes a password, but I suppose that can only come in through args, not a mounted secret. So perhaps we could say it would be unlikely for a future source to require a mounted secret.

I wouldn't want to have secretConfiguration only work for in-tree providers. That would be an odd discontinuity.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added specific extraVolumeMounts for sidecar with 9109597

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't think we should be explicitly using secretConfiguration here, the secret COULD be mounted by the extraVolumeMounts but we should be recommending external secrets be used if needed.

{{- with $.Values.secretConfiguration.subPath }}
subPath: {{ tpl . $ }}
{{- end }}
{{- end }}
{{- with .extraVolumeMounts }}
{{- toYaml . | nindent 12 }}
{{- end }}
{{- end }}
{{- with .securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- end }}
{{- end }}

{{- if or .Values.secretConfiguration.enabled .Values.extraVolumes }}
volumes:
{{- if .Values.secretConfiguration.enabled }}
Expand Down
31 changes: 31 additions & 0 deletions charts/external-dns/templates/servicemonitor.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
{{- if .Values.serviceMonitor.enabled -}}
{{- $providerName := include "external-dns.providerName" . }}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
Expand Down Expand Up @@ -48,6 +49,36 @@ spec:
relabelings:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- if eq $providerName "webhook" }}
{{- with .Values.provider.webhook.serviceMonitor }}
- port: webhook-metrics
path: /metrics
{{- with .interval }}
interval: {{ . }}
{{- end }}
{{- with .scheme }}
scheme: {{ . }}
{{- end }}
{{- with .bearerTokenFile }}
bearerTokenFile: {{ . }}
{{- end }}
{{- with .tlsConfig }}
tlsConfig:
{{- toYaml .| nindent 8 }}
{{- end }}
{{- with .scrapeTimeout }}
scrapeTimeout: {{ . }}
{{- end }}
{{- with .metricRelabelings }}
metricRelabelings:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .relabelings }}
relabelings:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- end }}
{{- end }}
{{- with .Values.serviceMonitor.targetLabels }}
targetLabels:
{{- toYaml . | nindent 4 }}
Expand Down
57 changes: 54 additions & 3 deletions charts/external-dns/values.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ dnsPolicy:
# -- [Init containers](https://kubernetes.io/docs/concepts/workloads/pods/init-containers/) to add to the `Pod` definition.
initContainers: []

# -- [Security context](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#securitycontext-v1-core) for the `external-dns` container.
# -- [Security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) for the `external-dns` container.
# @default -- See _values.yaml_
securityContext:
privileged: false
Expand All @@ -115,7 +115,7 @@ livenessProbe:
failureThreshold: 2
successThreshold: 1

# -- Readiness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `external-dns` container.
# -- [Readiness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `external-dns` container.
# @default -- See _values.yaml_
readinessProbe:
httpGet:
Expand Down Expand Up @@ -213,8 +213,59 @@ txtSuffix:
domainFilters: []

provider:
# -- _ExternalDNS_ provider name; for the available providers and how to configure them see the [README](https://github.com/kubernetes-sigs/external-dns#deploying-to-a-cluster).
# -- _ExternalDNS_ provider name. When the provider is `webhook` it will add a sidecar using the `webhook` values.
mloiseleur marked this conversation as resolved.
Show resolved Hide resolved
name: aws
webhook:
image:
# -- (string) Image repository for the `webhook` container.
repository:
# -- (string) Image tag for the `webhook` container.
tag:
# -- Image pull policy for the `webhook` container.
pullPolicy: IfNotPresent
# -- [Environment variables](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/) for the `webhook` container.
env: []
# -- Extra arguments to provide for the `webhook` container.
args: []
# -- Extra [volume mounts](https://kubernetes.io/docs/concepts/storage/volumes/) for the `webhook` container.
extraVolumeMounts: []
# -- [Resources](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/) for the `webhook` container.
resources: {}
# -- [Pod security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/#set-the-security-context-for-a-container) for the `webhook` container.
# @default -- See _values.yaml_
securityContext: {}
# -- [Liveness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `external-dns` container.
# @default -- See _values.yaml_
livenessProbe:
httpGet:
path: /healthz
port: http-webhook
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 2
successThreshold: 1
# -- [Readiness probe](https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/) configuration for the `webhook` container.
# @default -- See _values.yaml_
readinessProbe:
httpGet:
path: /healthz
port: http-webhook
initialDelaySeconds: 5
periodSeconds: 10
timeoutSeconds: 5
failureThreshold: 6
successThreshold: 1
# -- Optional [Service Monitor](https://prometheus-operator.dev/docs/operator/design/#servicemonitor) configuration for the `webhook` container.
# @default -- See _values.yaml_
serviceMonitor:
interval:
scheme:
tlsConfig: {}
bearerTokenFile:
scrapeTimeout:
metricRelabelings: []
relabelings: []

# -- Extra arguments to provide to _ExternalDNS_.
extraArgs: []
Expand Down
18 changes: 13 additions & 5 deletions docs/tutorials/webhook-provider.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,18 +16,26 @@ Providers implementing the HTTP API have to keep in sync with changes to the JSO

The following table represents the methods to implement mapped to their HTTP method and route.

| Provider method | HTTP Method | Route |
| --- | --- | --- |
| Records | GET | /records |
| AdjustEndpoints | POST | /adjustendpoints |
| ApplyChanges | POST | /records |
| Provider method | HTTP Method | Route |
| --- | --- | --- |
| Records | GET | /records |
| AdjustEndpoints | POST | /adjustendpoints |
| ApplyChanges | POST | /records |
| K8s probe | GET | /healthz |

ExternalDNS will also make requests to the `/` endpoint for negotiation and for deserialization of the `DomainFilter`.

The server needs to respond to those requests by reading the `Accept` header and responding with a corresponding `Content-Type` header specifying the supported media type format and version.

The default recommended port is 8888, and should listen only on localhost (ie: only accessible for k8s probes and external-dns).

**NOTE**: only `5xx` responses will be retried and only `20x` will be considered as successful. All status codes different from those will be considered a failure on ExternalDNS's side.


## Metrics support

The metrics should listen ":8080" on `/metrics` following [Open Metrics](https://github.com/OpenObservability/OpenMetrics) format.

## Provider registry

To simplify the discovery of providers, we will accept pull requests that will add links to providers in the [README](../../README.md) file. This list will only serve the purpose of simplifying finding providers and will not constitute an official endorsement of any of the externally implemented providers unless otherwise stated.
Expand Down
Loading