diff --git a/docs/bundle-secrets.md b/docs/bundle-secrets.md new file mode 100644 index 00000000..2dea083f --- /dev/null +++ b/docs/bundle-secrets.md @@ -0,0 +1,172 @@ +# Bundle Secrets Injection + +Ways of injecting secrets when deploying applications with Timoni [Bundles](bundle.md): + +- Using [runtime](#runtime-secrets) attributes `@timoni(runtime:string:SECRET-NAME)`. +- Using [SOPS](#sops-secrets) encrypted YAML or JSON files. + +## Runtime Secrets + +To showcase how secrets injections works, we'll assume we are deploying an application +that connects to an S3-compatible API and needs two secrets: `ACCESS_KEY` and `SECRET_KEY`. + +### Injecting secrets from CI secret store + +When using a CI runner to deploy apps with Timoni, +we can pass secrets from the runner's secret store to Timoni's Bundle. + +Example of a Bundle that contains runtime attributes: + +```cue +bundle: { + apiVersion: "v1alpha1" + name: "my-app" + instances: { + "my-app-storage": { + module: url: "oci://my-registry/timoni/modules/my-app-storage" + namespace: "my-app" + values: { + endpoint: "https://my-acc.r2.cloudflarestorage.com" + accessKey: string @timoni(runtime:string:ACCESS_KEY) + secretKey: string @timoni(runtime:string:SECRET_KEY) + } + } + } +} +``` + +In a GitHub workflow, we can map secrets from GitHub secrets to env vars, +that Timoni will use at apply-time: + +```shell +export ACCESS_KEY=${{ secrets.ACCESS_KEY }} +export SECRET_KEY=${{ secrets.SECRET_KEY }} + +timoni bundle apply -f bundle.cue --runtime-from-env +``` + +### Injecting secrets from Kubernetes + +The same secrets from the above example, can be injected from a Kubernetes Secret, +assuming we're using some external-secret controller that syncs secrets from a Vault in etcd. + +Example of a Timoni [Bundle Runtime](bundle-runtime.md) that fetches the secrets from the cluster: + +```cue +runtime: { + apiVersion: "v1alpha1" + name: "production" + values: [ + { + query: "k8s:v1:Secret:my-namespace:my-secret-name" + for: { + "ACCESS_KEY": "obj.data.r2_access_key" + "SECRET_KEY": "obj.data.r2_secret_key" + } + }, + ] +} +``` + +At apply-time we pass the runtime definition and +Timoni will read the secrets from the Kubernetes cluster +and use them when applying the bundle: + +```shell +timoni bundle apply -f bundle.cue --runtime runtime.cue +``` + +### Secrets interpolation + +When the secrets stored in external system do not map to a value key in the Bundle, +we can use CUE's string interpolation to compose the desired value. + +Assuming an application config expects a Redis URL, but the secret store contains +`REDIS_HOST` and `REDIS_PASS`. + +```cue +bundle: { + apiVersion: "v1alpha1" + name: "podinfo" + _secrets: { + host: string @timoni(runtime:string:REDIS_HOST) + password: string @timoni(runtime:string:REDIS_PASS) + } + instances: { + "podinfo-backend": { + module: url: "oci://ghcr.io/stefanprodan/modules/podinfo" + namespace: "podinfo" + values: caching: { + enabled: true + redisURL: "tcp://:\(_secrets.password)@\(_secrets.host):6379" + } + } + } +} +``` + +In the above example, we define a CUE hidden filed `_secrets`, where we set the +runtime secrets mappings. Then in the instance values, we use string interpolation +to set the `redisURL` containing the secrets. + +Using the build command, we can see the URL value set in the podinfo container args: + +```console +$ export REDIS_HOST=redis.svc +$ export REDIS_PASS=testpass +$ timoni bundle build -f bundle.cue --runtime-from-env | grep redis + - --cache-server=tcp://:testpass@redis.svc:6379 +``` + +## SOPS secrets + +When using [SOPS](https://github.com/getsops/sops), +we can decrypt the secrets and pipe +those values to env vars, then use `--runtime-from-env`. + +Another option is to extract the secret values of a Timoni Bundle to an YAML or JSON file, +that we encrypt/decrypt with SOPS. + +### Injecting secrets from SOPS + +Main bundle file `bundle.main.cue`: + +```cue +bundle: { + apiVersion: "v1alpha1" + name: "my-app" + instances: { + "my-app-storage": { + module: url: "oci://my-registry/timoni/modules/my-app-storage" + namespace: "my-app" + values: { + endpoint: "https://my-acc.r2.cloudflarestorage.com" + // The secrets are omitted here! + } + } + } +} +``` + +Bundle partial in YAML format `bundle.secret.yaml`: + +```yaml +bundle: + instances: + my-app-storage: + values: + accessKey: ENC[AES256_GCM,data:..] + secretKey: ENC[AES256_GCM,data:..] +``` + +Assuming the `bundle.secret.yaml` file is kept encrypted with SOPS, +at apply-time we can run the SOPS decryption, +and pass the plain YAML to Timoni's apply command like so: + +```shell +sops -d bundle.secret.yaml > bundle.secret.plain.yaml + +timoni bundle apply -f bundle.main.cue -f bundle.secret.plain.yaml + +rm bundle.secret.plain.yaml +``` diff --git a/docs/bundle.md b/docs/bundle.md index c55cb49e..bf3ac17c 100644 --- a/docs/bundle.md +++ b/docs/bundle.md @@ -116,240 +116,7 @@ Build the Bundle and print the resulting Kubernetes resources for all the Bundle name: redis namespace: podinfo --- - apiVersion: v1 - data: - redis.conf: | - maxmemory 256mb - maxmemory-policy allkeys-lru - - dir /data - save "" - appendonly yes - - protected-mode no - rename-command CONFIG "" - kind: ConfigMap - metadata: - labels: - app.kubernetes.io/part-of: redis - app.kubernetes.io/version: 7.2.2 - name: redis - namespace: podinfo - --- - apiVersion: v1 - kind: Service - metadata: - labels: - app.kubernetes.io/name: redis-master - app.kubernetes.io/part-of: redis - app.kubernetes.io/version: 7.2.2 - name: redis - namespace: podinfo - spec: - ports: - - name: redis - port: 6379 - protocol: TCP - targetPort: redis - selector: - app.kubernetes.io/name: redis-master - type: ClusterIP - --- - apiVersion: v1 - kind: Service - metadata: - labels: - app.kubernetes.io/name: redis-replica - app.kubernetes.io/part-of: redis - app.kubernetes.io/version: 7.2.2 - name: redis-readonly - namespace: podinfo - spec: - ports: - - name: redis - port: 6379 - protocol: TCP - targetPort: redis - selector: - app.kubernetes.io/name: redis-replica - type: ClusterIP - --- - apiVersion: apps/v1 - kind: Deployment - metadata: - labels: - app.kubernetes.io/name: redis-master - app.kubernetes.io/part-of: redis - app.kubernetes.io/version: 7.2.2 - name: redis-master - namespace: podinfo - spec: - selector: - matchLabels: - app.kubernetes.io/name: redis-master - strategy: - type: Recreate - template: - metadata: - labels: - app.kubernetes.io/name: redis-master - spec: - containers: - - command: - - redis-server - - /redis-master/redis.conf - image: cgr.dev/chainguard/redis@sha256:9cdc90a57fa0cc23dce9a934313cc5412a3b8415a60e79797ee9cb4ca04a3968 - imagePullPolicy: IfNotPresent - livenessProbe: - initialDelaySeconds: 2 - tcpSocket: - port: redis - timeoutSeconds: 2 - name: redis - ports: - - containerPort: 6379 - name: redis - protocol: TCP - readinessProbe: - exec: - command: - - redis-cli - - ping - initialDelaySeconds: 2 - timeoutSeconds: 5 - resources: - limits: - memory: 288Mi - requests: - memory: 64Mi - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: - - ALL - readOnlyRootFilesystem: true - runAsNonRoot: true - seccompProfile: - type: RuntimeDefault - volumeMounts: - - mountPath: /data - name: data - - mountPath: /redis-master - name: config - securityContext: - fsGroup: 1001 - runAsGroup: 1001 - runAsUser: 1001 - serviceAccountName: redis - volumes: - - name: data - persistentVolumeClaim: - claimName: redis-master - - configMap: - items: - - key: redis.conf - path: redis.conf - name: redis - name: config - --- - apiVersion: apps/v1 - kind: Deployment - metadata: - labels: - app.kubernetes.io/name: redis-replica - app.kubernetes.io/part-of: redis - app.kubernetes.io/version: 7.2.2 - name: redis-replica - namespace: podinfo - spec: - replicas: 1 - selector: - matchLabels: - app.kubernetes.io/name: redis-replica - strategy: - type: RollingUpdate - template: - metadata: - labels: - app.kubernetes.io/name: redis-replica - spec: - containers: - - command: - - redis-server - - --replicaof - - redis.podinfo.svc.cluster.local - - "6379" - - --include - - /redis-replica/redis.conf - image: cgr.dev/chainguard/redis@sha256:9cdc90a57fa0cc23dce9a934313cc5412a3b8415a60e79797ee9cb4ca04a3968 - imagePullPolicy: IfNotPresent - livenessProbe: - initialDelaySeconds: 2 - tcpSocket: - port: redis - timeoutSeconds: 2 - name: redis - ports: - - containerPort: 6379 - name: redis - protocol: TCP - readinessProbe: - exec: - command: - - redis-cli - - ping - initialDelaySeconds: 2 - timeoutSeconds: 5 - resources: - limits: - memory: 288Mi - requests: - memory: 64Mi - securityContext: - allowPrivilegeEscalation: false - capabilities: - drop: - - ALL - readOnlyRootFilesystem: true - runAsNonRoot: true - seccompProfile: - type: RuntimeDefault - volumeMounts: - - mountPath: /data - name: data - - mountPath: /redis-replica - name: config - securityContext: - fsGroup: 1001 - runAsGroup: 1001 - runAsUser: 1001 - serviceAccountName: redis - volumes: - - emptyDir: {} - name: data - - configMap: - items: - - key: redis.conf - path: redis.conf - name: redis - name: config - --- - apiVersion: v1 - kind: PersistentVolumeClaim - metadata: - labels: - app.kubernetes.io/part-of: redis - app.kubernetes.io/version: 7.2.2 - name: redis-master - namespace: podinfo - spec: - accessModes: - - ReadWriteOnce - resources: - requests: - storage: 8Gi - storageClassName: standard - + # Redis deployments omitted for brevity --- # Instance: podinfo --- @@ -362,63 +129,7 @@ Build the Bundle and print the resulting Kubernetes resources for all the Bundle name: podinfo namespace: podinfo --- - apiVersion: v1 - kind: Service - metadata: - labels: - app.kubernetes.io/name: podinfo - app.kubernetes.io/version: 6.5.2 - name: podinfo - namespace: podinfo - spec: - ports: - - name: http - port: 80 - protocol: TCP - targetPort: http - selector: - app.kubernetes.io/name: podinfo - type: ClusterIP - --- - apiVersion: apps/v1 - kind: Deployment - metadata: - labels: - app.kubernetes.io/name: podinfo - app.kubernetes.io/version: 6.5.2 - name: podinfo - namespace: podinfo - spec: - replicas: 1 - selector: - matchLabels: - app.kubernetes.io/name: podinfo - template: - metadata: - labels: - app.kubernetes.io/name: podinfo - spec: - containers: - - command: - - ./podinfo - - --level=info - - --cache-server=tcp://redis:6379 - image: ghcr.io/stefanprodan/podinfo:6.5.2 - imagePullPolicy: IfNotPresent - livenessProbe: - httpGet: - path: /healthz - port: http - name: podinfo - ports: - - containerPort: 9898 - name: http - protocol: TCP - readinessProbe: - httpGet: - path: /readyz - port: http - serviceAccountName: podinfo + # Podinfo deployment omitted for brevity ``` List the managed resources from a bundle and their rollout status: @@ -464,7 +175,7 @@ List the instances in Bundle `podinfo` across all namespaces: ```text NAME NAMESPACE MODULE VERSION LAST APPLIED BUNDLE podinfo podinfo oci://ghcr.io/stefanprodan/modules/podinfo 6.5.2 2023-09-10T16:20:07Z podinfo - redis podinfo oci://ghcr.io/stefanprodan/modules/redis 7.2.2 2023-09-10T16:20:00Z podinfo + redis podinfo oci://ghcr.io/stefanprodan/modules/redis 7.2.2 2023-09-10T16:20:00Z podinfo ``` ## Writing a Bundle spec diff --git a/mkdocs.yml b/mkdocs.yml index 610bd56f..277209f8 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -94,6 +94,7 @@ nav: - Bundle files: bundle.md - Bundle runtime: bundle-runtime.md - Bundle distribution: bundle-distribution.md + - Bundle secrets injection: bundle-secrets.md - Module Development: - Module structure: module.md - Module distribution: module-distribution.md