Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for cfg via env vars and define external volumes #601

Merged
merged 4 commits into from
Jun 9, 2022

Conversation

mszostok
Copy link
Contributor

@mszostok mszostok commented May 31, 2022

ISSUE TYPE
  • Feature Pull Request
SUMMARY
  • Allows providing communicator configuration via env variables
  • Env variables have higher priority that config from file
  • Helm chart has:
    • extraEnv
    • extraVolumeMounts
    • extraVolumes

Fixes #480

Related documentation: kubeshop/botkube-docs#82

TESTING

Unit test proves that the reading configuration works as expected. However, below you will find an e2e tutorial.

BotKube with Vault via CSI driver

  1. Create K8s cluster, e.g. k3s via lima-vm: limactl start template://k3s

    NOTE: The CSI needs to be supported, on k3d is problematic: [BUG] Fail to start longhorn with k3d k3d-io/k3d#206. Alternative is to just not play with the CSI driver and create your own volume that will be mounted, e.g. with predefined secret.

  2. Install Vault:

    helm repo add hashicorp https://helm.releases.hashicorp.com
    helm repo update
    helm install vault hashicorp/vault \
        --set "server.dev.enabled=true" \
        --set "injector.enabled=false" \
        --set "csi.enabled=true"
  3. Set Slack token:

    kubectl exec -it vault-0 -- /bin/sh
    vault kv put secret/slack token={token}
  4. Configure Kubernetes authentication:

    vault auth enable kubernetes
    vault write auth/kubernetes/config \
        kubernetes_host="https://$KUBERNETES_PORT_443_TCP_ADDR:443"
    vault policy write internal-app - <<EOF
    path "secret/data/slack" {
      capabilities = ["read"]
    }
    EOF
    vault write auth/kubernetes/role/database \
        bound_service_account_names=botkube-sa \
        bound_service_account_namespaces=default \
        policies=internal-app \
        ttl=20m
  5. Install the secrets store CSI driver:

    helm repo add secrets-store-csi-driver https://kubernetes-sigs.github.io/secrets-store-csi-driver/charts
    helm install csi secrets-store-csi-driver/secrets-store-csi-driver --set syncSecret.enabled=true
  6. Create install parameters:

    cat > /tmp/values.yaml << ENDOFFILE
    extraObjects:
      - apiVersion: secrets-store.csi.x-k8s.io/v1
        kind: SecretProviderClass
        metadata:
          name: vault-database
        spec:
          provider: vault
          secretObjects:
            - data:
                - key: token
                  objectName: "slack-token"
              secretName: communication-slack
              type: Opaque
          parameters:
            vaultAddress: "http://vault.default:8200"
            roleName: "database"
            objects: |
              - objectName: "slack-token"
                secretPath: "secret/data/slack"
                secretKey: "token"
    
    communications:
      # Settings for Slack
      slack:
        enabled: true
        channel: 'random'
        notiftype: short
        # token - specified via env
    
    extraEnv:
      - name: COMMUNICATION_SLACK_TOKEN
        valueFrom:
          secretKeyRef:
            name: communication-slack
            key: token
    
    extraVolumeMounts:
      - name: secrets-store-inline
        mountPath: "/mnt/secrets-store"
        readOnly: true
    
    extraVolumes:
      - name: secrets-store-inline
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: "vault-database"
    image:
      registry: mszostok
      repository: botkube
      tag: env-test-v2
    ENDOFFILE
  7. Checkout this PR: gh pr checkout 601

  8. Install BotKube:

    helm install botkube -f /tmp/values.yaml ./helm/botkube

@mszostok mszostok added enhancement New feature or request helm-chart Related to helm chart improvements labels May 31, 2022
@pkosiec pkosiec self-assigned this Jun 1, 2022
@pkosiec pkosiec self-requested a review June 1, 2022 12:45
@PrasadG193
Copy link
Collaborator

Is there an issue to track docs changes?

@PrasadG193 PrasadG193 requested a review from bhavin192 June 2, 2022 05:30
Copy link
Member

@pkosiec pkosiec left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works well 👍 Just one minor comment

pkg/config/config.go Outdated Show resolved Hide resolved
helm/botkube/values.yaml Show resolved Hide resolved
Copy link
Contributor

@bhavin192 bhavin192 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just looking at the Helm chart. Apart from the comment about extraEnv and few minor ones, rest chart changes look good!

helm/botkube/templates/deployment.yaml Outdated Show resolved Hide resolved
helm/botkube/values.yaml Outdated Show resolved Hide resolved
helm/botkube/values.yaml Outdated Show resolved Hide resolved
helm/botkube/values.yaml Show resolved Hide resolved
helm/botkube/values.yaml Outdated Show resolved Hide resolved
helm/botkube/values.yaml Outdated Show resolved Hide resolved
helm/botkube/values.yaml Outdated Show resolved Hide resolved
@mszostok
Copy link
Contributor Author

mszostok commented Jun 3, 2022

Hi @bhavin192!

Thanks for the review 👍 Here is the commit with applied changes: 32cb93c

Currently, I'm working on a document as mentioned in #601 (comment).

EDIT: PR for docs is ready: kubeshop/botkube-docs#82


Regarding README.md for Helm Chart, it would be nice to have it automatically generated, e.g. via helm-docs, or readme-generator-for-helm. For now, I described only properties that I added on this PR. However, I spot that the table is not complete and also contains removed properties too, e.g. ingress.urlPath

I created an issue to track that: #609

@mszostok
Copy link
Contributor Author

mszostok commented Jun 6, 2022

@PrasadG193

Is there an issue to track docs changes?

Docs changes are handled in the same issue, here is the related PR: kubeshop/botkube-docs#82

Here is also a recording that shows how it works: https://www.loom.com/share/37f6a60f88194b55916ee256a534336c

@pierluigilenoci
Copy link
Contributor

@mszostok could you please resolve the conflicts?

@mszostok mszostok added the breaking Contains breaking change label Jun 9, 2022
@mergify mergify bot merged commit ab86a23 into kubeshop:develop Jun 9, 2022
@mszostok mszostok deleted the support-env-and-vols branch June 9, 2022 09:32
mergify bot pushed a commit that referenced this pull request Jun 21, 2022
##### ISSUE TYPE

 - Feature Pull Request

##### SUMMARY

Add an option to push the BotKube image automatically on PR. It's alternative approach for #604.

This PR will solve the problem with manual PR builds, e.g. we had that issue here:
- #601
- #593
- #582
- #583

Example run: https://github.com/mszostok/botkube/runs/6714112689?check_suite_focus=true

Fixes #590 

To ensure that secrets won't be available for untrusted code, first we need to build the image and share it with the second job, which doesn't check out the untrusted code and can safely push an artifact to ghcr.io.

The flow is as follows:
```
Job1: image build -> image save -> artifact upload 
Job2: artifact download -> image load -> image push
```
Job1—runs untrusted code but without write repo perms
Job2—push built image with package write perms

#### Security

This article describes it well: https://securitylab.github.com/research/github-actions-preventing-pwn-requests/
@mszostok mszostok removed the breaking Contains breaking change label Aug 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request helm-chart Related to helm chart improvements
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Configure tokens and credentials from secrets/envs
5 participants