Delete Consul custom resources with consul-k8s uninstall

[t-eckert]

Changes proposed in this PR

• All custom resources managed by Consul will be fetched before Helm uninstall is run.
• These custom resources will be sent a deletion request with a timeout of 1 second and 5 retries if the custom resources are not deleted.
• If custom resources remain after these 5 attempts, their finalizers will be patched to be an empty list, freeing them up to be deleted by Kubernetes.
• Changed timeout to be a duration var instead of a string that gets parsed to a duration.
• Fixed the "header" when installing.

How I've tested this PR

• Unit tests to cover CR fetching, deletion, and patching.
• Manually tested that CRs are successfully deleted in the following scenarios:
  • Consul deployed with controller.
  • Consul deployed with controller, all Consul servers deleted (making CR deletion impossible) and falling back to patching finalizers to get rid of CRs
  • Consul deployed without controller (but CRs exist from an earlier installation). Deletion of CRs fails without a controller and uninstall falls back to patching finalizers.

There is a case that this does not cover, but I feel it is too much of an edge case to go after. That is, the user deploys Consul with a controller, the user manually deletes the controller deployment. The user does not delete the mutatingwebhookconfiguration for the controller. This will cause the uninstall request to fail. If the user deletes mutatingwebhookconfiguration for the controller, uninstall will succeed.

How I expect reviewers to test this PR:

A good, simple run manual test is

Deploy Consul with this configuration:

  global:
    name: consul
    logLevel: debug
  controller:
    enabled: true
  server:
    replicas: 1

Then create a resource that will create a custom resource:

# server.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: server
spec:
  replicas: 3
  selector:
    matchLabels:
      app: server
  template:
    metadata:
      name: server
      labels:
        app: server
      annotations:
        consul.hashicorp.com/connect-inject: "true"
    spec:
      containers:
        - name: server
          image: docker.mirror.hashicorp.services/hashicorp/http-echo:latest
          args:
            - -text="hello world"
            - -listen=:8080
          ports:
            - containerPort: 8080
              name: http
          resources:
            requests:
              memory: "64Mi"
              cpu: "250m"
            limits:
              memory: "128Mi"
              cpu: "500m"
      serviceAccountName: server
      terminationGracePeriodSeconds: 0
---
apiVersion: v1
kind: Service
metadata:
  name: server
spec:
  selector:
    app: server
  ports:
    - name: http
      port: 80
      targetPort: 8080
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: server
---
apiVersion: consul.hashicorp.com/v1alpha1
kind: ServiceDefaults
metadata:
  name: server
spec:
  protocol: http

Then perform consul-k8s uninstall using this branch's CLI.

Checklist

• Tests added
• CHANGELOG entry added

  HashiCorp engineers only, community PRs should not add a changelog entry.
  Entries should use present tense (e.g. Add support for...)

[jmurret]

nice work. I have some nits and just the mention of those two remaining uninstall tests that should validate the new output. approving and will let you figure that all out. 🎉

[ishustava]

Great work on this, Thomas! I'm leaving some comments in-line, but I don't think they are too significant. Good work!

[ishustava]

Great work!