Helm 3 fails to install CRDs

[nicklasfrahm]

Describe the bug
If you install the helm chart from the stable repository via helm 3.0.0, the CRDs will not be installed.

To Reproduce

$ helm install ambassador stable/ambassador --namespace ambassador
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
NAME: ambassador
LAST DEPLOYED: Mon Nov 25 11:38:09 2019
NAMESPACE: ambassador
STATUS: deployed
REVISION: 1
NOTES:
Congratulations! You've successfully installed Ambassador.

For help, visit our Slack at https://d6e.co/slack or view the documentation online at https://www.getambassador.io.

To get the IP address of Ambassador, run the following commands:
NOTE: It may take a few minutes for the LoadBalancer IP to be available.
     You can watch the status of by running 'kubectl get svc -w  --namespace ambassador ambassador'

  On GKE/Azure:
  export SERVICE_IP=$(kubectl get svc --namespace ambassador ambassador -o jsonpath='{.status.loadBalancer.ingress[0].ip}')

  On AWS:
  export SERVICE_IP=$(kubectl get svc --namespace ambassador ambassador -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')

  echo http://$SERVICE_IP:

$ helm ls --all-namespaces
NAME            NAMESPACE               REVISION        UPDATED                                 STATUS          CHART                   APP VERSION
ambassador      ambassador              1               2019-11-25 11:38:09.9841677 +0100 CET   deployed        ambassador-4.4.7        0.85.0

No CRDs are registered as the following command does not return anything:

$ kubectl get crd --all-namespaces | grep ambassador

Expected behavior

CRDs should be installed and the following command should show at least a single CRD:

$ kubectl get crd --all-namespaces | grep ambassador
NAME                                                         CREATED AT
something.getambassador.io                                   2019-11-17T13:12:50Z

Versions (please complete the following information):

• Ambassador (chart: 4.4.7, app version: 0.85.0)
• Kubernetes environment (Baremetal via Rancher Kubernetes Engine)
• Version (v1.15.5)

Additional context
This bug is tracked in the helm/charts#19008 already, but I wanted to make sure that you guys are aware as the helm chart is currently unusable.

[containerpope]

Helm 3 deprecated the hook for crds and has now a policy that there should be a crd directory

https://v3.helm.sh/docs/topics/charts_hooks/

[SupremeTechnopriest]

Just hit this as well. Any time estimate on a fix?

[kflynn]

The current chart (at stable/ambassador) should work for both Helm 2 and Helm 3. You'll still see the "unknown hook" error with Helm 3, but it will create the CRDs anyway. Let me know if that's your situation...

[iNoahNothing]

This should be resolved as of Nov 29th 2019 with helm/charts#19225. Make sure you helm repo update to pull the latest chart.

Please reopen this issue if you see different behavior.

[bcorijn]

I still (or again?) see these errors when installing from the datawire repo with Helm 3. Should I re-create an issue over there or can this be reused?

[iNoahNothing]

@bcorijn you will see the manifest_sorter errors because the CRDs are annotated with the CRD hook for Helm 2 support.

This hook has been deprecated in Helm 3 so it is throwing an error but the chart should still be installing correctly. Can you confirm the chart installs in your cluster?

[dreic]

I can confirm, error is present in helm 3.

[hayorov]

As for Feb 4, 2020:

• helm version.BuildInfo{Version:"v3.0.2", GitCommit:"19e47ee3283ae98139d98460de796c1be1e3975f", GitTreeState:"clean", GoVersion:"go1.13.5"}
• ArgoCD chart argo-cd-1.7.0 app version 1.4.1

Error presents:
❯ h install argo argo/argo-cd -n argocd
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
manifest_sorter.go:175: info: skipping unknown hook: "crd-install"
NAME: argo
LAST DEPLOYED: Tue Feb 4 14:18:37 2020
NAMESPACE: argocd
STATUS: deployed
REVISION: 1
TEST SUITE: None

[Mokto]

Just hit this issue with helm 3.1.1.

Note: this happens only for me with helm template. (crd are not generated)

helm install works properly, with warnings.

[richarddli]

Per the documentation (https://www.getambassador.io/user-guide/helm/) this is an issue with Helm 3 and it can be ignored. This is not a bug that can be addressed in Ambassador.

[jonathanmccracken]

would it be possible to avoid these warnings if the helm chart used the CRD folder for Helm3, while still supporting the crd-hooks for Helm v2?

I understand that these are warnings only, but ideally, the install can run clean for Helm v3 users. In my case, I am running this helm through AWS CDK and it causes a failure in the stack (this is an issue with Amazon's CDK), but if this chart did not throw this warning it would work fine and comply with the Helm V3 specification.

[zsjohny]

Method 1: Let helm Do It For You
With the arrival of Helm 3, we removed the old crd-install hooks for a more simple methodology. There is now a special directory called crds that you can create in your chart to hold your CRDs. These CRDs are not templated, but will be installed by default when running a helm install for the chart. If the CRD already exists, it will be skipped with a warning. If you wish to skip the CRD installation step, you can pass the --skip-crds flag.

Some caveats (and explanations)
There is no support at this time for upgrading or deleting CRDs using Helm. This was an explicit decision after much community discussion due to the danger for unintentional data loss. Furthermore, there is currently no community consensus around how to handle CRDs and their lifecycle. As this evolves, Helm will add support for those use cases.

The --dry-run flag of helm install and helm upgrade is not currently supported for CRDs. The purpose of "Dry Run" is to validate that the output of the chart will actually work if sent to the server. But CRDs are a modification of the server's behavior. Helm cannot install the CRD on a dry run, so the discovery client will not know about that Custom Resource (CR), and validation will fail. You can alternatively move the CRDs to their own chart or use helm template instead.

Another important point to consider in the discussion around CRD support is how the rendering of templates is handled. One of the distinct disadvantages of the crd-install method used in Helm 2 was the inability to properly validate charts due to changing API availability (a CRD is actually adding another available API to your Kubernetes cluster). If a chart installed a CRD, helm no longer had a valid set of API versions to work against. This is also the reason behind removing templating support from CRDs. With the new crds method of CRD installation, we now ensure that helm has completely valid information about the current state of the cluster.

Method 2: Separate Charts
Another way to do this is to put the CRD definition in one chart, and then put any resources that use that CRD in another chart.

In this method, each chart must be installed separately. However, this workflow may be more useful for cluster operators who have admin access to a cluster