Defer initialisation of kubectl provider

[ddelange]

Scenario

Provision a kubernetes cluster - and apply some default configuration to it using kubectl - in the same terraform module

Example use-case: eddycharly/terraform-provider-kops#438 (comment)

Problem

Supplying dynamic (computed during apply) values to a provider is currently not supported (ref hashicorp/terraform#27934), AKA "provision a thing, then provision another thing inside the thing you just provisioned" ref hashicorp/terraform#2430 (comment)

Workaround

• Avoid providing dynamic value to provider: pass a constant filename
• Lazy loading provider: e.g. providing config_path to the helm provider will only initialise when it's needed for a helm_release resource ref Configuring one provider with a dynamic attribute from another (was: depends_on for providers) hashicorp/terraform#2430 (comment)
• Add appropriate depends_on to all its resources to ensure proper creation and destruction order

Expected behaviour

kubectl provider only initialises once the first kubectl_manifest is applied (at this point in time, the kubernetes cluster is running and the kubeconfig file now exists/is updated on disk)

Actual behaviour

kubectl_manifest resources will fail to apply on the first run:

failed to create kubernetes rest client for update of resource: Get "http://localhost/api?timeout=32s": dial tcp [::1]:80: connect: connection refused

Related issues

• PostgreSQL provider is not using lazy connections hashicorp/terraform-provider-postgresql#140

[stevehipwell]

@ddelange you need to use the exec plugin authentication for Kubernetes or other kubernetes based providers such as helm or kubectl, AFAIK this is the only really reliable way to authenticate against Kubernetes.

[ddelange]

Thanks for the info, will close with that 💥