Is there some way to make resource settings configurable within the provider?

[sharkymcdongles]

When flux is bootstrapped into our cluster we end up with OOM kills of the source controller pods. I see no way to adjust these settings dynamically when the provider is used.

[sharkymcdongles]

Since I only needed to adjust memory I did a hacky workaround to get through this issue:

# Apply manifests on the cluster
resource "kubectl_manifest" "apply" {
  for_each           = { for v in local.apply : lower(join("/", compact([v.data.apiVersion, v.data.kind, lookup(v.data.metadata, "namespace", ""), v.data.metadata.name]))) => v.content }
  yaml_body          = replace(each.value, "1Gi", "3Gi")
  override_namespace = local.flux_namespace
  depends_on = [
    module.add_rancher_cluster,
  ]
}

[onelapahead]

Based on fluxcd/source-controller#303 we decided to leave memory unbounded for the source-controller. This was achieved doing a hacky kubectl patch after the upstream manifests are applied:

# lastly, patch the source-controller to avoid OOMs
# source for kubectl patch: https://github.com/hashicorp/terraform-provider-kubernetes/issues/723#issuecomment-914593460
# source for leaving memory unbounded: https://github.com/fluxcd/source-controller/issues/303#issuecomment-784631181

resource "local_file" "config" {
  content  = ""
  filename = "${path.module}/.terraform/config.yaml"
}

resource "local_file" "kube_ca" {
  content  = base64decode(data.aws_eks_cluster.cluster.certificate_authority[0].data)
  filename = "${path.module}/.terraform/ca.crt"
}

resource "null_resource" "patch_source_controller" {
  depends_on = [kubectl_manifest.apply]

  triggers = {
    always_run = timestamp()
  }

  provisioner "local-exec" {
    command = "kubectl patch deploy -n flux-system source-controller --server=$KUBESERVER --token=$KUBETOKEN --certificate-authority=$KUBECA --type json  -p='[{\"op\": \"remove\", \"path\": \"/spec/template/spec/containers/0/resources/limits/memory\"}]'"

    environment = {
      KUBECONFIG = local_file.config.filename
      KUBESERVER = data.aws_eks_cluster.cluster.endpoint
      KUBETOKEN  = data.aws_eks_cluster_auth.cluster.token
      KUBECA     = local_file.kube_ca.filename
    }
  }
}

[Efrat19]

I used kustomize PatchesStrategicMerge to override the fields I wanted changed