authenticator_groups_config forces cluster replacement

[joe-a-t]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Terraform v0.15.3
Google provider v3.85.0

Affected Resource(s)

• google_container_cluster

Terraform Configuration Files

resource "google_container_cluster" "primary" {
  name     = "my-gke-cluster"
  location = "us-central1"

  remove_default_node_pool = true
  initial_node_count       = 1

  # Uncomment after the cluster already exists and change the domain
  # authenticator_groups_config {
  #   security_group = "[email protected]"
  # }
}

Debug Output

Panic Output

Expected Behavior

Based on https://cloud.google.com/kubernetes-engine/docs/how-to/role-based-access-control#groups-create-cluster, it should update existing clusters without forcing re-creation.

Actual Behavior

It forces recreation of the cluster.

Steps to Reproduce

1. terraform apply
2. Uncomment the authenticator_groups_config block and change the domain name
3. terraform plan/apply

Important Factoids

It is a fairly recent development that GKE allows updating existing clusters to enable this feature, previously it was only possible to enable to feature by replacing the cluster.

For anyone else seeing this issue, the current workaround is to enable Google Groups for RBAC in the UI.

References

• #0000

b/299312181

[rileykarson]

Retriaging to enhancement, not supporting update is considered a feature request and not a bug.

[Feggah]

I've experienced a similar problem, but for me it was when trying to update the security_group value.

      ~ authenticator_groups_config {
          - security_group = "[email protected]" -> null # forces replacement
        }

I couldn't reproduce the problem commenting the block as @joe-a-t did, because for me Terraform isn't detecting this change. I'm receiving "Infrastructure is up to date" even with the whole block commented.

I can update the security_group value through the GCP interface and it does not enforces replacement, so this is not behaving as expected.

Investigation

I've spend some time digging into this problem and I realized that when I manually disable the Google Groups for RBAC option in the GCP interface, it actually does not remove the security_group value, I noticed this because when I tried to enable it again, it automatically filled the security_group value that I've set before.

I suspected that there could be another attribute that could set this config to enabled/disabled.

Digging into this provider code, I noticed that the Go struct that represents the block authenticator_groups_config has a field named Enabled and its value is hard coded to true:

terraform-provider-google/google/resource_container_cluster.go

Lines 2728 to 2734 in 3dc7a34

	result := &container.AuthenticatorGroupsConfig{}
	config := l[0].(map[string]interface{})
	if securityGroup, ok := config["security_group"]; ok {
	result.Enabled = true
	result.SecurityGroup = securityGroup.(string)
	}
	return result

I was wondering if the GCP interface instead of literally removing the whole configuration of Google Groups for RBAC, actually just set this Enabled attribute to false and the security_group value wasn't even modified.

Can we partially solve this feature/bug by just removing this hard coded true from the code and exposing to the provider's users the Enabled field, so they can enable/disable it, @rileykarson ? Notice that I wrote "partially" because in the GCP interface I can update the security_group value without needing to recreate the GKE cluster.

I'm not familiar at all with the terraform implementation, but if you can guide me through what I need to do to fix this issue, I can try to contribute.

Problems encountered

1. We can't change the security_group value without recreating the cluster through Terraform.
2. We can't configure the attribute enabled to enable/disable the Google Groups for RBAC configuration
3. Terraform does not detect when I remove the whole authenticator_groups_config block, it actually returns "Infrastructure is up to date", when it should've returned that it needs to remove the configuration.

I imagine that exposing the Enabled attribute would solve the second problem for the users, but how can we update the security_group value without recreating the cluster? Where in the code it determines if the resource needs to be replaced or not?

How can we solve the 3rd problem?

[rd-michel]

i would also suggest to add a new enable/disable option for "Google Groups for RBAC" because not the feature itself is the problem here ... but the value of the security group.

i can enable/disable "Google Groups for RBAC" without any problems. its the change of the security-group field that forces me to replace the cluster

~ authenticator_groups_config {
~ security_group = "[email protected]" -> "[email protected]" # forces replacement
}

[caiodelgadonew]

This also happens when you change or add cluster_dns

@rileykarson to me it does not looks like a feature, it looks like a bug, because you can go to the UI and make it happen, but when you do it on the code it does not work.

So, the workaround is:

1. set the configuration in the google UI
2. set the terraform code

And then when you apply it, it says:

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # google_container_cluster.kubernetes_1 has changed
  ~ resource "google_container_cluster" "kubernetes_1" {
        id                          = "kubernetes-staging-1"
        name                        = "kubernetes-staging-1"
        # (28 unchanged attributes hidden)


      + authenticator_groups_config {
          + security_group = "gke-security-groups@<REDACTED>"
        }

So... with that workaround it is possible to make the code match it.

I would expect the provider to reflect the capability of the API.

[DanishKhakwani]

It seems like a bug. I am trying to enable the group rbac feature on currently running cluster and it shows the cluster replacement.

      + authenticator_groups_config { # forces replacement
          + security_group = "[email protected]" # forces replacement
        }

[derek-burdick]

#11720 potential fix. Have not had time to test yet. Hoping to verify in next 72 hours.

[DanishKhakwani]

Is the fix available now? @slevenick
@derek-burdick not sure why the PR was not merged.

[DavidMij]

any updates?