azurerm_key_vault data source causes Terraform to think its ID has changed

[MarcDufresne]

Hello, I have an issue with the azurerm_key_vault_secret resource. Basically, Terraform detects that the Key Vault ID has changed, however it did not, and wants to delete my secrets to recreate them in the "new" key vault.

The Kay Vault itself has been created in another module earlier, and this doesn't trigger anything in Terraform. I am using the data provider azurerm_key_vault to get my previously created key vault, I am thinking this might be the cause. That, or I am somehow doing something wrong.

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

Terraform v0.12.24

• provider.azurerm v2.11.0

Affected Resource(s)

• azurerm_key_vault_secret
• data.azurerm_key_vault

Terraform Configuration Files

# This key vault was created earlier in another module and is unchanged
data "azurerm_key_vault" "keyvault" {
  name                = var.keyvault_name  # output from the module that creates the Key Vault
  resource_group_name = var.res_group_name
}

resource "azurerm_key_vault_secret" "secret" {
  for_each     = var.secrets  # this is a map of "secret-name":"secret-value"
  key_vault_id = data.azurerm_key_vault.keyvault.id
  name         = each.key
  value        = each.value
}

Debug Output

This is the result of running terraform plan:

  # module.core_keyvault_secrets.azurerm_key_vault_secret.secret["postgres-password"] must be replaced
-/+ resource "azurerm_key_vault_secret" "secret" {
      ~ id           = "https://test-kv.vault.azure.net/secrets/postgres-password/b95ab91a7fb3597ff390ad8a9fe9859e" -> (known after apply)
      ~ key_vault_id = "/subscriptions/e43e6844-1632-41ab-99a4-bb747399dd22/resourceGroups/core-test/providers/Microsoft.KeyVault/vaults/test-kv" -> (known after apply) # forces replacement
        name         = "postgres-password"
      - tags         = {} -> null
        value        = (sensitive value)
      ~ version      = "b95ab91a7fb3597ff390ad8a9fe9859e" -> (known after apply)
    }

Plan: 1 to add, 0 to change, 1 to destroy.

And then this happens if I execute the plan

module.core_keyvault_secrets.azurerm_key_vault_secret.secret["postgres-password"]: Destroying... [id=https://test-kv.vault.azure.net/secrets/postgres-password/b95ab91a7fb3597ff390ad8a9fe9859e]
module.core_keyvault_secrets.data.azurerm_key_vault.keyvault: Refreshing state...
module.core_keyvault_secrets.azurerm_key_vault_secret.secret["postgres-password"]: Destruction complete after 2s
module.core_keyvault_secrets.azurerm_key_vault_secret.secret["postgres-password"]: Creating...

Error: keyvault.BaseClient#RecoverDeletedSecret: Failure responding to request: StatusCode=409 -- Original Error: autorest/azure: Service returned an error. Status=409 Code="Conflict" Message="Secret postgres-password is currently being deleted." InnerError={"code":"ObjectIsBeingDeleted"}

  on ../modules/az_keyvault_secrets/main.tf line 16, in resource "azurerm_key_vault_secret" "secret":
  16: resource "azurerm_key_vault_secret" "secret" {

And then if I plan again:

  # module.core_keyvault_secrets.azurerm_key_vault_secret.secret["postgres-password"] will be created
  + resource "azurerm_key_vault_secret" "secret" {
      + id           = (known after apply)
      + key_vault_id = (known after apply)
      + name         = "postgres-password"
      + value        = (sensitive value)
      + version      = (known after apply)
    }

Plan: 1 to add, 0 to change, 0 to destroy.

Expected Behavior

Terraform should not have detected any changes since no values or resources were changed.

Actual Behavior

Terraform detected a Key Vault ID change, thus planning to destroy my secret and recreate it. On top of that it also fails to do it with a 409 from Azure if I do run it.

Steps to Reproduce

1. terraform apply

Workarounds

• Output the ID of the key vault in my other module and use it directly in my azurerm_key_vault_secret, instead of the name and using the azurerm_key_vault data source (I already outputted the KV name for another thing, so I figured I would reuse it instead of adding a new output)
• Use lifecycle.ignore_change = [key_vault_id] on my secret resource

[tombuildsstuff]

hi @MarcDufresne

Thanks for opening this issue

Taking a look into this this appears to be an issue in Terraform Core where Data Sources aren't cached in the Terraform Statefile and as such Terraform needs to refresh them on every launch - ultimately meaning that Terraform believes there's changes here. This bug is being tracked in this issue in Terraform Core which I'm going to close this in favour of - would you mind subscribing to that issue for updates?

Thanks!

[magodo]

@MarcDufresne Can I ask a quick question, do you explicitly suppress refresh when doing plan/apply?

[MarcDufresne]

@magodo Do you mean explicitly passing -refresh=false? I don't use this option, I only run terraform apply

[magodo]

OK, I suppose you did not. I actually tried to reproduce your issue in my local setup but failed.
If you still interested in digging into this issue, can you try to reproduce it using the following configs? This is trying to see whether if that's a terraform core issue or this azure particular resource issue.

main.tf

module "a" {
  source = "./moduleA"
  name   = "foo"
}

module "b" {
  source = "./moduleB"
  a_id = module.a.id
}

output "b_id" {
  value = module.b.id
}

moduleA/main.tf

variable "name" {
  type = string
}

resource "null_resource" "resource_in_a" {
  triggers = {
    name = var.name
  }
}

output "id" {
  value = null_resource.resource_in_a.id
}

moduleB/main.tf

variable "a_id" {
  type = string
}

data "null_data_source" "data_in_b" {
  inputs = {
    id = var.a_id
  }
}

resource "null_resource" "resource_in_b" {
  triggers = data.null_data_source.data_in_b.outputs
}

output "id" {
  value = null_resource.resource_in_b.id
}

(moduleA represents your module which provisions key_vault, while moduleB represents your module in question).

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!