Outputs are evaluated before resource updates which can cause unnecessary errors

[antonosmond]

Terraform Version

Terraform v0.12.14

Terraform Configuration Files

variable "my_list" {
  default = [
    "foo",
    "bar",
  ]
}

resource "random_uuid" "example" {
  count = length(var.my_list)
}

output "example" {
  value = zipmap(var.my_list, random_uuid.example.*.id)
}

Expected Behavior

Given the above config, I run terraform apply.
Everything should run successfully and my outputs should be as expected.
Then I decide I no longer need bar so I remove it from my_list and run terraform apply.
I'd expect this to work.

Actual Behavior

I get the following error:

Error: Error in function call

  on example.tf line 12, in output "example":
  12:   value = zipmap(var.my_list, random_uuid.example.*.id)
    |----------------
    | random_uuid.example is tuple with 2 elements
    | var.my_list is tuple with 1 element

Call to function "zipmap" failed: number of keys (1) does not match number of
values (2).

Steps to Reproduce

1. create config as above
2. terraform init
3. terraform apply
4. remove bar from my_list
5. terraform apply

Additional Context

I can understand why it's failing - zipmap requires that the 2 lists are of equal length and the removal of bar from the list means this is no longer true on the second apply however I'd expect the output to be evaluated AFTER the resource updates.

I'd expect the following order of events:

1. evaluate var.my_list (length has changed from 2 to 1)
2. evaluate random_uuid.example and count = length(var.my_list) (count has changed from 2 to 1)
3. remove resource random_uuid.example[1] due to the changed count
4. evaluate output.example where var.my_list now has a length of 1 and random_uuid.example.*.id also now has a length of 1.

I assume there's some validation happening before the resource is updated and the new var.my_list.length is being compared against the old random_uuid.example.length

[teamterraform]

Hi @antonosmond! Sorry for this weird behavior, and thanks for reporting it.

Although the details are a little different, this problem is in the same family of problems as discussed in #17034 for data resources: Terraform currently has a separate "refresh" step that runs before planning, and that step can get itself into a strange state where values in existing objects have changed but we've not yet considered the possibility of new objects being created or existing objects being destroyed.

The fix for this one is likely to be in the same overall area as for #17034: we need to merge the refresh step into the plan step so that refreshing and planning happen at the same time, and thus Terraform will be able to react to changes it detects during planning when dealing with objects such as data resources and outputs that can often be evaluated immediately without waiting for the apply step.

We're going to leave this issue open because it's describing a different problem, but we wanted to leave a connection between the two here as a reminder that the solutions for them both are probably overlapping quite a bit.

[antonosmond]

@teamterraform Thanks for the reply. These kinds of issues have caused me a lot of frustration over the last few years. My experience has been that terraform is great at apply => destroy cycles, however given an apply => modify => apply cycle, it really struggles.
As terraform has been improved over the years e.g. with the addition of modules, data sources and more complex functions and expressions, the reliability of apply => modify => apply cycles continues to decrease and I've found myself actively trying to avoid using what should be helpful functionality.
Data sources are a great example. Given module A creates resources which are utilised by module B, a data source is a great way to limit the data that one module needs to pass to the other. Why pass a whole bunch of outputs from module A to module B if module B can simply "lookup" what it needs via a data source? This should also make each module much more independent. It turns out this is a really bad use of a data source! Any changes that cause module A to update resources which module B depends on will likely cause a subsequent apply to fail i.e. apply => modify => apply becomes unreliable because of the use of data sources. The better option is to avoid using data sources and just pass the data directly between module A and module B using outputs and inputs.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.