UX for Beanstalk changes is abysmal

[kevinburkenotion]

A single change to a typical Elastic Beanstalk configuration will generate 100-500 lines of Terraform output depending on how many settings you have configured. A lot of these have low information value, for example, here the configuration is changing from "false" to "false". There are tens to hundreds of lines printed for any change that look exactly like this.

      setting.2322805565.namespace: "aws:elbv2:listener:default" => "aws:elbv2:listener:default"
      setting.2322805565.resource:  "" => ""
      setting.2322805565.value:     "false" => "false"

I just deployed an error to production that broke software at the company I'm working with because I missed one line of changed diff out of 250 that remained the same and I would be interested in solutions that would reduce the risk of making a similar mistake.

[apparentlymart]

Hi @kevinburkenotion! Sorry for the low-quality of the diff output in this case.

The verbosity of this output results from the fact that so much of the schema for this resource type is backed by sets of objects, and thus any change to any part of a set can only be understood as removing the entire element and replacing it with another one -- set elements have no identity separate from their content.

The forthcoming v0.12.0 release of Terraform has an entirely new diff renderer that uses a nested, structural diff approach where nested blocks are described via indentation rather than by repeated prefixes like the setting.2322805565. here. I expect this will go some way to improving the readability of the result since all of the attributes of a common parent block will be rendered visually separated from those of others, similar to the layout of the configuration itself.

That will not, however, address the fact that sets of complex objects do not diff well: Terraform will still show an entire block as being removed and another created whenever something changes inside one of the nested blocks backed by sets, because there's no correlation key to recognize when an existing object is being edited.

In the provider protocol changes for Terraform v0.12 we've made some room for further improvement here by supporting complex nested types involving maps, where the old protocol could only support maps of primitive types. The SDK does not yet support this since it is currently required to support both the Terraform v0.11 and v0.12 protocols, but once the SDK gains support for arbitrary map element types this setting block could be recast as a map of maps of strings, which would then diff in a more intuitive way:

settings = {
  "aws:ec2:vpc" = {
    VPCId   = aws_vpc.example.id
    Subnets = join(",", aws_subnet.example.*.id)
  }
  "aws:elb:listener" = {
    InstancePort     = 443
    InstanceProtocol = "HTTPS"
    # ...
  }
}

The exact details of how that is modeled will ultimately be up to the AWS provider team to decide, but the next step will be to update the provider SDK so that it can make use of this new Terraform Core feature (which the diff renderer for v0.12 already knows how to display) and then the AWS provider team can decide how best to make use of that new capability, and (importantly) how to transition gracefully away from the old setting block without breaking everyone.

Once v0.12 is released and stable, the SDK improvements project will get underway. I'm going to label this so that it'll be visible to those planning that project so that this use-case can be used as design input.

[kevinburkenotion]

Thanks and glad to hear you are thinking about this.

[molleafauss]

Thanks for sharing the updates.
I also want to raise the fact that in 0.11 this way of calculating differences seems actually causing beanstalk to trigger a reload of the environment like when a configuration changes is been applied through the console, while instead nothing has effectively changed.
We created a few beanstalks together by creating few sets of variables and using a common module for beanstalks, as we basically want similarly configured environment, and when we apply a change to one, also all the other applications seems to trigger a reload. As you can understand this isn't ideal.

[GRoguelon]

Hi @apparentlymart, any update? Thank you in advance.

[paultyng]

As Martin states above, rich map support would solve this, going to merge this in to #62

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.