Removing OCI security list and updating associated subnet fails

[martijndwars]

Terraform Version

Terraform v1.1.9
on darwin_amd64
+ provider registry.terraform.io/hashicorp/oci v4.75.0

Terraform Configuration Files

locals {
  tenancy_id = "ocid1.tenancy.oc1..xyz"
}

resource "oci_core_vcn" "main_vcn" {
  compartment_id = local.tenancy_id
  cidr_blocks    = ["10.0.0.0/16"]
  display_name   = "main-vcn"
}

/*resource "oci_core_security_list" "main_security_list" {
  compartment_id = local.tenancy_id
  vcn_id         = oci_core_vcn.main_vcn.id
  display_name   = "main_security_list"
}*/

resource "oci_core_subnet" "main_subnet" {
  cidr_block        = "10.0.0.0/24"
  compartment_id    = local.tenancy_id
  vcn_id            = oci_core_vcn.main_vcn.id
  display_name      = "main-subnet"
  security_list_ids = [/*oci_core_security_list.main_security_list.id*/]
}

Debug Output

See https://gist.github.com/martijndwars/4d9ef71f59fbc9c273dcb60dcc17cae2 for the debug output of the planning phase. I'm not sure if this is enough, because the bug manifests during the applying phase.

Expected Behavior

Terraform should 1. update the subnet (to no longer use the security list) and 2. remove the security list (note the order!).

Actual Behavior

Terraform tries to 1. remove the security list before 2. updating the subnet. But because the subnet uses the security list, it is not possible to remove the security list, so Terraform gets stuck at step 1 until it times out:

Terraform will perform the following actions:

  # oci_core_security_list.main_security_list will be destroyed
  # (because oci_core_security_list.main_security_list is not in configuration)
  - resource "oci_core_security_list" "main_security_list" {
      - compartment_id = "ocid1.tenancy.oc1..aaaaaaaai2u6rtfadkxu6hcejqeopdreavdrtapp6lixzfzfqohiiz3v3mga" -> null
      - defined_tags   = {} -> null
      - display_name   = "main_security_list" -> null
      - freeform_tags  = {} -> null
      - id             = "ocid1.securitylist.oc1.eu-zurich-1.aaaaaaaanhjyvuu6uw45jag7vtn6ehcurs4pdpb3x7kljog5hrkg7vl6hjha" -> null
      - state          = "AVAILABLE" -> null
      - time_created   = "2022-05-16 07:47:22.409 +0000 UTC" -> null
      - vcn_id         = "ocid1.vcn.oc1.eu-zurich-1.amaaaaaaoma2o6iayouiaizwudf47mp25sh7ayoj73s6nbohp7jd432rkyfa" -> null
    }

  # oci_core_subnet.main_subnet will be updated in-place
  ~ resource "oci_core_subnet" "main_subnet" {
        id                         = "ocid1.subnet.oc1.eu-zurich-1.aaaaaaaaprazpyewkbunmi6c6hx5rsnyrwkzmdb5gezbdoq5p26ctgy7hofa"
      ~ security_list_ids          = [
          - "ocid1.securitylist.oc1.eu-zurich-1.aaaaaaaanhjyvuu6uw45jag7vtn6ehcurs4pdpb3x7kljog5hrkg7vl6hjha",
        ]
        # (15 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 1 to destroy.

Steps to Reproduce

Given the configuration from the Terraform Configuration Files section,

0. Enter a valid tenancy ID (and setup your OCI configuration in ~/.oci/config)
1. Uncomment the commented blocks
2. Run a plan & apply
3. Add back the comments (to simulate the removal of the security list)
4. Run a plan & apply

[martijndwars]

I asked this on https://discuss.hashicorp.com/t/removing-oci-security-list-and-updating-associated-subnet/39539 as well, but since it looks like a bug, I figure I'd report it here as well. As a start, I would like to learn if this is a Terraform bug or a bug in the OCI provider. Perhaps someone with more experience can shed some light on this?

[martijndwars]

I did some digging and found basically the same bug report in the OCI provider repository: oracle/terraform-provider-oci#909. That was back in 2019 and closed with a comment that it's a bug in Terraform. Now that we are 2,5 years further, has the situation improved?

[jbardin]

Hi @martijndwars,

Sorry you are having difficulty with this. The replacement order for resource is well-defined, with the default always being to destroy before creating or updating. If you want a dependent resource to update before the removal of it's dependency, you need to use the create_before_destroy option in the lifecycle meta block.

Thanks!

[martijndwars]

@jbardin thanks for your reply. The create_before_destroy option solves the case where a resource needs to be deleted and recreated, but the creation needs to happen before the deletion. In my case I want to remove a security list (and by extension remove it from the subnet). Terraform deletes the security list before unlinking it from the subnet, so this fails. The correct order is to first unlink the security list from the subnet and then delete the security list. Note that no resource is created in this scenario, so I don't see how create_before_destroy solves this.

Fwiw, it's also not clear which resource would get the create_before_destroy option. In this scenario I'm removing the security list resource, so that's not an option. I can add the create_before_destroy option to the subnet, but the subnet is neither created nor destroyed (it is updated to unlink the security list).

[jbardin]

Yes, technically create_before_destroy defines an order of operations for resource replacement, but addition, update, and removal of resources are simply subsets of that replacement order. The option could very well have been called create_or_update_before_destroy, but even that doesn't completely capture the nuance.

I made some notes about the operational orderings in a document here: https://github.com/hashicorp/terraform/blob/main/docs/destroying.md

[martijndwars]

Thanks again for all your help @jbardin. I noticed that if I add create_before_destroy to the subnet and remove the security list from the subnet in the same terraform plan/apply, then the problem still exists. However, if I first add create_before_destroy, then refresh the state (either by running terraform refresh or terraform apply), and finally remove the security list from the subnet, then create_before_destroy has the desired effect.

That's with version 1.1.9. With version 0.13.5, which is admittedly a bit old, this does not work as intended (Terraform still tries to delete the security list before the subnet).

In the future, would it be possible for Terraform providers to declare these ordering constraints, so that end-users are not required to sprinkle their code with create_before_destroy? Is this technically possible/something that's on the Terraform roadmap?

[jbardin]

It's been discussed several times about having providers include some feedback about dependencies and ordering. The problem is that if a resource requires a different order of operations, that ordering can affect other resources, which may be from other providers that are not able to handle create_before_destroy. The ordering may also be dependent on how the configuration is structured, some resources may not want it universally and only need to use create_before_destroy in some user-defined situations.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.