Error: Internal error encountered while processing change (1005)

[istvanfedak]

Confirmation

• My issue isn't already found on the issue tracker.
• I have replicated my issue using the latest version of the provider and it is still present.

Terraform and Cloudflare provider version

Terraform v1.2.3
on darwin_amd64

Affected resource(s)

• cloudflare_zone_settings_override

Terraform configuration files

- resource "cloudflare_zone_settings_override" "enforced_settings" {
      - id                       = "<id-value>" -> null
      - initial_settings         = [
          - {
              - always_online               = "off"
              - always_use_https            = "off"
              - automatic_https_rewrites    = "off"
              - binary_ast                  = "off"
              - brotli                      = "off"
              - browser_cache_ttl           = 14400
              - browser_check               = "on"
              - cache_level                 = "aggressive"
              - challenge_ttl               = 1800
              - ciphers                     = []
              - cname_flattening            = "flatten_at_root"
              - development_mode            = "off"
              - early_hints                 = "off"
              - email_obfuscation           = "on"
              - filter_logs_to_cloudflare   = "off"
              - h2_prioritization           = "off"
              - hotlink_protection          = "off"
              - http2                       = "on"
              - http3                       = "off"
              - image_resizing              = "off"
              - ip_geolocation              = "on"
              - ipv6                        = "on"
              - log_to_cloudflare           = "on"
              - max_upload                  = 100
              - min_tls_version             = "1.0"
              - minify                      = [
                  - {
                      - css  = "off"
                      - html = "off"
                      - js   = "off"
                    },
                ]
              - mirage                      = "off"
              - mobile_redirect             = [
                  - {
                      - mobile_subdomain = ""
                      - status           = "off"
                      - strip_uri        = false
                    },
                ]
              - opportunistic_encryption    = "off"
              - opportunistic_onion         = "on"
              - orange_to_orange            = "off"
              - origin_error_page_pass_thru = "off"
              - polish                      = "off"
              - prefetch_preload            = "off"
              - privacy_pass                = "on"
              - proxy_read_timeout          = "100"
              - pseudo_ipv4                 = "off"
              - response_buffering          = "off"
              - rocket_loader               = "off"
              - security_header             = [
                  - {
                      - enabled            = false
                      - include_subdomains = false
                      - max_age            = 0
                      - nosniff            = false
                      - preload            = false
                    },
                ]
              - security_level              = "medium"
              - server_side_exclude         = "on"
              - sort_query_string_for_cache = "off"
              - ssl                         = "full"
              - tls_1_2_only                = "off"
              - tls_1_3                     = "on"
              - tls_client_auth             = "off"
              - true_client_ip_header       = "off"
              - universal_ssl               = "on"
              - visitor_ip                  = "on"
              - waf                         = "off"
              - webp                        = "off"
              - websockets                  = "on"
              - zero_rtt                    = "off"
            },
        ] -> null
      - initial_settings_read_at = "2022-07-18T18:29:49.909005246Z" -> null
      - readonly_settings        = [
          - "advanced_ddos",
        ] -> null
      - zone_id                  = "<id>" -> null
      - zone_status              = "pending" -> null
      - zone_type                = "partial" -> null
      - settings {
          - always_online               = "off" -> null
          - always_use_https            = "on" -> null
          - automatic_https_rewrites    = "off" -> null
          - binary_ast                  = "off" -> null
          - brotli                      = "off" -> null
          - browser_cache_ttl           = 14400 -> null
          - browser_check               = "on" -> null
          - cache_level                 = "aggressive" -> null
          - challenge_ttl               = 1800 -> null
          - ciphers                     = [] -> null
          - cname_flattening            = "flatten_at_root" -> null
          - development_mode            = "off" -> null
          - early_hints                 = "off" -> null
          - email_obfuscation           = "on" -> null
          - filter_logs_to_cloudflare   = "off" -> null
          - h2_prioritization           = "off" -> null
          - hotlink_protection          = "off" -> null
          - http2                       = "on" -> null
          - http3                       = "off" -> null
          - image_resizing              = "off" -> null
          - ip_geolocation              = "on" -> null
          - ipv6                        = "on" -> null
          - log_to_cloudflare           = "on" -> null
          - max_upload                  = 100 -> null
          - min_tls_version             = "1.2" -> null
          - mirage                      = "off" -> null
          - opportunistic_encryption    = "off" -> null
          - opportunistic_onion         = "on" -> null
          - orange_to_orange            = "off" -> null
          - origin_error_page_pass_thru = "off" -> null
          - polish                      = "off" -> null
          - prefetch_preload            = "off" -> null
          - privacy_pass                = "on" -> null
          - proxy_read_timeout          = "100" -> null
          - pseudo_ipv4                 = "off" -> null
          - response_buffering          = "off" -> null
          - rocket_loader               = "off" -> null
          - security_level              = "medium" -> null
          - server_side_exclude         = "on" -> null
          - sort_query_string_for_cache = "off" -> null
          - ssl                         = "full" -> null
          - tls_1_2_only                = "off" -> null
          - tls_1_3                     = "on" -> null
          - tls_client_auth             = "off" -> null
          - true_client_ip_header       = "off" -> null
          - universal_ssl               = "off" -> null
          - visitor_ip                  = "on" -> null
          - waf                         = "off" -> null
          - webp                        = "off" -> null
          - websockets                  = "on" -> null
          - zero_rtt                    = "off" -> null
          - minify {
              - css  = "off" -> null
              - html = "off" -> null
              - js   = "off" -> null
            }
          - mobile_redirect {
              - status    = "off" -> null
              - strip_uri = false -> null
            }
          - security_header {
              - enabled            = false -> null
              - include_subdomains = false -> null
              - max_age            = 0 -> null
              - nosniff            = false -> null
              - preload            = false -> null
            }
        }
    }

Debug output

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  - destroy

Terraform will perform the following actions:

  # module.waf-terraform-core.module.cloudflare.module.zone["test.com"].module.zone_settings[0].cloudflare_zone_settings_override.enforced_settings[0] will be destroyed
  # (because module.waf-terraform-core.module.cloudflare.module.zone["test.com"].module.zone_settings[0] is not in configuration)
  - resource "cloudflare_zone_settings_override" "enforced_settings" {
      - id                       = "<id>" -> null
      - initial_settings         = [
          - {
              - always_online               = "off"
              - always_use_https            = "off"
              - automatic_https_rewrites    = "off"
              - binary_ast                  = "off"
              - brotli                      = "off"
              - browser_cache_ttl           = 14400
              - browser_check               = "on"
              - cache_level                 = "aggressive"
              - challenge_ttl               = 1800
              - ciphers                     = []
              - cname_flattening            = "flatten_at_root"
              - development_mode            = "off"
              - early_hints                 = "off"
              - email_obfuscation           = "on"
              - filter_logs_to_cloudflare   = "off"
              - h2_prioritization           = "off"
              - hotlink_protection          = "off"
              - http2                       = "on"
              - http3                       = "off"
              - image_resizing              = "off"
              - ip_geolocation              = "on"
              - ipv6                        = "on"
              - log_to_cloudflare           = "on"
              - max_upload                  = 100
              - min_tls_version             = "1.0"
              - minify                      = [
                  - {
                      - css  = "off"
                      - html = "off"
                      - js   = "off"
                    },
                ]
              - mirage                      = "off"
              - mobile_redirect             = [
                  - {
                      - mobile_subdomain = ""
                      - status           = "off"
                      - strip_uri        = false
                    },
                ]
              - opportunistic_encryption    = "off"
              - opportunistic_onion         = "on"
              - orange_to_orange            = "off"
              - origin_error_page_pass_thru = "off"
              - polish                      = "off"
              - prefetch_preload            = "off"
              - privacy_pass                = "on"
              - proxy_read_timeout          = "100"
              - pseudo_ipv4                 = "off"
              - response_buffering          = "off"
              - rocket_loader               = "off"
              - security_header             = [
                  - {
                      - enabled            = false
                      - include_subdomains = false
                      - max_age            = 0
                      - nosniff            = false
                      - preload            = false
                    },
                ]
              - security_level              = "medium"
              - server_side_exclude         = "on"
              - sort_query_string_for_cache = "off"
              - ssl                         = "full"
              - tls_1_2_only                = "off"
              - tls_1_3                     = "on"
              - tls_client_auth             = "off"
              - true_client_ip_header       = "off"
              - universal_ssl               = "on"
              - visitor_ip                  = "on"
              - waf                         = "off"
              - webp                        = "off"
              - websockets                  = "on"
              - zero_rtt                    = "off"
            },
        ] -> null
      - initial_settings_read_at = "2022-07-19T17:50:44.587194Z" -> null
      - readonly_settings        = [
          - "advanced_ddos",
        ] -> null
      - zone_id                  = "<id>" -> null
      - zone_status              = "pending" -> null
      - zone_type                = "partial" -> null

      - settings {
          - always_online               = "off" -> null
          - always_use_https            = "on" -> null
          - automatic_https_rewrites    = "off" -> null
          - binary_ast                  = "off" -> null
          - brotli                      = "off" -> null
          - browser_cache_ttl           = 14400 -> null
          - browser_check               = "on" -> null
          - cache_level                 = "aggressive" -> null
          - challenge_ttl               = 1800 -> null
          - ciphers                     = [] -> null
          - cname_flattening            = "flatten_at_root" -> null
          - development_mode            = "off" -> null
          - early_hints                 = "off" -> null
          - email_obfuscation           = "on" -> null
          - filter_logs_to_cloudflare   = "off" -> null
          - h2_prioritization           = "off" -> null
          - hotlink_protection          = "off" -> null
          - http2                       = "on" -> null
          - http3                       = "off" -> null
          - image_resizing              = "off" -> null
          - ip_geolocation              = "on" -> null
          - ipv6                        = "on" -> null
          - log_to_cloudflare           = "on" -> null
          - max_upload                  = 100 -> null
          - min_tls_version             = "1.2" -> null
          - mirage                      = "off" -> null
          - opportunistic_encryption    = "off" -> null
          - opportunistic_onion         = "on" -> null
          - orange_to_orange            = "off" -> null
          - origin_error_page_pass_thru = "off" -> null
          - polish                      = "off" -> null
          - prefetch_preload            = "off" -> null
          - privacy_pass                = "on" -> null
          - proxy_read_timeout          = "100" -> null
          - pseudo_ipv4                 = "off" -> null
          - response_buffering          = "off" -> null
          - rocket_loader               = "off" -> null
          - security_level              = "medium" -> null
          - server_side_exclude         = "on" -> null
          - sort_query_string_for_cache = "off" -> null
          - ssl                         = "full" -> null
          - tls_1_2_only                = "off" -> null
          - tls_1_3                     = "on" -> null
          - tls_client_auth             = "off" -> null
          - true_client_ip_header       = "off" -> null
          - universal_ssl               = "off" -> null
          - visitor_ip                  = "on" -> null
          - waf                         = "off" -> null
          - webp                        = "off" -> null
          - websockets                  = "on" -> null
          - zero_rtt                    = "off" -> null

          - minify {
              - css  = "off" -> null
              - html = "off" -> null
              - js   = "off" -> null
            }

          - mobile_redirect {
              - status    = "off" -> null
              - strip_uri = false -> null
            }

          - security_header {
              - enabled            = false -> null
              - include_subdomains = false -> null
              - max_age            = 0 -> null
              - nosniff            = false -> null
              - preload            = false -> null
            }
        }
    }

Plan: 0 to add, 0 to change, 1 to destroy.

Do you want to perform these actions in workspace "master"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.waf-terraform-core.module.cloudflare.module.zone["test.com"].module.zone_settings[0].cloudflare_zone_settings_override.enforced_settings[0]: Destroying... [id=<id>]
╷
│ Error: Internal error encountered while processing change (1005)
│ 
│ 
╵
Releasing state lock. This may take a few moments...

Panic output

No response

Expected output

Apply complete! Resources: 0 added, 0 changed, 1 destroyed.

Actual output

Error: Internal error encountered while processing change (1005)

Steps to reproduce

1. Create a zone and add the zone_settings_override resource to it.

resource "cloudflare_zone_settings_override" "enforced_settings" {
      zone_id = "<id>"
      settings {
              always_online               = "off"
              always_use_https            = "off"
              automatic_https_rewrites    = "off"
              binary_ast                  = "off"
              brotli                      = "off"
              browser_cache_ttl           = 14400
              browser_check               = "on"
              cache_level                 = "aggressive"
              challenge_ttl               = 1800
              ciphers                     = []
              cname_flattening            = "flatten_at_root"
              development_mode            = "off"
              early_hints                 = "off"
              email_obfuscation           = "on"
              filter_logs_to_cloudflare   = "off"
              h2_prioritization           = "off"
              hotlink_protection          = "off"
              http2                       = "on"
              http3                       = "off"
              image_resizing              = "off"
              ip_geolocation              = "on"
              ipv6                        = "on"
              log_to_cloudflare           = "on"
              max_upload                  = 100
              min_tls_version             = "1.0"
              minify  {
                      css  = "off"
                      html = "off"
                      js   = "off"
              }
              mirage                      = "off"
              mobile_redirect  {
                      mobile_subdomain = ""
                      status           = "off"
                      strip_uri        = false
              }
              opportunistic_encryption    = "off"
              opportunistic_onion         = "on"
              orange_to_orange            = "off"
              origin_error_page_pass_thru = "off"
              polish                      = "off"
              prefetch_preload            = "off"
              privacy_pass                = "on"
              proxy_read_timeout          = "100"
              pseudo_ipv4                 = "off"
              response_buffering          = "off"
              rocket_loader               = "off"
              security_header {
                      enabled            = false
                      include_subdomains = false
                      max_age            = 0
                      nosniff            = false
                      preload            = false
              }
              security_level              = "medium"
              server_side_exclude         = "on"
              sort_query_string_for_cache = "off"
              ssl                         = "full"
              tls_1_2_only                = "off"
              tls_1_3                     = "on"
              tls_client_auth             = "off"
              true_client_ip_header       = "off"
              universal_ssl               = "on"
              visitor_ip                  = "on"
              waf                         = "on"
              webp                        = "off"
              websockets                  = "on"
              zero_rtt                    = "off"
}

2. Delete the zone_settings_override resource.

Additional factoids

No response

References

No response

[github-actions]

Thank you for reporting this issue! For maintainers to dig into issues it is required that all issues include the entirety of TF_LOG=DEBUG output to be provided. The only parts that should be redacted are your user credentials in the X-Auth-Key, X-Auth-Email and Authorization HTTP headers. Details such as zone or account identifiers are not considered sensitive but can be redacted if you are very cautious. This log file provides additional context from Terraform, the provider and the Cloudflare API that helps in debugging issues. Without it, maintainers are very limited in what they can do and may hamper diagnosis efforts.

This issue has been marked with triage/needs-information and is unlikely to receive maintainer attention until the log file is provided making this a complete bug report.

[steve-mt]

I'm not sure if this is related but we've hit a similar 1005 error for cloudflare_page_rule where always_online seem to have been removed from the actions?

Cloudflare Page Rule: An invalid target or action was specified: Missing or unconfigured setting: 'always_online' (1005)`

I don't see it in the UI either

We also have Always Online disabled on the caching page, I wonder if that is related

---

Looking at the original issue description always_online seems to be present as well.

[jacobbednarz]

always_online v1 has been deprecated and everyone should have been migrated automatically to v2. can you try removing that from your resource?

[rquadling]

With regard the the always_online issue mentioned (seen today for the first time in our pipelines), what is the corresponding change we need to make to our resources other than removing the defunct action from the list of actions?

Having said that, the entries we have were all "off", so, in theory, there's no additional change/migration to be made.

[rquadling]

OOI, how much effort is it to mark an action as deprecated in the provider? That would have helped a lot in this situation.

[steve-mt]

always_online v1 has been deprecated and everyone should have been migrated automatically to v2. can you try removing that from your resource?

@jacobbednarz thank you for that information! I was not aware! Was there any official announcement? I can't seem to find it and looking at Cloudflare's official documentation below it seems like Always Online for Page Rules is still valid?

• https://developers.cloudflare.com/cache/how-to/enable-always-online/
• https://support.cloudflare.com/hc/en-us/articles/218411427-Understanding-and-configuring-Cloudflare-Page-Rules-Page-Rules-Tutorial-/#h_18YTlvNlZET4Poljeih3TJ

Also, what is the difference between v1 and v2? Is v2 zone-wide or is page rule specific?

[jacobbednarz]

a deprecation in the provider wouldn't have helped here and manual intervention was always going to be required due to the way the resource tracks initial settings and attempts to reset them (unless it had identical payloads and lived under the same endpoint). in the future, this resource is planned to be replaced to combat similar problems; see #1646.

re: API, I'm unsure if this has a publicly documented endpoint so I'd need to confirm with the service team. I'm also not certain if those docs on it being available in page rules are still accurate.

[jacobbednarz]

cc @zaidoon1

[steve-mt]

Thanks for the explanation @jacobbednarz 🙇

Removing the always_online from the resource helped in my case and the terraform plan showed that it was a no-op so this is resolved from my end, thanks again!

[istvanfedak]

Also @jacobbednarz when I run the code below I get this error message Error: filter_logs_to_cloudflare cannot be modified for this zone (1057), log_to_cloudflare cannot be modified for this zone (1057), Unrecognized zone setting name: binary_ast (1006), It is not possible to change Orange To Orange settings for this zone. (1024), It is not possible to change Visitor IP settings for this zone. (1025), Unrecognized zone setting name: h2_prioritization (1006).

I know that that error is mentioned in the Plan-Dependent Settings that those settings should be set to null but its counter intuitive to set it as the default value and still get that error. I'm assuming in the back end if its null it doesn't attempt to change the setting and if its not null then it attempts to set it to the value and then it gets an error from Cloudflare's API. It would be nice if cloudflare_zone_settings_override would check if the value you're setting it to is the same as the default and then not attempt the API call in the backend.

When I look at the terraform state file I see the default values being set for the resource when I set them to null.

provider "cloudflare" {
    # email pulled from $CLOUDFLARE_EMAIL environment variable
    # token pulled from $CLOUDFLARE_API_KEY environment variable
}

terraform {
  required_providers {
    cloudflare = {
      source  = "cloudflare/cloudflare"
      version = "= 3.16.0"
    }
  }
}

resource "cloudflare_zone" "this" {
  zone = "test.com"
  plan = "enterprise"
  type = "partial"
}

resource "cloudflare_zone_settings_override" "enforced_settings" {
  zone_id = cloudflare_zone.this.id
  settings {
    always_online               = "off"
    always_use_https            = "on"
    automatic_https_rewrites    = "off"
    binary_ast                  = "off"
    brotli                      = "off"
    browser_cache_ttl           = 14400
    browser_check               = "on"
    cache_level                 = "aggressive"
    challenge_ttl               = 1800
    ciphers                     = []
    cname_flattening            = "flatten_at_root"
    development_mode            = "off"
    early_hints                 = "off"
    email_obfuscation           = "on"
    filter_logs_to_cloudflare   = "off"
    h2_prioritization           = "off"
    hotlink_protection          = "off"
    http2                       = "on"
    http3                       = "off"
    image_resizing              = "off"
    ip_geolocation              = "on"
    ipv6                        = "on"
    log_to_cloudflare           = "on"
    max_upload                  = 100
    min_tls_version             = "1.2"
    minify  {
            css  = "off"
            html = "off"
            js   = "off"
    }
    mirage                      = "off"
    mobile_redirect  {
            mobile_subdomain = ""
            status           = "off"
            strip_uri        = false
    }
    opportunistic_encryption    = "off"
    opportunistic_onion         = "on"
    orange_to_orange            = "off"
    origin_error_page_pass_thru = "off"
    polish                      = "off"
    prefetch_preload            = "off"
    privacy_pass                = "on"
    proxy_read_timeout          = "100"
    pseudo_ipv4                 = "off"
    response_buffering          = "off"
    rocket_loader               = "off"
    security_header {
            enabled            = false
            include_subdomains = false
            max_age            = 0
            nosniff            = false
            preload            = false
    }
    security_level              = "medium"
    server_side_exclude         = "on"
    sort_query_string_for_cache = "off"
    ssl                         = "full"
    tls_1_2_only                = "off"
    tls_1_3                     = "on"
    tls_client_auth             = "off"
    true_client_ip_header       = "off"
    universal_ssl               = "on"
    visitor_ip                  = "on"
    waf                         = "off"
    webp                        = "off"
    websockets                  = "on"
    zero_rtt                    = "off"
  }
}

[jacobbednarz]

cloudflare_zone_settings_override is quite problematic here (related to why i'm deprecating it) and it really shouldn't be used to manage all settings due to issues like this. the resource itself attempts to maintain the defaults which commonly fails and breaks alot of Terraform's assumptions.

i'd advise only managing the settings in code that you've actually changed until this resource is addressed.

if you're having issues with a specific field that should be editable but isn't, are you able to open a specific issue or customer support ticket so it can be routed to the correct service team?

[istvanfedak]

Thank you thats great news, the resource has been giving us some trouble.

Yes i'm able to create customer support tickets.

[jacobbednarz]

feel free to ping me on the numbers once you have them and i can poke some folks internally with this thread when it comes up.

[jacobbednarz]

regarding comms on the change, i've confirmed with the service team.

• there was enterprise out reach emails for this deprecation
• there was a banner in the UI related to the swap (not great for API only users, i've provided some feedback on this)
• the API docs were updated for the zone (newer version) of always online however the deprecated version (page rule) doesn't have the update but we can get this addressed.
• the developer docs that mention page rules being able to control this are incorrect. the newer version only has zone level support. tickets are being raised to clean this up.

thanks for the feedback!

[dynek]

always_online v1 has been deprecated and everyone should have been migrated automatically to v2. can you try removing that from your resource?

From my understanding, we can't enable Always Online for a given URL anymore, only for a whole domain/zone. Is that correct ?

Thank you

[jacobbednarz]

only for a whole domain/zone. Is that correct ?

correct

[alex-feel]

always_online v1 has been deprecated and everyone should have been migrated automatically to v2. can you try removing that from your resource?

@jacobbednarz could you tell which provider version it is deprecated from? Is it v3.19.0?