[Bug]: Provider does not handle C2S region failures for multiple resource types

[thefirstofthe300]

Terraform Core Version

1.5.7

AWS Provider Version

5.63.1

Affected Resource(s)

aws_cloudwatch_log_group
aws_cloudwatch_event_rule

Expected Behavior

Terraform should have handled a failure with the API call due to the region gracefully

Actual Behavior

Provider fails due to error returned stating the operation does not exist in the region.

Relevant Error/Panic Output Snippet

Error: listing tags for EventBridge Rule (arn:aws-iso:events:us-iso-east-1:<SNIP>:rule/<SNIP>): operation error EventBridge: ListTagsForResource, https response error StatusCode: 400, RequestID: , deserialization failed, failed to decode response body, invalid character '<' looking for beginning of value

Terraform Configuration Files

resource "aws_cloudwatch_event_rule" "main" {
  name                = "repro"
  schedule_expression = "rate(1 day)"
}

provider "aws" {
  region = "us-iso-east-1"
}

Steps to Reproduce

terraform apply

Debug Output

HTTP status code of error is 400
Returned payload is

<?xml version="1.0" encoding="UTF-8" standalone="no"?><Response><Errors><Error><Code>UnsupportedOperation</Code><Message>The functionality you requested is not available in this region.</Message></Error><RequestID>7b389a8cd1914c259f9937f2609d4eeb</RequestID></Errors></Response>

Panic Output

No response

Important Factoids

We are currently using an emulation service to emulate the C2S regions in a public AWS cloud region to validate our service works: https://www.applied-insight.com/

References

No response

Would you like to implement a fix?

No

[github-actions]

Community Note

Voting for Prioritization

• Please vote on this issue by adding a 👍 reaction to the original post to help the community and maintainers prioritize this request.
• Please see our prioritization guide for information on how we prioritize.
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

Volunteering to Work on This Issue

• If you are interested in working on this issue, please leave a comment.
• If this would be your first contribution, please review the contribution guide.

[thefirstofthe300]

Would be part of the effort here: #18593

[justinretzolk]

Hey @thefirstofthe300 👋 Thank you for taking the time to raise this!

We are currently using an emulation service to emulate the C2S regions in a public AWS cloud region to validate our service works

I appreciate you mentioning this; that's really helpful information to have. While I'm not familiar with the offering, I believe I'm understanding correctly that the service you're using is essentially an API-compatible service for mocking deployments to AWS itself, is that correct? And if so, are you able to confirm whether the same behavior occurs when interacting with AWS itself?

While we try not to break compatibility with services like that (you'll find there's a few open issues related to LocalStack, for instance), it's not something that we actively test against. For the sake of transparency, those sorts of issues also generally take more time to be prioritized, with compatibility with the upstream API being more critical to the provider experience.

[thefirstofthe300]

I don't have access to a C2S environment and our partners with access to the environment have given us very little feedback on whether these errors occur so I can't say with any certainty whether the issue occurs in an actual C2S region. That being said, the documentation on the region states in the "How CloudWatch Logs Differs for AWS Top Secret Regions" section that

• Tagging CloudWatch Logs Groups is unsupported.

as well as

The following actions are not supported:
• list-tags-for-resource
• tag-resource
• untag-resource

in the "How CloudWatch Events Differs for AWS Top Secret Regions" leading me to believe that these are still the case and this particular bug is valid.

[thefirstofthe300]

To explain what happened here, our emulation provider was returning invalid JSON which caused TF to fail instead of use the fallback logic which doesn't set tags. They fixed that bug and everything appears to be hunky-dory now.

[github-actions]

Warning

This issue has been closed, meaning that any additional comments are hard for our team to see. Please assume that the maintainers will not see them.

Ongoing conversations amongst community members are welcome, however, the issue will be locked after 30 days. Moving conversations to another venue, such as the AWS Provider forum, is recommended. If you have additional concerns, please open a new issue, referencing this one where needed.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.