Datasources silently return success on 404s when they should throw errors

[rileykarson]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

Not specific to a given core or provider version. I reproduced with:

$ terraform -v
Terraform v1.3.2
on darwin_arm64
+ provider registry.terraform.io/hashicorp/google v4.41.0

Affected Resource(s)

• Most data.google_* resources, verified with data.google_container_cluster

Terraform Configuration Files

data "google_container_cluster" "my_cluster" {
  location = "us-central1"
  name = "does-not-exist"
}

Debug Output

https://gist.github.com/rileykarson/29dc7cde1d6f6eed3953432a95f6cdb8

Panic Output

N/A

Expected Behavior

An error should be returned to the user

Actual Behavior

$ terraform apply
data.google_container_cluster.my_cluster: Reading...
data.google_container_cluster.my_cluster: Read complete after 0s

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Steps to Reproduce

1. terraform apply

Important Factoids

We reuse Read functions between resources and their corresponding datasources. That means on a 404 error we set the resource id to "". In a resource, that means that the resource doesn't exist. However in a datasource, it's meaningless. This manifests as Terraform believing it was successful in retrieving results for a datasource even though it's entirely null, as the underlying GET call failed. These differences are explained in the Read function documentation:

// Managed resources can signal to Terraform that the managed resource
// instance no longer exists and potentially should be recreated by calling
// the SetId method with an empty string ("") parameter and without
// returning an error.

// Data resources that are designed to return state for a singular
// infrastructure component should conventionally return an error if that
// infrastructure does not exist and omit any calls to the
// SetId method.

References

• google_gke_hub_membership does not respect the depends_on argument #12850

[lemoo5sg]

@rileykarson
We have a use case where we need to use a data source within a module or tf code before the actual dependant resource is created. If the data is not found we will use counts or for_each to prevent the creation of the other resources.

The request #14701 has been closed because it is against the solution strategy.

However, what alternative do we have in order not to block the process for this normal use case ?

[rileykarson]

I'm not sure, unfortunately! HashiCorp may have guidance on how to accomplish this. Returning an error when not found is- to our knowledge- the correct contract for datasources, and one we don't currently follow consistently.

[c2thorn]

closed with GoogleCloudPlatform/magic-modules#8858

This will release with v5.0.0. Check #15582 for more details on the major release.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.