Add update support for google_data_catalog_tag_template.fields

[danawillow]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

Follow-up from #5892. This was broken out because it requires extra work in MM to be able to update a field on a resource by calling add/remove/patch on a URL specific to that field. Might be able to work around by just using custom code.

New or Affected Resource(s)

• google_data_catalog_tag_template

[tmshn]

I've just started to see the API reference, and learned that Data Catalog supports renaming of fields (not only the display name, but also id).
https://cloud.google.com/data-catalog/docs/reference/rest/v1/projects.locations.tagTemplates.fields/rename

Then, how should these changes treated?

• Delete foo and create bar
• Rename foo into bar

 resource "google_data_catalog_tag_template" "table_business_basic" {
   tag_template_id = "..."

   fields {
-    field_id = "foo"
+    field_id = "bar"
     type {
       primitive_type = "STRING"
     }
   }
 }

(I think it's okay not to support renaming at the next step, but anyway field schema seems need to be changed to support this someday)

[tmshn]

One possible solution for this is to introduce deleted field to each tag.

With that field, the following example denotes rename operation

   fields {
-    field_id = "foo"
+    field_id = "bar"
     type {
       primitive_type = "STRING"
     }
   }

And the following example denotes delete+create operation:

   fields {
-    field_id = "foo"
+    deleted = true
+  }
+  fields {
+    field_id = "bar"
     type {
       primitive_type = "STRING"
     }
   }

Having almost empty {deleted = true} field looks a little bit tricky, but it's explicit and the behavior is predictable. Also, this doesn't require any breaking change to the schema.

[tmshn]

I've spared a little bit time on tackling on this, and found some limitations. Let me share.

Strategy 1. Use deleted=true with ordered list

This first strategy is the one I describe above. It requires fields ordering, but

• Current tfstate does not preserve order because type of fields is now TypeSet
• API response does not preserve order (even more, it seems to be indeterministic), so it's hard to calculate accurate diff even using CustomizeDiff

Strategy 2. Split fields into different resource like tag_template_field

I feel this is quite natural, because it has own separate API endpoints for Create / Update(patch) / Delete and rename (but not for Read).

Missing of read API won't be a problem; we can customize Read method to call the API of their parent template.

However, this strategy also has a problem:

• TagTemplate does not allow fields to be empty, which might happen if we split tf resources into template and its fields

---

So due to nature/limitations both of Terraform and GCP API, it's impossible to support all capability Data Catalog offers.

As renaming operation is most tricky part, that's what we should give up. Without it, things must go easier.

[danieldeleo]

How is the work progressing on supporting field addition/modifications? (excluding rename functionality)

[roaks3]

b/251966466

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.