Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enforce .Elem.Fields (instead of .Fields) for object types #2309

Merged
merged 3 commits into from
Aug 15, 2024

Conversation

iwahbe
Copy link
Member

@iwahbe iwahbe commented Aug 13, 2024

This is a reversal of our previous stance that .Fields should be used. This aligns the
fields argument with schema generation.

We need to choose .Elem.Fields as our 1 representation (instead of .Fields) because
of an ambiguity in shim.Schema. From a comment in the PR:

To prevent confusion, users are barred from specifying
information on ps directly, they must set .Fields on ps.Elem:
ps.Elem.Fields.

We need to make this choice (instead of having users set
information on .Fields (and forbidding ps.Elem.Fields) because
the [shim.Schema] representation is not capable of
distinguishing between Map[Object] and Object. SDKv{1,2}
providers are not able to express Map[Object], but Plugin
Framework based providers are.

While this is technically a breaking change, I don't expect users to be broken. Any user
that would break on this change would have been unable to addapt the last several bridge
versions (for example: pulumiverse/pulumi-talos#99).

Fixes #2185

I have tested this change on talos, AWS, GCP and Azure and it doesn't trigger an error.

@iwahbe iwahbe self-assigned this Aug 13, 2024
@iwahbe iwahbe force-pushed the iwahbe/unify-elem-fields-warning branch from d5c0859 to 44e39ea Compare August 13, 2024 12:21
Copy link

codecov bot commented Aug 13, 2024

Codecov Report

Attention: Patch coverage is 83.78378% with 6 lines in your changes missing coverage. Please review.

Project coverage is 56.93%. Comparing base (992ffd4) to head (5c7dcd1).
Report is 3 commits behind head on master.

Files Patch % Lines
pkg/tfgen/generate.go 58.33% 4 Missing and 1 partial ⚠️
pkg/tfbridge/info/validate.go 96.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2309      +/-   ##
==========================================
+ Coverage   56.86%   56.93%   +0.06%     
==========================================
  Files         365      365              
  Lines       49962    49957       -5     
==========================================
+ Hits        28413    28444      +31     
+ Misses      20004    19972      -32     
+ Partials     1545     1541       -4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@iwahbe iwahbe requested a review from a team August 14, 2024 08:36
Copy link
Contributor

@VenelinMartinov VenelinMartinov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add some tfgen type override integration tests? We need one for 2185 as well as some tests around Set, List, Object, Map[Object], Map etc.

pkg/tfgen/generate.go Outdated Show resolved Hide resolved
@iwahbe iwahbe force-pushed the iwahbe/unify-elem-fields-warning branch 2 times, most recently from 32f875b to ca0ce3b Compare August 14, 2024 14:57
@iwahbe iwahbe force-pushed the iwahbe/unify-elem-fields-warning branch 2 times, most recently from 7d28415 to 0da65b6 Compare August 14, 2024 15:39
@iwahbe
Copy link
Member Author

iwahbe commented Aug 14, 2024

Can we add some tfgen type override integration tests? We need one for 2185 as well as some tests around Set, List, Object, Map[Object], Map etc.

I have added integration tests. See Add integration tests.

@iwahbe iwahbe force-pushed the iwahbe/unify-elem-fields-warning branch from 0da65b6 to 567b0ef Compare August 14, 2024 15:57
Copy link
Contributor

@VenelinMartinov VenelinMartinov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for adding the tests!

pf/tfgen/tfgen_test.go Outdated Show resolved Hide resolved
pkg/tfbridge/info/validate.go Outdated Show resolved Hide resolved
pkg/tfbridge/info/validate.go Outdated Show resolved Hide resolved
pf/tfgen/tfgen_test.go Outdated Show resolved Hide resolved
pf/tfgen/tfgen_test.go Show resolved Hide resolved
pkg/tfgen/generate_schema_test.go Show resolved Hide resolved
Copy link
Contributor

@VenelinMartinov VenelinMartinov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good - two last small comments

@iwahbe iwahbe force-pushed the iwahbe/unify-elem-fields-warning branch 2 times, most recently from 2b550fa to 96ac411 Compare August 15, 2024 09:51
@iwahbe iwahbe enabled auto-merge (rebase) August 15, 2024 09:51
@iwahbe iwahbe force-pushed the iwahbe/unify-elem-fields-warning branch from 96ac411 to 986aa53 Compare August 15, 2024 10:05
This is a reversal of our previous stance that .Fields should be used. This aligns the
fields argument with [schema
generation](https://github.com/pulumi/pulumi-terraform-bridge/blob/72b0b7d33b6d9f0dce7c09b31951ade3711a025d/pkg/tfgen/generate.go#L395-L398).

We *need* to choose `.Elem.Fields` as our 1 representation (instead of `.Fields`) because
of an ambiguity in `shim.Schema`. From a comment in the PR:

> To prevent confusion, users are barred from specifying
> information on ps directly, they must set .Fields on ps.Elem:
> ps.Elem.Fields.
>
> We need to make this choice (instead of having users set
> information on .Fields (and forbidding ps.Elem.Fields) because
> the [shim.Schema] representation is not capable of
> distinguishing between Map[Object] and Object. SDKv{1,2}
> providers are not able to express Map[Object], but Plugin
> Framework based providers are.

While this is technically a breaking change, I don't expect users to be broken. Any user
that would break on this change would have been unable to addapt the last several bridge
versions (for example: pulumiverse/pulumi-talos#99).

Fixes #2185
@iwahbe iwahbe force-pushed the iwahbe/unify-elem-fields-warning branch from 986aa53 to 47aa692 Compare August 15, 2024 10:07
This commit unifies the functionality of `Generator.Generate` and `GenerateWithOptions`,
so that they now both call `ProviderInfo.Validate` (previously, only `Generator.Generate`
called `ProviderInfo.Validate`).

This caused 2 unrelated tests to fail. Both can be fixed by removing the invalid mappings
they used to have.
This PR moves an invalid mapping in the
`pkg/tf2pulumi/convert/testdata/mappings/renames.json` to a valid mapping. You can see the
newly valid mapping take effect in the schema file:
`pkg/tf2pulumi/convert/testdata/schemas/renames.json`.

This PR exposes a bug where tf2pulumi doesn't properly propagate `MaxItems: 1`
information. The bug is shown in the changes in
`pkg/tf2pulumi/convert/testdata/renames/pcl/main.pp`.
@iwahbe iwahbe force-pushed the iwahbe/unify-elem-fields-warning branch from 47aa692 to 5c7dcd1 Compare August 15, 2024 10:07
@iwahbe iwahbe merged commit 0f1e694 into master Aug 15, 2024
11 checks passed
@iwahbe iwahbe deleted the iwahbe/unify-elem-fields-warning branch August 15, 2024 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Cannot set .Elem on a singly nested object block error when upgrading bridge
2 participants