Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Epic] Switch all tfgen uses to the new converter #1280

Closed
10 of 15 tasks
Tracked by #3780 ...
Frassle opened this issue Jul 7, 2023 · 15 comments
Closed
10 of 15 tasks
Tracked by #3780 ...

[Epic] Switch all tfgen uses to the new converter #1280

Frassle opened this issue Jul 7, 2023 · 15 comments
Assignees
Labels
kind/engineering Work that is not visible to an external user resolution/fixed This issue was fixed
Milestone

Comments

@Frassle
Copy link
Member

Frassle commented Jul 7, 2023

This epic tracks rollout of Terraform to Pulumi example converter to improve coverage and quality of examples in Registry.

https://docs.google.com/document/d/1FRQAJd17sTjfKYVhfuN9N9AtZNfRLlQv37-sIwEnDyA/edit?usp=sharing

Work Items ⚙️

Tier 2 Provider Upgrade Procedure

Since there are many of these we will use a simplified procedure. ci-mgmt will be updated to set PULUMI_CONVERT=true and ensure Pulumi CLI is pinned; then these providers will receive the improved converter during routine updates as they pull in new tfbrdige versions. Apparently there is a system in place that tracks example cov metrics on S3, so we can monitor the process using this system.

Associated Docs Bugs

Fixes #1399

@Frassle Frassle added the kind/engineering Work that is not visible to an external user label Jul 7, 2023
@t0yv0

This comment was marked as off-topic.

@t0yv0 t0yv0 self-assigned this Jul 31, 2023
@mikhailshilkov mikhailshilkov added this to the 0.93 milestone Aug 11, 2023
@t0yv0

This comment was marked as outdated.

@t0yv0

This comment was marked as outdated.

@t0yv0

This comment was marked as outdated.

@mikhailshilkov mikhailshilkov modified the milestones: 0.93, 0.94 Sep 5, 2023
@mikhailshilkov mikhailshilkov modified the milestones: 0.94, 0.95 Oct 2, 2023
t0yv0 added a commit to pulumi/ci-mgmt that referenced this issue Oct 6, 2023
To be reviewed with pulumi/pulumi-azuread#484

Part of pulumi/pulumi-terraform-bridge#1280

As a pre-requisite to PULUMI_CONVERT being enabled, it's necessary to
pin Pulumi CLI so that examples continue to be deterministic. This PR
borrows targets from pulumi-azure-native that perform CLI pinning.

## Updates

### CLI 

Unlike azure-native, it assumes that the CLI version we want to use to
convert examples is the same as pulumi/pkg version we want to use to
build this provider. This removes a degree of freedom but buys automatic
upgrades - the existing mechanisms that update pulumi/pkg (in
upgrade-provider) should also update this Pulumi CLI version.

### Converter

This is not enforced here but
pulumi/pulumi-azuread#484 adds TF converter
pinning to the CI-mgmt plugins section. This PR makes changes to allow
opting into converter and not just resource plugins. There is currently
no mechanism to upgrade this, but presumably upgrade-provider can build
mechanisms to upgrade all plugins referenced in this config file. Any
such upgrade is likely to trigger the need to regenerate SDKs.
@mikhailshilkov mikhailshilkov modified the milestones: 0.95, 0.97 Oct 26, 2023
@mnlumi mnlumi changed the title Switch all tfgen uses to the new converter [Epic] Switch all tfgen uses to the new converter Oct 30, 2023
@t0yv0
Copy link
Member

t0yv0 commented Dec 4, 2023

@cnunciato kindly agreed to help out making provider registry builds to help validate the Tier 1 PRs. But unfortunately since that conversation I started rebasing the PRs and hit a new blocker. pulumi/pulumi#14718 I believe Fraser is looking at this presently.

@lukehoban lukehoban modified the milestones: 0.97, 0.98 Dec 20, 2023
@t0yv0
Copy link
Member

t0yv0 commented Jan 4, 2024

Blocked on pulumi/pulumi-yaml#539

@t0yv0
Copy link
Member

t0yv0 commented Jan 19, 2024

This just shipped so we should be unblocked! Wonderful news.

@Frassle
Copy link
Member Author

Frassle commented Jan 19, 2024

This just shipped so we should be unblocked!

It merged, we need to do a yaml and pulumi release still.

@t0yv0
Copy link
Member

t0yv0 commented Jan 31, 2024

Almost there, next pulumi release I can finish this off for the Tier1 providers at least I believe.

@t0yv0
Copy link
Member

t0yv0 commented Feb 13, 2024

Another round of iteration, Fraser unblocked another problem (🙇 ), the latest version of the converter gives non-deterministic build results again, digging into that tomorrow. That seems like the last known blocker-level problem for rolling out to tier1.

@t0yv0
Copy link
Member

t0yv0 commented Feb 13, 2024

pulumi/pulumi-azure#1377 PR is in the best shape of how this looks like (latest version).

@t0yv0
Copy link
Member

t0yv0 commented May 22, 2024

AFAIK all our providers have migrated (perhaps need to check two newly added ones like meraki CC @mjeffryes). Exploring cleanup in #2004 that would propagate the change out to bridged providers outside of Pulumi org.

@t0yv0
Copy link
Member

t0yv0 commented May 28, 2024

Wrote comms #2037 we should circulate and get feedback on before pulling the trigger.

@t0yv0
Copy link
Member

t0yv0 commented May 29, 2024

Per conversation with @mjeffryes , closing as completed and tracking comms+cleanup in a new epic https://github.com/pulumi/home/issues/3452

@t0yv0 t0yv0 closed this as completed May 29, 2024
@pulumi-bot pulumi-bot reopened this May 29, 2024
@pulumi-bot
Copy link
Contributor

Cannot close issue:

  • does not have required labels: resolution/

Please fix these problems and try again.

@t0yv0 t0yv0 added the resolution/fixed This issue was fixed label May 29, 2024
@t0yv0 t0yv0 closed this as completed May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/engineering Work that is not visible to an external user resolution/fixed This issue was fixed
Projects
None yet
Development

No branches or pull requests

6 participants