Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cli: store upgrade files in versioned folders #1929

Merged
merged 6 commits into from
Jun 21, 2023
Merged

Conversation

msanft
Copy link
Contributor

@msanft msanft commented Jun 14, 2023

Proposed change(s)

  • Implement versioning of Constellation upgrades, so that a user can perform multiple upgrades without cleaning the constellation-upgrade directory. Each upgrade is marked by a timestamp string and a unique identifier and the corresponding files get placed into a unique subdirectory for their upgrade ID. (e.g. constellation-upgrade/upgrade-20230614105831-976b92d0)

Additional info

Checklist

  • Add labels (e.g., for changelog category)
  • Link to Milestone

@msanft msanft added the feature This introduces new functionality label Jun 14, 2023
@msanft msanft added this to the v2.9.0 milestone Jun 14, 2023
@msanft msanft requested a review from derpsteb June 14, 2023 09:04
@netlify
Copy link

netlify bot commented Jun 14, 2023

Deploy Preview for constellation-docs canceled.

Name Link
🔨 Latest commit 2f10c3f
🔍 Latest deploy log https://app.netlify.com/sites/constellation-docs/deploys/649170e95692f200080ce532

// If checkOnly is true, the upgrader will only create the resources necessary for upgrade checking. (i.e. no Terraform client)
// Not creating the Terraform client in the check-case makes sense as the initialization of the Terraform client copies the
// executable to the upgrade directory, which is not necessary for the check-only case.
func NewUpgrader(ctx context.Context, outWriter io.Writer, log debugLog, checkOnly bool) (*Upgrader, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we might not always want to initialize the terraform client here. But if we don't, we should guard all access to tfUpgrade with a null-pointer check. if u.tfUpgrade == nil {...}. Alternatively you could omit the flag, put the initialization into a function and call that init function in case u.tfUpgrade is nil.

One could also argue that our upgrade check is incomplete, as it does not check if there are migrations that would need to be applied. If I am not mistaken we are applying migrations during upgrade apply unconditionally. Meaning that it makes no difference if the user is only upgrading services, image or k8s - migrations are applied in any case. Because of that, upgrade check could do a dry-run terraform apply to inform the user about the upcoming migration. I don't expect that you do this right now. I just had this thought and would like to hear what others think.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re. upgrade check:
We could certainly do this! However I think we should not add the "cloud resources" as a singular upgradable component such as microservices, k8s, etc. - Since the CLI gets upgraded either way, it might expect the cloud resources to exist no matter which component got upgraded. I think both upgrade check and upgrade apply should always consider the Terraform migrations.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

re. nil check:
The main reason for not initializing the Terraform client here was that in the case of an upgrade check, the Terraform upgrade directories were created as well, as they get initialized with the Terraform client. If we dry-run the migrations in upgrade check, this would be obsolete.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As it probably needs some more code changes which are out of scope for this PR, I track this in AB#3226.

@derpsteb
Copy link
Member

derpsteb commented Jun 15, 2023

Looks great. Just that one comment.

Started another e2e testrun as I think the one you posted didn't upgrade the services and thus didn't execute the changed code.

I am removing the feature label to place the change in the changelog's "general" section instead of "new features". I agree that it's a new functionality. But this feels more like an improvement of an existing feature.

@derpsteb derpsteb removed the feature This introduces new functionality label Jun 15, 2023
@msanft
Copy link
Contributor Author

msanft commented Jun 19, 2023

See #1942

@msanft msanft requested a review from derpsteb June 19, 2023 12:49
Copy link
Member

@derpsteb derpsteb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

* dry-run Terraform migrations on upgrade check

* clean whole upgrade dir

* clean up check workspace after planning

* fix parsing

* extend upgrade check test

* rename unused parameters

* exclude false positives in test
@derpsteb
Copy link
Member

If you could update the PR title to something slightly more descriptive. That would be great. E.g. "cli: store backups during upgrade in timestamped folders". Thanks!

@msanft msanft changed the title cli: upgrade versioning cli: store upgrade files in versioned folders Jun 20, 2023
@msanft msanft merged commit b25228d into main Jun 21, 2023
4 of 5 checks passed
@msanft msanft deleted the feat/cli/upgrade-versioning branch June 21, 2023 07:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants