Corso Release tooling

[vkamra]

Create a Corso release job that will publish the released images and documentation,

[vkamra]

The release job should have a dry-run or test mode which should run on some schedule that goes through all the mechanics but does not tag binaries/images with a release version and pushes docs to a test location.

[vkamra]

What is a release:

• Tag a version
• Build Corso binary for different architectures (amd64, arm64, windows-x64)
• Build and push corresponding Corso docker images
• Build and push Corso docs to production target (S3?)
• Post-validation?

How does it get triggered?

• Pre-req: Release testing is done
• We create a release github workflow to do above

Versioning/Dry-run:

• We should have the ability to trigger the job whenever we want. If not part of a release, should do a dry-run
• Dry-run should include all build steps and pushing artifacts (docker images, binaries, docs) to non-production targets

[ryanfkeepers]

Potential things to investigate:

• go-releaser: https://goreleaser.com/
• github repo version control: https://docs.github.com/en/repositories/releasing-projects-on-github/managing-releases-in-a-repository

[meain]

This is the basic structure that I have in mind for the release CI. Since the failures in separate jobs are reported as different checks in the GitHub UI, I was thinking of combining a few CI jobs to deduplicate the CI workflow files.

%%{init: {"theme": "dark"}}%%

flowchart
c(Checkout)
subgraph Tests["Tests"]
subgraph Code
l(Lint)
t(Test-Suite)
end
subgraph Docs
g(Generate-Markdown) --> dl(Lint Docs)
end
end
c --> Tests --> Publish
subgraph Publish["Publish"]
dp("Docs-Push [Release:prod-s3] [Commit:test-s3]")
bi("Build-Image [Release:tag=gh-tag] [Commit:tag=sha]")
gr("Release-Binary [Release:release-artifact] [Commit:run-artifact]")
end

Loading

[meain]

Had two more questions around the release pipeline:

• Is there plans to add a CHANGELOG.md? As of now, we can rely on goreleaser to add the commits as changelog to the release.
• Should the regex for release tag be v*.*.* or *.*.* (should we have a v prefix for the tag? We could also go with just *, but thought I would bring up this discussion anyways.

[meain]

I have updated the above diagram for the flow, the idea is that we can treat a commit on main to kick off a dry run and push the artifacts to non release/test destinations. This has two advantages:

• Make sure every commit to main is ready for a release
• Folks will have access to the latest docs, binaries and docker images

[vkamra]

• Is there plans to add a CHANGELOG.md? As of now, we can rely on goreleaser to add the commits as changelog to the release.

It'll be good to have this be generated automagically. Using goreleaser will be ideal but it does look like we'll need to start adding more info in commit messages. That should work. When you get a chance to do a test run - can you share an example of what the Changelog will look like?

• Should the regex for release tag be v*.*.* or *.*.* (should we have a v prefix for the tag? We could also go with just *, but thought I would bring up this discussion anyways.

Let's add the v prefix. I've seen both but I believe v is more common in Go projects.

[meain]

Just to add some context, the main problem with using commit messages as changelog is that we might end up with not so useful entries in the changelog. We could probably use a prefix in the commit message which we can filter out but that will mean we will end up having to enforce the commit message format with everyone who will be contributing to the repo once we open source it.

PS: If we are keeping a changelog file, we can follow something like https://keepachangelog.com/

[vkamra]

Just to add some context, the main problem with using commit messages as changelog is that we might end up with not so useful entries in the changelog.

You're right - using commits will make this not so useful and it's good to maintain something separate. Let's start with CHANGELOG.md using the guidance from https://keepachangelog.com/ i.e. can you commit one with an UNRELEASED section that we as a team can update prior to the release.