Improvement tasks for v1.6 release cycle

[furkatgofurov7]

NOTE: Issue is created based on #8418 to avoid ownership issues when there is a need for editing issue descriptions, assigning it to folks, etc. To check conversations in the previous cycle improvement tasks tracking issue, refer to #8418 (v1.5 release cycle) and #7524 (v1.4 release cycle).

NOTE2: The list of team-specific sub-tasks below are prioritized from highest priority sub-tasks to lowest from top to down accordingly, and priorities were agreed upon after triaging this issue collectively with the RT folks and repo maintainers. To see the notes from the triaging, please see: https://hackmd.io/@BcPM2Mr1SieWG3EZ0p5rGg/capi-improvement-tasks-v1-6

NOTE3: Each sub-task description has a GH handle next to it, meaning a sub-task is assigned to that person and he/she is actively working on it. If you would like to work on any unassigned sub-task feel free to comment on this umbrella issue 🙂

CI Signal/Bug Triage/Automation Manager Team sub-tasks

• Automate release branch and tag creation (@CecileRobertMichon)
  • Issue: Automate release branch and tag creation #7128
  • PR: ✨ Add automation to create release branch and tags #9111
• [❌ won't do/cancelled] Having a team alias in the OWNER/OWNER_ALIASES file so that it could be potentially used as a single source of truth where we need to know the list of release team members - could be used in some kind of automation. (@furkatgofurov7)
  • Example: To tag members of the release team on PR (may be auto generated) on repos outside of the kubernetes-sigs org, like kubernetes/test-infra repo on image promotion PRs
  • Context: 🌱 cc @cluster-api-release-team when auto-opening the PR to promote images from staging to prod #7487 (comment)
  • Based on the slack discussion, marking this as won't do/cancelled since anyways there is no point of cc ng RT members who themselves will be creating image promotion PR on the release cutting call.
• [❌ won't do/cancelled] cc release team on cherry-pick PRs (@cahillsf)
  • Best way to do it, is probably by extending the cherrypicker in test-infra
  • Alternative or temporary solution could be a GitHub action in the cluster-api repo
  • Issue: Provide a method to CC Release Team on cherry-pick PRs to Releases #9212
• [❌ won't do/cancelled] Improve the Homebrew publish process
  • Context: Homebrew publish on release does not work #6613
• Consider implementing a tool to generate our ProwJob YAMLs to make the jobs easier to manage (@killianmuldoon)
  • also check if there is already something like that
  • Issue: Generate ProwJob YAMLs and automate changes to test jobs #9257
• Resolve release series labels in e2e config (@adil)
  • Issue: Resolve release series labels in e2e config #5008
  • PR: ✨ Resolve release markers #9265
• Clearly define the CI team's scope (@nawazkh)
• Improve the Netlify access model. Right now only limit people have access to the Netlify and for doing any thing regarding Netlify the people with access have to do some manual work. There is scope to improve this, either through more automation or more shared access. (@nawazkh/@furkatgofurov7)
• Create SBoM of all the Cluster API components and verify checksum as a post build action (@kranurag7)
  • Issue: Security Self-Assessment: [STRIDE-TAMPER-1] Produce a SBoM #6153
• Theoretically we can use k8s-triage to generate bug reports for flaky tests via "file bug" (link). Unfortunately the issues are always filed against k/k so we have to manually copy&paste the issue text to a Cluster API repo. Let's see if we can improve k8s-triage to be able to create issues in the Cluster API repo. (@Sunnatillo)

Communications/Docs/Release Notes Manager Team sub-tasks

• Add a new "release trigger PRs" category for PRs (using 🚀 emoji) to be able to exclude these PRs from the release notes (more on slack thread) @g-gaston
  • 🌱 Exclude release trigger PRs from release notes #9444
  • ✨ Add 🚀 as a valid prefix for release PRs kubebuilder-release-tools#55
• Improve release notes formatting
  • PR: 🌱 Improve release notes formatting #9337
• Create a checklist that enumerates all the manual edits necessary in the release notes so it can followed on each release day. (@g-gaston)
  • PR: 🌱 Update instructions checklist to generate release notes #9443
• Add missing "info blocks" to the release notes tool output (supported k8s versions, etc.). Even if the content is not automated, output a "template" with placeholders. (@mjlshen)
  • Issue: Add additional headers to the release notes tool output #9248
  • PR: 🌱 Add additional blocks to release note generation #9247
• Deduplicate prefixes in release notes. Where the PR title already contains a prefix which is the same as its area, e.g. 🌱 test/e2e: improve logging for a detected rollout #8953. The prefix should be deduplicated. (@Dhairya-Arora01)
  • Tracking issue: Deduplicate prefixes in release notes. #9190
  • PR: 🐛 Deduplicating area in pr title in release notes #9186
• Create announcement templates to help specify all the comms needed and have a pre-established format to make it easier on the team. This includes slack messages and emails for releases, weekly updates, etc. (@willie-yao)
  • Related PR: 🌱 hack: add weekly update script for Slack #9343
  • PR to add announcement templates: 📖 Add release templates #9572
• Update documentation to build release notes tool from main before generating notes. We don't want to use go run anymore. Setup make target and ensuring the same binary is run on each release branch. @cahillsf
  • Issue: Improve and clarify release-notes tool usage #9556
  • PR: 🌱 update release-notes make target + corresponding doc #9573
• Ensure we have documentation on how to consume nightly / beta / RC Cluster API releases with clusterctl and the test framework (for users) @cahillsf
  • Issue: Ensure we have user documentation on how to consume nightly / beta / RC releases with clusterctl and test framework. #9604
• Consider adding expandable auto-generated release notes for beta and RC (non-GA) releases
  • Issue: Add expandable auto-generated release notes for beta and RC (non-GA) releases #9319
• Automate parts of the release notes that are "constant" through a minor release by reading from some file to populate the data. For example, the supported kubernetes versions can be maintained in a file on each release branch independently.
• Format dependencies to exclude multiple bumps in the same set of release notes. For Go dependencies there's a discussion in this thread: https://kubernetes.slack.com/archives/C8TSNPY4T/p1686045243091189. For Github actions and other dependencies an alternative solution will have to be found. @willie-yao
• Define details of "Communicate key dates to the community" @furkatgofurov7
  • PR: 📖 Define details of "Communicate key dates to the community"  #9590
• Potential improvements to generating release notes that automatically have the clusterctl: or E2E: or KCP: against the PR titles. Currently this process is done manually. May be we can improve this my looking at the area/xxx label of the PR or even consider some kind of PR title validation to make sure that users provide the prefix (kind of like how they currently provide ✨ , 🌱 , etc)
  • PRs:
    • 🌱 Add area label lookup for prefixes to release notes tool #8780
    • 🌱 Improve release notes output #8928
    • 🌱 Improve release notes formatting #8826
    • 🌱 Improve multiple areas PRs with user friendly subs #9071
• Doc improvement: communication guidelines (moved from Misc section) @furkatgofurov7
  • Public should be default - it gives visibility on the work being done, it is inclusive, it is a track of record -
  • Group DM as a safe space, but not for the actual work
  • PR: 📖 Remove maintainer role in release team and improve release team docs #9640
• Release note tool should not ignore squashed PRs (@mjlshen)
  • Issue: Release note tool should not ignore squashed PRs #9249
• add utility to create git issues on provider repos signalling a new CAPI minor release candidate is ready for testing. (@nawazkh)
  • PR: ✨ add utility to create git issues on provider repo #9110

Misc sub-tasks

• start using github project boards to track release improvements (@furkatgofurov7)
  • Board link: https://github.com/orgs/kubernetes-sigs/projects/55
• [❌ won't do/cancelled] Explore the idea of "an issue triage team" - a team that is responsible for triaging issues that come into the repo. (@furkatgofurov7)
  • Issue: Release Team: Explore the idea of "an issue triage team"  #9271
• automate updates to specific versions of https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api after release is cut (@cahillsf)
  • the link: https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api@latest needs to be called at least once before it's cached as the base page for the repo. The idea is to automate that update - we could initially document checking that page as a task during the release process manually. Automation can also be considered, i.e the same way how it is done in https://github.com/oracle/cluster-api-provider-oci/blob/main/.github/workflows/release.yaml#L28-L31
  • Issue: Ensure doc.crds.dev is updated on new release #9290
  • PR: 📖 add step for indexing the CRDs of the latest release #9291
• Doc improvement (add a note about checkpoint): Consider expanding the team even during the release cycle to handle no-shows. (@furkatgofurov7)
  • Setup guidelines for off-boarding.
  • Have a check-point to re-confirm with members of the release team about commitment for the rest of the release cycle.
  • Issue: Release Team: Consider expanding the team even during the release cycle to handle no-shows #9273
  • PR: 📖 Clarify adding/removing release team members #9550
• Doc improvement (task for onboarding/first weeks): The release team should review documentation at the beginning of the cycle to become aware of the existing process and tools. (@furkatgofurov7)
  • Issue: Release Team: Add Release Team members onboarding documentation #9274
  • PR: 📖 Add onboarding documentation for the Release Team #9565
• Doc improvement (add a recommendation to leads tasks):Communicate and set clear expectations between the leads and the shadows about timeline and rotation. (@Prajyot-Parab )
  • Clearly communicate that we will aim for rotation of ownership.
  • Issue: Documentation Improvements (v1.6 release cycle) #9216
  • 📖 Documentation improvements: suggestions for team leads #9270
• Doc improvement: consider "public release session" (@Prajyot-Parab )
  • Suggestion: Start a zoom meeting and make it public. Let's kick start the process in the zoom meeting. If there is time to just wait for some process to complete, use the time to discuss any issues/questions or just hop back on when ready to continue.
  • Issue: Documentation Improvements (v1.6 release cycle) #9216
  • PR: 📖 Documentation improvements: public release session #9232
• Document in the RL release tasks about removing unsupported release version of CAPI while working on preparing main branch PR (remove v1.3 while working on v1.6) (@chiukapoor)
  • Issue: Release Team: Add release tasks about removing unsupported release version of CAPI #9485
  • PR: 📖 Add release task about removing unsupported release version of CAPI #9487
• Document in the RL release tasks about making sure to update the GitHub actions to work with the new v1.6 release during week 15. Maybe make check lists tasks for future releases. (@chiukapoor)
  • Issue: Release Team: Add release tasks to update the GitHub actions to work with the new v1.6 release #9486
  • PR: 📖 Add release tasks to update the GitHub actions to work with the new release #9488

[furkatgofurov7]

/cc @kubernetes-sigs/cluster-api-release-team

[killianmuldoon]

/triage accepted

[furkatgofurov7]

@killianmuldoon @sbueringer @joekr while the all improvements tasks polished up now, i have only one sub task without a clear understanding what needs to be done:

automate updates to specific versions of https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api after release is cut

Do you have any idea what exactly needs to be done to accomplish ^ task?

[sbueringer]

I can only guess, but I think doc.crds.dev only adds a new version to the list if someone calls it once. Probably after that it's also used as "latest". I think the links in our docs/book point to "latest", e.g. https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api

[killianmuldoon]

Yeah - if I remember right this link: https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api@latest needs to be called at least once before it's cached as the base page for the repo.

The idea of that task was to automate that update - but it might be a good idea to initially add checking that page as a task during the release process. Automation can come after documented the manual process initially.

[sbueringer]

To be honest. I'm not sure if it's worth automating clicking a link if we do it once every 4 months. Maintaining automation also comes with a cost

[furkatgofurov7]

I can only guess, but I think doc.crds.dev only adds a new version to the list if someone calls it once. Probably after that it's also used as "latest". I think the links in our docs/book point to "latest", e.g. https://doc.crds.dev/github.com/kubernetes-sigs/cluster-api

I do not know why, but when I click the above link, I have list of CRDs from v1.4.5, should we change it to point to latest?

[furkatgofurov7]

The idea of that task was to automate that update - but it might be a good idea to initially add checking that page as a task during the release process. Automation can come after documented the manual process initially.

@killianmuldoon is it after the release is cut when we have to check it?

Also, talking to @joekr offline, got to know about this automation: https://github.com/oracle/cluster-api-provider-oci/blob/main/.github/workflows/release.yaml#L28-L31

Perhaps, we could also make use of it same way?

[sbueringer]

I do not why, but when I click the above link, I have list of CRDs from v1.4.5, should we change it to point to latest?

I get the same, I have no idea

[sbueringer]

Perhaps, we could also make use of it same way?

Sounds okay as a very last step. I don't want to manually re-run stuff if this curl somehow fails

[killianmuldoon]

@furkatgofurov7 can you assign me to Consider implementing a tool to generate our ProwJob YAMLs to make the jobs easier to manage

I'll open an issue to track it

[furkatgofurov7]

@furkatgofurov7 can you assign me to Consider implementing a tool to generate our ProwJob YAMLs to make the jobs easier to manage

I'll open an issue to track it

Sure, assigned to you @killianmuldoon

[furkatgofurov7]

@furkatgofurov7 can you assign me to Consider implementing a tool to generate our ProwJob YAMLs to make the jobs easier to manage

I'll open an issue to track it

@killianmuldoon did you have time to open a separate issue for this, if so, can you please point me to it?

[killianmuldoon]

@furkatgofurov7 I've created and issue here: #9257

[cahillsf]

@furkatgofurov7 can you add a task for ensuring the new 🚀 emoji passes the PR verify checks when you have a chance? example failed run

source code courtesy of Cecile (slack comment and src link)

related PR

[furkatgofurov7]

@furkatgofurov7 can you add a task for ensuring the new 🚀 emoji passes the PR verify checks when you have a chance? example failed run

source code courtesy of Cecile (slack comment and src link)

related PR

Related issue: kubernetes-sigs/kubebuilder-release-tools#54

[cahillsf]

👋 @furkatgofurov7 I can take this one:

Update documentation to build release notes tool from main before generating notes. We don't want to use go run anymore. Setup make target and ensuring the same binary is run on each release branch.

would be helpful if someone from comms team can provide a bit of context for why we are making this change as well

[furkatgofurov7]

👋 @furkatgofurov7 I can take this one:

Update documentation to build release notes tool from main before generating notes. We don't want to use go run anymore. Setup make target and ensuring the same binary is run on each release branch.

would be helpful if someone from comms team can provide a bit of context for why we are making this change as well

Thanks @cahillsf, updated the issue with your name. @g-gaston was the idea here to build the tool once from main branch and use that for release branches without needing to regenerate it separately?

@cahillsf in any case, let's open a separate issue to track it and we can discuss the details over there

[cahillsf]

👋 @furkatgofurov7 I can take this one too:

Ensure we have documentation on how to consume beta / RC releases (for users)

Although I need to understand this a bit better myself. I will miss the beta release cut tomorrow unfortunately but can watch the recording to catch up.

[furkatgofurov7]

@cahillsf in any case, let's open a separate issue to track it and we can discuss the details over there

@cahillsf ^^^ 😄

[cahillsf]

👋 hey @furkatgofurov7 all In Progress, To Do issues have been added to the v1.7 project board: https://github.com/orgs/kubernetes-sigs/projects/59

feel free to close out this issue

[furkatgofurov7]

@cahillsf how about:

• Improve the Netlify access model. Right now only limit people have access to the Netlify and for doing any thing regarding Netlify the people with access have to do some manual work. There is scope to improve this, either through more automation or more shared access
• Clearly define the CI team's scope
• Theoretically we can use k8s-triage to generate bug reports for flaky tests via "file bug" (link). Unfortunately the issues are always filed against k/k so we have to manually copy&paste the issue text to a Cluster API repo. Let's see if we can improve k8s-triage to be able to create issues in the Cluster API repo
• Automate parts of the release notes that are "constant" through a minor release by reading from some file to populate the data. For example, the supported kubernetes versions can be maintained in a file on each release branch independently.
• Format dependencies to exclude multiple bumps in the same set of release notes. For Go dependencies there's a discussion in this thread: https://kubernetes.slack.com/archives/C8TSNPY4T/p1686045243091189. For Github actions and other dependencies an alternative solution will have to be found.

they seem to be not in the GH board, which is obvious because you need either a separate issue or PR to add it to the board. Are you planning to create a separate issue in the repo and then add them to the board?
I am fine closing this issue out, just making sure we move over everything here to somewhere where we can track in the future.

[cahillsf]

Ah right good catch - I only moved over the tasks with existing issues, will cut issues for the remainder to add to the new project board

[cahillsf]

hey @furkatgofurov7 should be good here. the only one that i did not open as issue for is:

Consider adding expandable auto-generated release notes for beta and RC (non-GA) releases

will bring up for discussion in our kickoff RT meeting to see if we want to formalize this as an issue

[furkatgofurov7]

hey @furkatgofurov7 should be good here. the only one that i did not open as issue for is:

Consider adding expandable auto-generated release notes for beta and RC (non-GA) releases

will bring up for discussion in our kickoff RT meeting to see if we want to formalize this as an issue

There is an existing issue for that already, so you can just track it: #9319

[furkatgofurov7]

Looks like everything is covered now from this tracking issue, and all open/ongoing items have been carried over to GH board to be tracked, so closing this issue. Thanks @cahillsf!

/close

[k8s-ci-robot]

@furkatgofurov7: Closing this issue.

In response to this:

Looks like everything is covered now from this tracking issue, and all open/ongoing items have been carried over to GH board to be tracked, so closing this issue. Thanks @cahillsf!

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.