Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Post release v0.5.0] Update release doc #1028

Merged
merged 3 commits into from
Apr 17, 2023
Merged

[Post release v0.5.0] Update release doc #1028

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
0b0dd7e
update doc
kevin85421 Apr 14, 2023
48da23e
post release
kevin85421 Apr 14, 2023
9a0da8d
update
kevin85421 Apr 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
119 changes: 84 additions & 35 deletions docs/development/release.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,62 +16,111 @@ If further changes need to be made to the release, pick changes from the master
Make as many release candidates as necessary until a stable final release commit is reached.
Then build final release artifacts, publish release notes, and announce the release.

### KubeRay release schedule
KubeRay release plans to synchronize with every two Ray releases. KubeRay v0.5.0 synchronizes with Ray 2.4.0, so v0.6.0 should synchronize with Ray 2.6.0.

* **KubeRay feature freeze**: Two weeks before the official Ray release.
* **KubeRay release**: One week before the official Ray release.
* **Update KubeRay documentation in Ray repository**: Finish before the official Ray release.

### Steps

1. Ensure that the desired master commit is stable by verifying the following:
#### Step 0. KubeRay feature freeze

Ensure the last master commit you want to release passes the [Go-build-and-test](https://github.com/ray-project/kuberay/actions/workflows/test-job.yaml) workflow before feature freeze.

#### Step 1. Ensure that the desired master commit is stable

Ensure that the desired master commit is stable by verifying the following:

* The KubeRay documentation is up-to-date.
* All example configurations use the latest released version of Ray.
* The example configurations work.

- The last master commit you want to release passes the [Go-build-and-test](https://github.com/ray-project/kuberay/actions/workflows/test-job.yaml) workflow.
- The KubeRay documentation is up-to-date.
- All example configurations use the latest released version of Ray.
- The example configurations work.
During the KubeRay `0.5.0` release, we used spreadsheets to track [manual testing](https://docs.google.com/spreadsheets/d/13q059_lcaKb3BFmOlmZTtOqZPuGPYjRuKPqI1FSpCO8/edit?usp=sharing) and [documentation updates](https://docs.google.com/spreadsheets/d/13q059_lcaKb3BFmOlmZTtOqZPuGPYjRuKPqI1FSpCO8/edit?usp=sharing). Instead of using the latest stable release of KubeRay (i.e., v0.4.0 for the v0.5.0 release process), we should verify the master branch using the following:

* The nightly KubeRay operator Docker image: `kuberay/operator:nightly`.
* The local CRD / YAML / Helm charts.

During the KubeRay `0.4.0` release, we used spreadsheets to track [manual testing](https://anyscaleteam.slack.com/archives/D0412MV3X3J/p1670030376001989) and [documentation updates](https://docs.google.com/spreadsheets/d/1wlTXCWNtQxCUENa0fP2-dV6UYNUhUCix4exiGsep5GQ/edit#gid=0). Most of the manual release testing was done immediately before making the branch cut. No substantive code changes were made between the time of the branch cut and final release.
Open PRs to track the progress of manual testing for documentation, but **avoid merging these PRs** until the Docker images and Helm charts for v0.5.0 are available
(example PRs: [#997](https://github.com/ray-project/kuberay/pull/997), [#999](https://github.com/ray-project/kuberay/pull/999), [#1004](https://github.com/ray-project/kuberay/pull/1004), [#1012](https://github.com/ray-project/kuberay/pull/1012)).
Bug fix pull requests to fix bugs which found in the documentation testing process **can be merged** (example PR: [#1000](https://github.com/ray-project/kuberay/pull/1000)).

For future releases, we plan to automate more of the relese testing process. Use your best judgement to decide when to test manually during the release process.
To track the progress of KubeRay test development, see the [CI label on the KubeRay GitHub](https://github.com/ray-project/kuberay/labels/ci).
Manual testing can be time-consuming, and to relieve the workload, we plan to add more CI tests. The minimum requirements to move forward are:

* All example configurations can work with `kuberay/operator:nightly` and the latest release of Ray (i.e. 2.3.0 for KubeRay v0.5.0).
* Update all version strings in the documents.

2. Depending on whether the release is for a major, minor, or patch version, take the following steps.
- **Major or Minor version** (e.g. `0.4.0` or `1.0.0`). Create a release branch named `release-X.Y`:
#### Step 2. Create a new branch in ray-project/kuberay repository

* Depending on whether the release is for a major, minor, or patch version, take the following steps.
* **Major or Minor version** (e.g. `0.5.0` or `1.0.0`). Create a release branch named `release-X.Y`:
```
git checkout -b release-0.4
git push -u upstream release-0.4
git checkout -b release-0.5
git push -u upstream release-0.5
```
- **Patch version** (e.g. `0.4.1`). You don't need to cut a release branch for a patch version. Instead add commits to the release branch.
* **Patch version** (e.g. `0.5.1`). You don't need to cut a release branch for a patch version. Instead add commits to the release branch.

3. Create a first release candidate (`v0.4.0-rc.0`).
#### Step 3. Create a first release candidate (`v0.5.0-rc.0`).

a. Merge a PR into the release branch updating Helm chart versions, Helm chart image tags, and kustomize manifest image tags.
For `v0.4.0-rc0`, we did this in two PRs [1](https://github.com/ray-project/kuberay/pull/784/files) [2](https://github.com/ray-project/kuberay/pull/804/files), but it's fine to do it in a single PR. Note that [we no longer include appVersion in the Helm charts](https://github.com/ray-project/kuberay/pull/810).
* Merge a PR into the release branch updating Helm chart versions, Helm chart image tags, and kustomize manifest image tags. For `v0.5.0-rc0`, we did this in [PR #1001](https://github.com/ray-project/kuberay/pull/1001)

b. Release `rc0` images using the [release-image-build](https://github.com/ray-project/kuberay/actions/workflows/image-release.yaml) workflow on GitHub actions.
You will prompted for a commit reference and an image tag. The commit reference should be the SHA of the tip of the release branch. The image tag should be `vX.Y.Z-rc.0`.
* Release `rc0` images using the [release-image-build](https://github.com/ray-project/kuberay/actions/workflows/image-release.yaml) workflow on GitHub actions.
You will prompted for a commit reference and an image tag. The commit reference should be the SHA of the tip of the release branch. The image tag should be `vX.Y.Z-rc.0`.

c. Tag the tip of release branch with `vX.Y.Z-rc.0`.
* Tag the tip of release branch with `vX.Y.Z-rc.0`.
```
git tag v0.4.0-rc.0
git push upstream v0.4.0-rc.0
git tag v0.5.0-rc.0
git push upstream v0.5.0-rc.0
```

d. Release rc0 Helm charts following the [instructions](../release/helm-chart.md).
* Release rc0 Helm charts following the [instructions](../release/helm-chart.md).

* Open a PR into the Ray repo updating the operator version used in the autoscaler integration test. Make any adjustments necessary for the test to pass ([example](https://github.com/ray-project/ray/pull/33987)). Make sure the test labelled [kubernetes-operator](https://buildkite.com/ray-project/oss-ci-build-pr/builds/17146#01873a69-5ccf-4c71-b06c-ae3a4dd9aecb) passes before merging.

* Announce the `rc0` release on the KubeRay slack, with deployment instructions ([example](https://ray-distributed.slack.com/archives/C02GFQ82JPM/p1680555251566609)).

#### Step 4. Create more release candidates (`rc1`, `rc2`, ...) if necessary

* Resolve issues with the release branch by cherry picking master commits into the release branch.
* When cherry-picking changes, it is best to open a PR against the release branch -- don't push directly to the release branch.
* Follow step 3 to create new Docker images and Helm charts for the new release candidate.

#### Step 5. Create a final release

* Create a final release (i.e. v0.5.0) by repeating step 4 once more using the tag of the release (`vX.Y.Z`) with no `-rc` suffix.

#### Step 6. Merge open PRs in step 1 and post-release PRs

Now, we have the Docker images and Helm charts for v0.5.0.

* Merge the pull requests in Step 1 (i.e. [#997](https://github.com/ray-project/kuberay/pull/997), [#999](https://github.com/ray-project/kuberay/pull/999), [#1004](https://github.com/ray-project/kuberay/pull/1004), [#1012](https://github.com/ray-project/kuberay/pull/1012))

* Merge post-release pull requests (example: [#1010](https://github.com/ray-project/kuberay/pull/1010)). See [here](https://github.com/ray-project/kuberay/issues/940) to understand the definition of "post-release" and the compatibility philosophy for KubeRay.



#### Step 7. Update KubeRay documentation in Ray repository.

* Update KubeRay documentation in Ray repository with v0.5.0. Examples for v0.5.0:
* https://github.com/ray-project/ray/pull/33339
* https://github.com/ray-project/ray/pull/34178

e. Open a PR into the Ray repo updating the operator version used in the autoscaler integration test. Make any adjustments necessary for the test to pass. [Example](https://github.com/ray-project/ray/pull/30944/files). Make sure the test labelled [kubernetes-operator](https://buildkite.com/ray-project/oss-ci-build-pr/builds/7141#0184ef25-e62c-4dab-9c7e-ddfd583803cd) passes before merging.
#### Step 8. Generate release

f. Announce the `rc0` release on the KubeRay slack, with deployment instructions. [Example.](https://ray-distributed.slack.com/archives/C02GFQ82JPM/p1670375020308739).
* Click "Create release" to create release for the tag v0.5.0 ([link](https://github.com/ray-project/kuberay/tags)).
* Run `make release` in cli folder and generate `kuberay-$VERSION-darwin-amd64.zip` and `kuberay-$VERSION-linux-amd64.zip` files. Upload them to the GitHub release.
* Follow the [instructions](../release/changelog.md) to generate release notes and add notes in the GitHub release.

4. If necessary, create more release candidates (`rc1`, `rc2`, ...)
- Resolve issues with the release branch by cherry picking master commits
into the release branch.
- When cherry-picking changes, it is best to open a PR against the release branch -- don't push directly to the release branch.
- When the next release candidate is ready, repeat step 4 above.
#### Step 9. Announce the release on the KubeRay slack!

5. Create a final release by repeating Step 4 once more using the tag of the release (`vX.Y.Z`) with no `-rc` suffix.
* Announce the release on the KubeRay slack ([example](https://ray-distributed.slack.com/archives/C02GFQ82JPM/p1681244150758839))!

6. Run `make release` in cli folder and generate `kuberay-$VERSION-darwin-amd64.zip` and `kuberay-$VERSION-linux-amd64.zip` files. Upload them to the GitHub release.
#### Step 10. Update CHANGELOG

7. Follow the [instructions](../release/changelog.md) to generate release notes and add notes in the GitHub release.
* Send a PR to add the release notes to [CHANGELOG.md](../../CHANGELOG.md).

8. Send a PR to add the release notes to [CHANGELOG.md](../../CHANGELOG.md).
#### Step 11. Update and improve this release document!

9. Update KubeRay versions in Ray and KubeRay master. [Ray Example](https://github.com/ray-project/ray/pull/30981), [KubeRay Example](https://github.com/ray-project/kuberay/pull/821).
* Update this document and optimize the release process!

10. Announce the release on the KubeRay slack!