Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[RFC] release: use GitHub's gh to create GitHub release #18649

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 3 additions & 7 deletions Documentation/contributor-guide/release.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ which don't need to be executed before releasing each version.
4. Authenticate the image registry, refer to [Authentication methods](https://cloud.google.com/container-registry/docs/advanced-authentication).
- `gcloud auth login`
- `gcloud auth configure-docker`
5. Install gh, refer to [GitHub's documentation](https://github.com/cli/cli#installation). Ensure that running `gh auth login` succeeds for the GitHub account you use to contribute to etcd.

### Release steps

Expand All @@ -81,13 +82,8 @@ which don't need to be executed before releasing each version.

**Remove the `quay.io` login entry from `~/.docker/config.json` after pushing Docker images.**
7. Publish the release page on GitHub
- Set the release title as the version name
- Choose the correct release tag (generated from step #4)
- Follow the format of previous release pages
- Attach the generated binaries and signature file
- Verify the historical binary size for each architecture. If there's a big difference, verify that it works for that architecture
- Select whether it's a pre-release
- Publish the release
- Open the **draft** release URL shown by the release script
- Review that it looks correct, then publish the release
8. Announce to the etcd-dev googlegroup

Follow the format of previous release emails sent to [email protected], see an example below. After sending out the email, ask one of the mailing list maintainers to approve the email from the pending list. Additionally, label the release email as `Release`.
Expand Down
56 changes: 52 additions & 4 deletions scripts/release.sh
Original file line number Diff line number Diff line change
Expand Up @@ -33,8 +33,7 @@ help() {
echo "WARNING: This does not perform the 'Add API capabilities', 'Performance testing' "
echo " or 'Documentation' steps. These steps must be performed manually BEFORE running this tool."
echo ""
echo "WARNING: This script does not sign releases, publish releases to github or sent announcement"
echo " emails. These steps must be performed manually AFTER running this tool."
echo "WARNING: This script does not send announcement emails. This step must be performed manually AFTER running this tool."
echo ""
echo " args:"
echo " version: version of etcd to release, e.g. 'v3.2.18'"
Expand Down Expand Up @@ -119,6 +118,23 @@ main() {
exit 1
fi

# Check that gh is installed and logged in.
log_callout "Check gh installation"
if ! command -v gh >/dev/null; then
log_error "Cannot find gh. Please follow the installation instructions at https://github.com/cli/cli#installation"
exit 1
fi
local gh_version
gh_version=$(gh --version | head -n1 | awk 'match($0,/([0-9]+\.){2}[0-9]+/) {print substr($0,RSTART,RLENGTH)}')
if [ "$(cut -d. -f1 <(echo "${gh_version}"))" -lt "2" ] || [ "$(cut -d. -f2 <(echo "${gh_version}"))" -lt "57" ]; then
log_error "The minimum version required for gh is 2.57.0, got ${gh_version}"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious - Why do we have a dependency on this version? Is it purely from a standpoint of what we have tested?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was having issues with gh auth status's exit code (the exit code was 0 even when it failed). I swear I saw a pull request/a release note for this behavior in 2.57.0 (which is also the latest version). But I just tested again with multiple versions and went back all the way to 2.37.0, and it works as expected.

So, I'm fine removing this check.

exit 1
fi
if [ "${DRY_RUN}" != "true" ] && ! gh auth status &>/dev/null; then
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we validate gh auth status irrespective of DRY_RUN? If I was using DRY_RUN=true I would still want to know that I had a problem with gh.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I avoided the check because it would fail when running on the CI. One option would be to add a new execution flag (i.e., --no-github-release) so we can set that in the release test and still keep it green. WDYT?

log_error "GitHub authentication failed for gh. Please run gh auth login."
exit 1
fi

# If the release tag does not already exist remotely, create it.
log_callout "Create tag if not present"
if [ "${remote_tag_exists}" -eq 0 ]; then
Expand Down Expand Up @@ -314,9 +330,41 @@ main() {
exit 1
fi

# TODO: signing process
local gh_repo
if [ "${REPOSITORY}" = "$(pwd)" ]; then
gh_repo=$(git remote get-url origin)
else
gh_repo="${REPOSITORY}"
fi

gh_repo=$(echo "${gh_repo}" | sed 's/^[^@]\+@//' | sed 's/https\?:\/\///' | sed 's/\.git$//' | tr ':' '/')
log_callout "Creating GitHub release for ${RELEASE_VERSION} on ${gh_repo}"

# Disable sellcheck SC2016, the single quoted syntax for sed is intentional.
# shellcheck disable=SC2016
local release_notes_temp_file=$(mktemp)
sed 's/${RELEASE_VERSION}/'"${RELEASE_VERSION}"'/' ./scripts/release_notes.tpl.txt > "${release_notes_temp_file}"
maybe_run gh release create "${RELEASE_VERSION}" \
--repo "${gh_repo}" \
--draft \
--prerelease \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to be marking as pre-release? I think this would depend on which branch we were creating a release for? I.e. if we were publishing an alpha/rc for 3.6.0 vs publishing a patch release for 3.5 or 3.4.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For main, it should be --prerelease, for release-3.5, it should be --latest and for release-3.4, it should not have an additional option.

I'm open to documenting it or adding a branch check. Let me know your thoughts.

--title "${RELEASE_VERSION}" \
--notes-file "${release_notes_temp_file}" \
./release/SHA256SUMS

# Upload files one by one, as gh doesn't support passing globs as input.
maybe_run find ./release '(' -name '*.tar.gz' -o -name '*.zip' ')' -exec \
gh --repo "${gh_repo}" release upload "${RELEASE_VERSION}" {} \;

local release_url
if [ "${DRY_RUN}" == "true" ]; then
release_url="<to be provided when running with DRY_MODE=false>"
else
release_url=$(gh --repo "${gh_repo}" release view "${RELEASE_VERSION}" --json url --jq '.url')
fi

log_warning ""
log_warning "WARNING: The release has not been signed and published to github. This must be done manually."
log_warning "WARNING: The GitHub release for ${RELEASE_VERSION} has been created as a draft, please go to ${release_url} and release it."
log_warning ""
log_success "Success."
exit 0
Expand Down
91 changes: 91 additions & 0 deletions scripts/release_notes.tpl.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
Please check out [CHANGELOG](https://github.com/etcd-io/etcd/blob/main/CHANGELOG/CHANGELOG-3.5.md) for a full list of changes. And make sure to read [upgrade guide](https://github.com/etcd-io/website/blob/main/content/en/docs/v3.6/upgrades/upgrade_3_6.md) before upgrading etcd (there may be breaking changes).

For installation guides, please check out [play.etcd.io](http://play.etcd.io) and [operating etcd](https://etcd.io/docs/v3.6/op-guide/). Latest support status for common architectures and operating systems can be found at [supported platforms](https://etcd.io/docs/v3.6/op-guide/supported-platform/).

###### Linux

```sh
ETCD_VER=${RELEASE_VERSION}

# choose either URL
GOOGLE_URL=https://storage.googleapis.com/etcd
GITHUB_URL=https://github.com/etcd-io/etcd/releases/download
DOWNLOAD_URL=${GOOGLE_URL}

rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
rm -rf /tmp/etcd-download-test && mkdir -p /tmp/etcd-download-test

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz
tar xzvf /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz -C /tmp/etcd-download-test --strip-components=1
rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz

/tmp/etcd-download-test/etcd --version
/tmp/etcd-download-test/etcdctl version
/tmp/etcd-download-test/etcdutl version

# start a local etcd server
/tmp/etcd-download-test/etcd

# write,read to etcd
/tmp/etcd-download-test/etcdctl --endpoints=localhost:2379 put foo bar
/tmp/etcd-download-test/etcdctl --endpoints=localhost:2379 get foo
```

###### macOS (Darwin)

```sh
ETCD_VER=${RELEASE_VERSION}

# choose either URL
GOOGLE_URL=https://storage.googleapis.com/etcd
GITHUB_URL=https://github.com/etcd-io/etcd/releases/download
DOWNLOAD_URL=${GOOGLE_URL}

rm -f /tmp/etcd-${ETCD_VER}-darwin-amd64.zip
rm -rf /tmp/etcd-download-test && mkdir -p /tmp/etcd-download-test

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-darwin-amd64.zip -o /tmp/etcd-${ETCD_VER}-darwin-amd64.zip
unzip /tmp/etcd-${ETCD_VER}-darwin-amd64.zip -d /tmp && rm -f /tmp/etcd-${ETCD_VER}-darwin-amd64.zip
mv /tmp/etcd-${ETCD_VER}-darwin-amd64/* /tmp/etcd-download-test && rm -rf mv /tmp/etcd-${ETCD_VER}-darwin-amd64

/tmp/etcd-download-test/etcd --version
/tmp/etcd-download-test/etcdctl version
/tmp/etcd-download-test/etcdutl version
```

###### Docker

etcd uses [`gcr.io/etcd-development/etcd`](https://gcr.io/etcd-development/etcd) as a primary container registry, and [`quay.io/coreos/etcd`](https://quay.io/coreos/etcd) as secondary.

```sh
ETCD_VER=${RELEASE_VERSION}

rm -rf /tmp/etcd-data.tmp && mkdir -p /tmp/etcd-data.tmp && \
docker rmi gcr.io/etcd-development/etcd:${ETCD_VER} || true && \
docker run \
-p 2379:2379 \
-p 2380:2380 \
--mount type=bind,source=/tmp/etcd-data.tmp,destination=/etcd-data \
--name etcd-gcr-${ETCD_VER} \
gcr.io/etcd-development/etcd:${ETCD_VER} \
/usr/local/bin/etcd \
--name s1 \
--data-dir /etcd-data \
--listen-client-urls http://0.0.0.0:2379 \
--advertise-client-urls http://0.0.0.0:2379 \
--listen-peer-urls http://0.0.0.0:2380 \
--initial-advertise-peer-urls http://0.0.0.0:2380 \
--initial-cluster s1=http://0.0.0.0:2380 \
--initial-cluster-token tkn \
--initial-cluster-state new \
--log-level info \
--logger zap \
--log-outputs stderr

docker exec etcd-gcr-${ETCD_VER}/usr/local/bin/etcd --version
docker exec etcd-gcr-${ETCD_VER}/usr/local/bin/etcdctl version
docker exec etcd-gcr-${ETCD_VER}/usr/local/bin/etcdutl version
docker exec etcd-gcr-${ETCD_VER}/usr/local/bin/etcdctl endpoint health
docker exec etcd-gcr-${ETCD_VER}/usr/local/bin/etcdctl put foo bar
docker exec etcd-gcr-${ETCD_VER}/usr/local/bin/etcdctl get foo
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is the markdown for a recent release if you wanted to diff it:
release.txt

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, it looks the same. I guess, maybe it would be a good idea to update the second paragraph (at least in the two latest release [3.5.16 and 3.4.34]) to the following (to address the broken links):

For installation guides, please check out [play.etcd.io](http://play.etcd.io) and [operating etcd](https://etcd.io/docs/v3.6/op-guide). Latest support status for common architectures and operating systems can be found at [supported platforms](https://github.com/etcd-io/website/blob/main/content/en/docs/v3.5/op-guide/supported-platform.md).

Loading