release, documentation, tools: Expand patch management support to the previous two minor versions

Documentation/branch_management.md

@@ -7,7 +7,7 @@
 * Backwards-compatible bug fixes should target the master branch and subsequently be ported to stable branches.
 * Once the master branch is ready for release, it will be tagged and become the new stable branch.
 
-The etcd team has adopted a *rolling release model* and supports one stable version of etcd.
+The etcd team has adopted a *rolling release model* and supports two stable versions of etcd.
 
 ### Master branch
 
@@ -21,6 +21,6 @@ Before the release of the next stable version, feature PRs will be frozen. We wi
 
 All branches with prefix `release-` are considered _stable_ branches.
 
-After every minor release (http://semver.org/), we will have a new stable branch for that release. We will keep fixing the backwards-compatible bugs for the latest stable release, but not previous releases. The _patch_ release, incorporating any bug fixes, will be once every two weeks, given any patches.
+After every minor release (http://semver.org/), we will have a new stable branch for that release. We will keep fixing the backwards-compatible bugs for the latest two stable releases. A _patch_ release to each supported release branch, incorporating any bug fixes, will be once every two weeks, given any patches.
 
 [master]: https://github.com/coreos/etcd/tree/master

Documentation/dev-internal/release.md

@@ -25,8 +25,10 @@ All releases version numbers follow the format of [semantic versioning 2.0.0](ht
 
 ### Patch version release
 
-- Discuss about commits that are backported to the patch release. The commits should not include merge commits.
-- Cherry-pick these commits starting from the oldest one into stable branch.
+- To request a backport, devlopers submit cherrypick PRs targeting the release branch. The commits should not include merge commits. The commits should be restricted to bug fixes and security patches.
+- The cherrypick PRs should target the appropriate release branch (`base:release-<major>-<minor>`). `hack/patch/cherrypick.sh` may be used to automatically generate cherrypick PRs.
+- The release patch manager reviews the cherrypick PRs. Please discuss carefully what is backported to the patch release. Each patch release should be strictly better than it's predecessor.
+- The release patch manager will cherry-pick these commits starting from the oldest one into stable branch.
 
 ## Write release note
 

hack/patch/README.md

@@ -0,0 +1,36 @@
+# hack/cherrypick.sh
+
+Handles cherry-picks of PR(s) from etcd master to a stable etcd release branch automatically.
+
+## Setup
+
+Set the `UPSTREAM_REMOTE` and `FORK_REMOTE` environment variables.
+`UPSTREAM_REMOTE` should be set to git remote name of `github.com/coreos/etcd`,
+and `FORK_REMOTE` should be set to the git remote name of your fork of the etcd
+repo (`github.com/${github-username}/etcd`). Use `git remotes -v` if you need to
+look up your git remote names. If you don't already have a fork of etcd create
+one on github.com and register it locally with `git remote add ...`.
+
+
+```
+export UPSTREAM_REMOTE=origin
+export FORK_REMOTE=${github-username}
+```
+
+Next, install hub from https://github.com/github/hub
+
+## Usage
+
+To cherry pick PR 12345 onto release-2.22 and propose is as a PR, run:
+
+```sh
+hack/cherrypick.sh upstream/release-2.2 12345
+```
+
+To cherry pick 12345 then 56789 and propose them togther as a single PR, run:
+
+```
+hack/cherrypick.sh upstream/release-2.2 12345 56789
+```
+
+

hack/patch/cherrypick.sh

@@ -0,0 +1,229 @@
+#!/usr/bin/env bash
+
+# Based on github.com/kubernetes/kubernetes/blob/v1.8.2/hack/cherry_pick_pull.sh
+
+# Checkout a PR from GitHub. (Yes, this is sitting in a Git tree. How
+# meta.) Assumes you care about pulls from remote "upstream" and
+# checks thems out to a branch named:
+#  automated-cherry-pick-of-<pr>-<target branch>-<timestamp>
+
+set -o errexit
+set -o nounset
+set -o pipefail
+
+declare -r ETCD_ROOT="$(dirname "${BASH_SOURCE}")/../.."
+cd "${ETCD_ROOT}"
+
+declare -r STARTINGBRANCH=$(git symbolic-ref --short HEAD)
+declare -r REBASEMAGIC="${ETCD_ROOT}/.git/rebase-apply"
+DRY_RUN=${DRY_RUN:-""}
+REGENERATE_DOCS=${REGENERATE_DOCS:-""}
+UPSTREAM_REMOTE=${UPSTREAM_REMOTE:-upstream}
+FORK_REMOTE=${FORK_REMOTE:-origin}
+
+if [[ -z ${GITHUB_USER:-} ]]; then
+  echo "Please export GITHUB_USER=<your-user> (or GH organization, if that's where your fork lives)"
+  exit 1
+fi
+
+if ! which hub > /dev/null; then
+  echo "Can't find 'hub' tool in PATH, please install from https://github.com/github/hub"
+  exit 1
+fi
+
+if [[ "$#" -lt 2 ]]; then
+  echo "${0} <remote branch> <pr-number>...: cherry pick one or more <pr> onto <remote branch> and leave instructions for proposing pull request"
+  echo
+  echo "  Checks out <remote branch> and handles the cherry-pick of <pr> (possibly multiple) for you."
+  echo "  Examples:"
+  echo "    $0 upstream/release-3.14 12345        # Cherry-picks PR 12345 onto upstream/release-3.14 and proposes that as a PR."
+  echo "    $0 upstream/release-3.14 12345 56789  # Cherry-picks PR 12345, then 56789 and proposes the combination as a single PR."
+  echo
+  echo "  Set the DRY_RUN environment var to skip git push and creating PR."
+  echo "  This is useful for creating patches to a release branch without making a PR."
+  echo "  When DRY_RUN is set the script will leave you in a branch containing the commits you cherry-picked."
+  echo
+  echo "  Set the REGENERATE_DOCS environment var to regenerate documentation for the target branch after picking the specified commits."
+  echo "  This is useful when picking commits containing changes to API documentation."
+  echo
+  echo " Set UPSTREAM_REMOTE (default: upstream) and FORK_REMOTE (default: origin)"
+  echo " To override the default remote names to what you have locally."
+  exit 2
+fi
+
+if git_status=$(git status --porcelain --untracked=no 2>/dev/null) && [[ -n "${git_status}" ]]; then
+  echo "!!! Dirty tree. Clean up and try again."
+  exit 1
+fi
+
+if [[ -e "${REBASEMAGIC}" ]]; then
+  echo "!!! 'git rebase' or 'git am' in progress. Clean up and try again."
+  exit 1
+fi
+
+declare -r BRANCH="$1"
+shift 1
+declare -r PULLS=( "$@" )
+
+function join { local IFS="$1"; shift; echo "$*"; }
+declare -r PULLDASH=$(join - "${PULLS[@]/#/#}") # Generates something like "#12345-#56789"
+declare -r PULLSUBJ=$(join " " "${PULLS[@]/#/#}") # Generates something like "#12345 #56789"
+
+echo "+++ Updating remotes..."
+git remote update "${UPSTREAM_REMOTE}" "${FORK_REMOTE}"
+
+if ! git log -n1 --format=%H "${BRANCH}" >/dev/null 2>&1; then
+  echo "!!! '${BRANCH}' not found. The second argument should be something like ${UPSTREAM_REMOTE}/release-0.21."
+  echo "    (In particular, it needs to be a valid, existing remote branch that I can 'git checkout'.)"
+  exit 1
+fi
+
+declare -r NEWBRANCHREQ="automated-cherry-pick-of-${PULLDASH}" # "Required" portion for tools.
+declare -r NEWBRANCH="$(echo "${NEWBRANCHREQ}-${BRANCH}" | sed 's/\//-/g')"
+declare -r NEWBRANCHUNIQ="${NEWBRANCH}-$(date +%s)"
+echo "+++ Creating local branch ${NEWBRANCHUNIQ}"
+
+cleanbranch=""
+prtext=""
+gitamcleanup=false
+function return_to_kansas {
+  if [[ "${gitamcleanup}" == "true" ]]; then
+    echo
+    echo "+++ Aborting in-progress git am."
+    git am --abort >/dev/null 2>&1 || true
+  fi
+
+  # return to the starting branch and delete the PR text file
+  if [[ -z "${DRY_RUN}" ]]; then
+    echo
+    echo "+++ Returning you to the ${STARTINGBRANCH} branch and cleaning up."
+    git checkout -f "${STARTINGBRANCH}" >/dev/null 2>&1 || true
+    if [[ -n "${cleanbranch}" ]]; then
+      git branch -D "${cleanbranch}" >/dev/null 2>&1 || true
+    fi
+    if [[ -n "${prtext}" ]]; then
+      rm "${prtext}"
+    fi
+  fi
+}
+trap return_to_kansas EXIT
+
+SUBJECTS=()
+function make-a-pr() {
+  local rel="$(basename "${BRANCH}")"
+  echo
+  echo "+++ Creating a pull request on GitHub at ${GITHUB_USER}:${NEWBRANCH}"
+
+  # This looks like an unnecessary use of a tmpfile, but it avoids
+  # https://github.com/github/hub/issues/976 Otherwise stdin is stolen
+  # when we shove the heredoc at hub directly, tickling the ioctl
+  # crash.
+  prtext="$(mktemp -t prtext.XXXX)" # cleaned in return_to_kansas
+  cat >"${prtext}" <<EOF
+Automated cherry pick of ${PULLSUBJ}
+
+Cherry pick of ${PULLSUBJ} on ${rel}.
+
+$(printf '%s\n' "${SUBJECTS[@]}")
+EOF
+
+  hub pull-request -F "${prtext}" -h "${GITHUB_USER}:${NEWBRANCH}" -b "etcd:${rel}"
+}
+
+git checkout -b "${NEWBRANCHUNIQ}" "${BRANCH}"
+cleanbranch="${NEWBRANCHUNIQ}"
+
+gitamcleanup=true
+for pull in "${PULLS[@]}"; do
+  echo "+++ Downloading patch to /tmp/${pull}.patch (in case you need to do this again)"
+  curl -o "/tmp/${pull}.patch" -sSL "http://github.com/coreos/etcd/pull/${pull}.patch"
+  echo
+  echo "+++ About to attempt cherry pick of PR. To reattempt:"
+  echo "  $ git am -3 /tmp/${pull}.patch"
+  echo
+  git am -3 "/tmp/${pull}.patch" || {
+    conflicts=false
+    while unmerged=$(git status --porcelain | grep ^U) && [[ -n ${unmerged} ]] \
+      || [[ -e "${REBASEMAGIC}" ]]; do
+      conflicts=true # <-- We should have detected conflicts once
+      echo
+      echo "+++ Conflicts detected:"
+      echo
+      (git status --porcelain | grep ^U) || echo "!!! None. Did you git am --continue?"
+      echo
+      echo "+++ Please resolve the conflicts in another window (and remember to 'git add / git am --continue')"
+      read -p "+++ Proceed (anything but 'y' aborts the cherry-pick)? [y/n] " -r
+      echo
+      if ! [[ "${REPLY}" =~ ^[yY]$ ]]; then
+        echo "Aborting." >&2
+        exit 1
+      fi
+    done
+
+    if [[ "${conflicts}" != "true" ]]; then
+      echo "!!! git am failed, likely because of an in-progress 'git am' or 'git rebase'"
+      exit 1
+    fi
+  }
+
+  # set the subject
+  subject=$(grep -m 1 "^Subject" "/tmp/${pull}.patch" | sed -e 's/Subject: \[PATCH//g' | sed 's/.*] //')
+  SUBJECTS+=("#${pull}: ${subject}")
+
+  # remove the patch file from /tmp
+  rm -f "/tmp/${pull}.patch"
+done
+gitamcleanup=false
+
+# Re-generate docs (if needed)
+if [[ -n "${REGENERATE_DOCS}" ]]; then
+  echo
+  echo "Regenerating docs..."
+  if ! hack/generate-docs.sh; then
+    echo
+    echo "hack/generate-docs.sh FAILED to complete."
+    exit 1
+  fi
+fi
+
+if [[ -n "${DRY_RUN}" ]]; then
+  echo "!!! Skipping git push and PR creation because you set DRY_RUN."
+  echo "To return to the branch you were in when you invoked this script:"
+  echo
+  echo "  git checkout ${STARTINGBRANCH}"
+  echo
+  echo "To delete this branch:"
+  echo
+  echo "  git branch -D ${NEWBRANCHUNIQ}"
+  exit 0
+fi
+
+if git remote -v | grep ^${FORK_REMOTE} | grep etcd/etcd.git; then
+  echo "!!! You have ${FORK_REMOTE} configured as your etcd/etcd.git"
+  echo "This isn't normal. Leaving you with push instructions:"
+  echo
+  echo "+++ First manually push the branch this script created:"
+  echo
+  echo "  git push REMOTE ${NEWBRANCHUNIQ}:${NEWBRANCH}"
+  echo
+  echo "where REMOTE is your personal fork (maybe ${UPSTREAM_REMOTE}? Consider swapping those.)."
+  echo "OR consider setting UPSTREAM_REMOTE and FORK_REMOTE to different values."
+  echo
+  make-a-pr
+  cleanbranch=""
+  exit 0
+fi
+
+echo
+echo "+++ I'm about to do the following to push to GitHub (and I'm assuming ${FORK_REMOTE} is your personal fork):"
+echo
+echo "  git push ${FORK_REMOTE} ${NEWBRANCHUNIQ}:${NEWBRANCH}"
+echo
+read -p "+++ Proceed (anything but 'y' aborts the cherry-pick)? [y/n] " -r
+if ! [[ "${REPLY}" =~ ^[yY]$ ]]; then
+  echo "Aborting." >&2
+  exit 1
+fi
+
+git push "${FORK_REMOTE}" -f "${NEWBRANCHUNIQ}:${NEWBRANCH}"
+make-a-pr