Skip to content

Commit

Permalink
Add batch merge and refspec support to git task
Browse files Browse the repository at this point in the history
Add refspec support to git task corresponding to tektoncd/pipeline#2320

Add a new batch merge task (tektoncd/pipeline#508) that re-uses
the git-init container and subsequently performs the requested batch merge mode on top of
the checked out revision
  • Loading branch information
anish-dangi-wdc committed Apr 28, 2020
1 parent db51b07 commit f942e9f
Show file tree
Hide file tree
Showing 4 changed files with 269 additions and 68 deletions.
31 changes: 31 additions & 0 deletions git/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ as well as

* **url**: git url to clone (_required_)
* **revision**: git revision to checkout (branch, tag, sha, ref…) (_default:_ master)
* **refspec**: git refspec to fetch before checking out revision (_default_:refs/heads/master:refs/heads/master)
* **submodules**: defines if the resource should initialize and fetch the submodules (_default_: true)
* **depth**: performs a shallow clone where only the most recent commit(s) will be fetched (_default_: 1)
* **sslVerify**: defines if http.sslVerify should be set to true or false in the global git config (_default_: true)
Expand Down Expand Up @@ -135,3 +136,33 @@ spec:
persistentVolumeClaim:
claimName: workspace-pvc
```

## `git-batch-merge`

This `Task` has four required inputs:

1. The URL of a git repo to clone provided with the `url` param.
1. A space separated string of refs `BatchedRefs` to fetch and batch over the given `revision`
1. Merge `mode` to use while batching (merge, merge-resolve, merge-squash, cherry-pick)
1. A Workspace called `output`.

There are 4 additional parameters in addition to the ones mentioned above for the git-clone task:
* **batchedRefs**: space separated git [refnames][git-ref] to fetch and batch on top of revision using the given mode
(must be a valid refs, no commit SHA's).
* **mode**: Batch mode to select (_default_: merge) <br>
&nbsp;&nbsp;`merge`: corresponds to git merge -s recursive. This is the default mode used by github <br>
&nbsp;&nbsp;`cherry-pick`: corresponds to git cherry-pick <br>
See [git-merge][git-merge] and [git-cherry-pick][git-cherry-pick]
* **gitUserName**: git user name to use for creating the batched commit (First Last)
(_default_: GitBatch Task). See [git-user-config][git-user-config]
* **gitUserEmail**: git user email to use for creating the batched commit ([email protected])
(_default_: [email protected]). See [git-user-config][git-user-config]

There is 1 additional result in addition to the one(s) mentioned for the git-clone task
* **tree**: The [git tree][git-tree] object SHA1 that was created after batch merging the refs on HEAD.

[git-ref]: https://git-scm.com/book/en/v2/Git-Internals-Git-References
[git-merge]: https://git-scm.com/docs/git-merge
[git-cherry-pick]: https://git-scm.com/docs/git-cherry-pick
[git-user-config]: https://git-scm.com/docs/git-config#Documentation/git-config.txt-username
[git-tree]: https://git-scm.com/book/en/v2/Git-Internals-Git-Objects
118 changes: 118 additions & 0 deletions git/git-batch-merge.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,118 @@
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: git-batch-merge
namespace: tekton-pipelines
spec:
workspaces:
- name: output
description: The git repo will be cloned onto the volume backing this workspace
params:
- name: url
description: git url to clone
type: string
- name: revision
description: base git revision to checkout (branch, tag, sha, ref…)
default: master
- name: refspec
description: base git refspec to fetch before checking out revision
default: "refs/heads/master:refs/heads/master"
- name: batchedRefs
description: git refs to fetch and batch on top of revision using the given mode (must be a valid refname, no commit SHA's)
- name: gitUserName
description: git user name to use for creating the batched commit (First Last)
default: GitBatch Task
- name: gitUserEmail
description: git user email to use for creating the batched commit ([email protected])
default: [email protected]
- name: mode
description: git operation to perform while batching (choose from merge, cherry-pick)
type: string
default: merge
- name: submodules
description: defines if the resource should initialize and fetch the submodules
type: string
default: "true"
- name: depth
description: performs a shallow clone where only the most recent commit(s) will be fetched
type: string
default: "1"
- name: sslVerify
description: defines if http.sslVerify should be set to true or false in the global git config
type: string
default: "true"
- name: subdirectory
description: subdirectory inside the "output" workspace to clone the git repo into
type: string
default: "src"
- name: deleteExisting
description: clean out the contents of the repo's destination directory (if it already exists) before trying to clone the repo there
type: string
default: "false"
results:
- name: commit
description: The final commit SHA that was obtained after batching all provided refs onto revision
- name: tree
description: The git tree SHA that was obtained after batching all provided refs onto revision.
steps:
- name: clone
image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.12.0
script: |
CHECKOUT_DIR="$(workspaces.output.path)/$(params.subdirectory)"
cleandir() {
# Delete any existing contents of the repo directory if it exists.
#
# We don't just "rm -rf $CHECKOUT_DIR" because $CHECKOUT_DIR might be "/"
# or the root of a mounted volume.
if [[ -d "$CHECKOUT_DIR" ]] ; then
# Delete non-hidden files and directories
rm -rf "$CHECKOUT_DIR"/*
# Delete files and directories starting with . but excluding ..
rm -rf "$CHECKOUT_DIR"/.[!.]*
# Delete files and directories starting with .. plus any other character
rm -rf "$CHECKOUT_DIR"/..?*
fi
}
if [[ "$(params.deleteExisting)" == "true" ]] ; then
cleandir
fi
p="$(params.batchedRefs)"
refs="$(params.refspec)"
for ref in $p; do
refs="$refs $ref:refs/batch/$ref"
done
/ko-app/git-init \
-url "$(params.url)" \
-revision "$(params.revision)" \
-refspec "$refs" \
-path "$CHECKOUT_DIR" \
-sslVerify="$(params.sslVerify)" \
-submodules="$(params.submodules)" \
-depth "$(params.depth)"
git -C $CHECKOUT_DIR config user.name "$(params.gitUserName)"
git -C $CHECKOUT_DIR config user.email "$(params.gitUserEmail)"
mode="$(params.mode)"
if [[ $mode == "merge" ]]; then
for ref in $p; do
git -C $CHECKOUT_DIR merge --quiet --allow-unrelated-histories refs/batch/$ref
done
elif [[ $mode == "cherry-pick" ]]; then
for ref in $p; do
git -C $CHECKOUT_DIR cherry-pick --allow-empty --keep-redundant-commits refs/batch/$ref
done
else
echo "unsupported mode $mode"
exit 1
fi
RESULT_SHA="$(git -C $CHECKOUT_DIR rev-parse HEAD)"
TREE_SHA="$(git -C $CHECKOUT_DIR rev-parse HEAD^{tree})"
# Make sure we don't add a trailing newline to the result!
echo -n "$(echo $RESULT_SHA | tr -d '\n')" > $(results.commit.path)
echo -n "$(echo $TREE_SHA | tr -d '\n')" > $(results.tree.path)
141 changes: 73 additions & 68 deletions git/git-clone.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,79 +2,84 @@ apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: git-clone
namespace: tekton-pipelines
spec:
workspaces:
- name: output
description: The git repo will be cloned onto the volume backing this workspace
- name: output
description: The git repo will be cloned onto the volume backing this workspace
params:
- name: url
description: git url to clone
type: string
- name: revision
description: git revision to checkout (branch, tag, sha, ref…)
type: string
default: master
- name: submodules
description: defines if the resource should initialize and fetch the submodules
type: string
default: "true"
- name: depth
description: performs a shallow clone where only the most recent commit(s) will be fetched
type: string
default: "1"
- name: sslVerify
description: defines if http.sslVerify should be set to true or false in the global git config
type: string
default: "true"
- name: subdirectory
description: subdirectory inside the "output" workspace to clone the git repo into
type: string
default: ""
- name: deleteExisting
description: clean out the contents of the repo's destination directory (if it already exists) before trying to clone the repo there
type: string
default: "false"
- name: url
description: git url to clone
type: string
- name: revision
description: git revision to checkout (branch, tag, sha, ref…)
type: string
default: master
- name: refspec
description: (optional) git refspec to fetch before checking out revision
default: ""
- name: submodules
description: defines if the resource should initialize and fetch the submodules
type: string
default: "true"
- name: depth
description: performs a shallow clone where only the most recent commit(s) will be fetched
type: string
default: "1"
- name: sslVerify
description: defines if http.sslVerify should be set to true or false in the global git config
type: string
default: "true"
- name: subdirectory
description: subdirectory inside the "output" workspace to clone the git repo into
type: string
default: ""
- name: deleteExisting
description: clean out the contents of the repo's destination directory (if it already exists) before trying to clone the repo there
type: string
default: "false"
results:
- name: commit
description: The precise commit SHA that was fetched by this Task
- name: commit
description: The precise commit SHA that was fetched by this Task
steps:
- name: clone
image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.11.0
script: |
CHECKOUT_DIR="$(workspaces.output.path)/$(params.subdirectory)"
- name: clone
image: gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.12.0
script: |
CHECKOUT_DIR="$(workspaces.output.path)/$(params.subdirectory)"
cleandir() {
# Delete any existing contents of the repo directory if it exists.
#
# We don't just "rm -rf $CHECKOUT_DIR" because $CHECKOUT_DIR might be "/"
# or the root of a mounted volume.
if [[ -d "$CHECKOUT_DIR" ]] ; then
# Delete non-hidden files and directories
rm -rf "$CHECKOUT_DIR"/*
# Delete files and directories starting with . but excluding ..
rm -rf "$CHECKOUT_DIR"/.[!.]*
# Delete files and directories starting with .. plus any other character
rm -rf "$CHECKOUT_DIR"/..?*
fi
}
cleandir() {
# Delete any existing contents of the repo directory if it exists.
#
# We don't just "rm -rf $CHECKOUT_DIR" because $CHECKOUT_DIR might be "/"
# or the root of a mounted volume.
if [[ -d "$CHECKOUT_DIR" ]] ; then
# Delete non-hidden files and directories
rm -rf "$CHECKOUT_DIR"/*
# Delete files and directories starting with . but excluding ..
rm -rf "$CHECKOUT_DIR"/.[!.]*
# Delete files and directories starting with .. plus any other character
rm -rf "$CHECKOUT_DIR"/..?*
fi
}
if [[ "$(params.deleteExisting)" == "true" ]] ; then
cleandir
fi
if [[ "$(params.deleteExisting)" == "true" ]] ; then
cleandir
fi
/ko-app/git-init \
-url "$(params.url)" \
-revision "$(params.revision)" \
-path "$CHECKOUT_DIR" \
-sslVerify="$(params.sslVerify)" \
-submodules="$(params.submodules)" \
-depth="$(params.depth)"
cd "$CHECKOUT_DIR"
RESULT_SHA="$(git rev-parse HEAD | tr -d '\n')"
EXIT_CODE="$?"
if [ "$EXIT_CODE" != 0 ]
then
exit $EXIT_CODE
fi
# Make sure we don't add a trailing newline to the result!
echo -n "$RESULT_SHA" > $(results.commit.path)
/ko-app/git-init \
-url "$(params.url)" \
-revision "$(params.revision)" \
-refspec "$(params.refspec)" \
-path "$CHECKOUT_DIR" \
-sslVerify="$(params.sslVerify)" \
-submodules="$(params.submodules)" \
-depth "$(params.depth)"
cd "$CHECKOUT_DIR"
RESULT_SHA="$(git rev-parse HEAD | tr -d '\n')"
EXIT_CODE="$?"
if [ "$EXIT_CODE" != 0 ]
then
exit $EXIT_CODE
fi
# Make sure we don't add a trailing newline to the result!
echo -n "$RESULT_SHA" > $(results.commit.path)
Loading

0 comments on commit f942e9f

Please sign in to comment.