Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce optional "refspec" to git PipelineResource, refactor #2320

Merged
merged 1 commit into from
Apr 17, 2020
Merged

Introduce optional "refspec" to git PipelineResource, refactor #2320

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 4 additions & 9 deletions cmd/git-init/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,16 @@ import (

var (
fetchSpec git.FetchSpec
submodules bool
terminationMessagePath string
)

func init() {
flag.StringVar(&fetchSpec.URL, "url", "", "Git origin URL to fetch")
flag.StringVar(&fetchSpec.Revision, "revision", "", "The Git revision to make the repository HEAD")
flag.StringVar(&fetchSpec.Refspec, "refspec", "", "The Git refspec to fetch the revision from (optional)")
flag.StringVar(&fetchSpec.Path, "path", "", "Path of directory under which Git repository will be copied")
flag.BoolVar(&fetchSpec.SSLVerify, "sslVerify", true, "Enable/Disable SSL verification in the git config")
flag.BoolVar(&submodules, "submodules", true, "Initialize and fetch Git submodules")
flag.BoolVar(&fetchSpec.Submodules, "submodules", true, "Initialize and fetch Git submodules")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change does not seem to be needed for this patch, but I guess it makes sense anyways.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is actually needed for the patch, to pass the submodule setting to git.go and prevent --recurse-submodules from being appended to the fetch command when
the user specified submodule: "false"

flag.UintVar(&fetchSpec.Depth, "depth", 1, "Perform a shallow clone to this depth")
flag.StringVar(&terminationMessagePath, "terminationMessagePath", "/tekton/termination", "Location of file containing termination message")
}
Expand All @@ -53,15 +53,10 @@ func main() {
if err := git.Fetch(logger, fetchSpec); err != nil {
logger.Fatalf("Error fetching git repository: %s", err)
}
if submodules {
if err := git.SubmoduleFetch(logger, fetchSpec.Path); err != nil {
logger.Fatalf("Error initializing or fetching the git submodules")
}
}

commit, err := git.Commit(logger, fetchSpec.Revision, fetchSpec.Path)
commit, err := git.ShowCommit(logger, "HEAD", fetchSpec.Path)
if err != nil {
logger.Fatalf("Error parsing commit of git repository: %s", err)
logger.Fatalf("Error parsing revision %s of git repository: %s", fetchSpec.Revision, err)
}
resourceName := os.Getenv("TEKTON_RESOURCE_NAME")
output := []v1alpha1.PipelineResourceResult{
Expand Down
42 changes: 38 additions & 4 deletions docs/resources.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -313,19 +313,53 @@ Params that can be added are the following:
change the repo, e.g. [to use a fork](#using-a-fork)
1. `revision`: Git [revision][git-rev] (branch, tag, commit SHA or ref) to
clone. You can use this to control what commit [or branch](#using-a-branch)
is used. _If no revision is specified, the resource will default to `latest`
from `master`._
is used. [git checkout][git-checkout] is used to switch to the
revision, and will result in a detached HEAD in most cases. Use refspec
along with revision if you want to checkout a particular branch without a
detached HEAD. _If no revision is specified, the resource will default to `master`._
1. `refspec`: (Optional) specify a git [refspec][git-refspec] to pass to git-fetch.
Note that if this field is specified, it must specify all refs, branches, tags,
or commits required to checkout the specified `revision`. An additional fetch
will not be run to obtain the contents of the revision field. If no refspec
is specified, the value of the `revision` field will be fetched directly.
The refspec is useful in manipulating the repository in several cases:
* when the server does not support fetches via the commit SHA (i.e. does
not have `uploadpack.allowReachableSHA1InWant` enabled) and you want
to fetch and checkout a specific commit hash from a ref chain.
* when you want to fetch several other refs alongside your revision
(for instance, tags)
* when you want to checkout a specific branch, the revision and refspec
fields can work together to be able to set the destination of the incoming
branch and switch to the branch.

Examples:
- Check out a specified revision commit SHA1 after fetching ref (detached) <br>
&nbsp;&nbsp;`revision`: cb17eba165fe7973ef9afec20e7c6971565bd72f <br>
&nbsp;&nbsp;`refspec`: refs/smoke/myref <br>
- Fetch all tags alongside refs/heads/master and switch to the master branch
(not detached) <br>
&nbsp;&nbsp;`revision`: master <br>
&nbsp;&nbsp;`refspec`: "refs/tags/\*:refs/tags/\* +refs/heads/master:refs/heads/master"<br>
- Fetch the develop branch and switch to it (not detached) <br>
&nbsp;&nbsp;`revision`: develop <br>
&nbsp;&nbsp;`refspec`: refs/heads/develop:refs/heads/develop <br>
- Fetch refs/pull/1009/head into the master branch and switch to it (not detached) <br>
&nbsp;&nbsp;`revision`: master <br>
&nbsp;&nbsp;`refspec`: refs/pull/1009/head:refs/heads/master <br>

1. `submodules`: defines if the resource should initialize and fetch the
submodules, value is either `true` or `false`. _If not specified, this will
default to true_
1. `depth`: performs a [shallow clone][git-depth] where only the most recent
commit(s) will be fetched. If set to `'0'`, all commits will be fetched. _If
not specified, the default depth is 1._
commit(s) will be fetched. This setting also applies to submodules. If set to
`'0'`, all commits will be fetched. _If not specified, the default depth is 1._
1. `sslVerify`: defines if [http.sslVerify][git-http.sslVerify] should be set
to `true` or `false` in the global git config. _Defaults to `true` if
omitted._

[git-rev]: https://git-scm.com/docs/gitrevisions#_specifying_revisions
[git-checkout]: https://git-scm.com/docs/git-checkout
[git-refspec]: https://git-scm.com/book/en/v2/Git-Internals-The-Refspec
[git-depth]: https://git-scm.com/docs/git-clone#Documentation/git-clone.txt---depthltdepthgt
[git-http.sslVerify]: https://git-scm.com/docs/git-config#Documentation/git-config.txt-httpsslVerify

Expand Down
2 changes: 1 addition & 1 deletion pkg/apis/pipeline/v1alpha1/pipeline_resource_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ var (
)

const (
// PipelineResourceTypeGit indicates that this source is a GitHub repo.
// PipelineResourceTypeGit indicates that this source is a Git repo.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice :)

PipelineResourceTypeGit PipelineResourceType = resource.PipelineResourceTypeGit

// PipelineResourceTypeStorage indicates that this source is a storage blob resource.
Expand Down
13 changes: 10 additions & 3 deletions pkg/apis/resource/v1alpha1/git/git_resource.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,10 @@ type Resource struct {
Name string `json:"name"`
Type resource.PipelineResourceType `json:"type"`
URL string `json:"url"`
// Git revision (branch, tag, commit SHA or ref) to clone. See
// https://git-scm.com/docs/gitrevisions#_specifying_revisions for more
// information.
// Git revision (branch, tag, commit SHA) to clone, and optionally the refspec to fetch from.
//See https://git-scm.com/docs/gitrevisions#_specifying_revisions for more information.
Revision string `json:"revision"`
Refspec string `json:"refspec"`
Submodules bool `json:"submodules"`

Depth uint `json:"depth"`
Expand Down Expand Up @@ -71,6 +71,8 @@ func NewResource(gitImage string, r *resource.PipelineResource) (*Resource, erro
gitResource.URL = param.Value
case strings.EqualFold(param.Name, "Revision"):
gitResource.Revision = param.Value
case strings.EqualFold(param.Name, "Refspec"):
gitResource.Refspec = param.Value
case strings.EqualFold(param.Name, "Submodules"):
gitResource.Submodules = toBool(param.Value, true)
case strings.EqualFold(param.Name, "Depth"):
Expand Down Expand Up @@ -133,6 +135,8 @@ func (s *Resource) Replacements() map[string]string {
"type": s.Type,
"url": s.URL,
"revision": s.Revision,
"refspec": s.Refspec,
"submodules": strconv.FormatBool(s.Submodules),
"depth": strconv.FormatUint(uint64(s.Depth), 10),
"sslVerify": strconv.FormatBool(s.SSLVerify),
"httpProxy": s.HTTPProxy,
Expand All @@ -149,6 +153,9 @@ func (s *Resource) GetInputTaskModifier(_ *v1alpha1.TaskSpec, path string) (v1al
"-path", path,
}

if s.Refspec != "" {
args = append(args, "-refspec", s.Refspec)
}
if !s.Submodules {
args = append(args, "-submodules=false")
}
Expand Down
103 changes: 103 additions & 0 deletions pkg/apis/resource/v1alpha1/git/git_resource_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,7 @@ func TestNewGitResource_Valid(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "test",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 1,
Expand All @@ -72,6 +73,50 @@ func TestNewGitResource_Valid(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 1,
SSLVerify: true,
HTTPProxy: "",
HTTPSProxy: "",
NOProxy: "",
},
}, {
desc: "With Refspec",
pipelineResource: tb.PipelineResource("git-resource", "default",
tb.PipelineResourceSpec(resourcev1alpha1.PipelineResourceTypeGit,
tb.PipelineResourceSpecParam("URL", "[email protected]:test/test.git"),
tb.PipelineResourceSpecParam("Refspec", "refs/changes/22/222134"),
),
),
want: &git.Resource{
Name: "git-resource",
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "refs/changes/22/222134",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 1,
SSLVerify: true,
HTTPProxy: "",
HTTPSProxy: "",
NOProxy: "",
},
}, {
desc: "Without Refspec",
pipelineResource: tb.PipelineResource("git-resource", "default",
tb.PipelineResourceSpec(resourcev1alpha1.PipelineResourceTypeGit,
tb.PipelineResourceSpecParam("URL", "[email protected]:test/test.git"),
),
),
want: &git.Resource{
Name: "git-resource",
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 1,
Expand All @@ -93,6 +138,7 @@ func TestNewGitResource_Valid(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "test",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 1,
Expand All @@ -115,6 +161,7 @@ func TestNewGitResource_Valid(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "test",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: false,
Depth: 1,
Expand All @@ -137,6 +184,7 @@ func TestNewGitResource_Valid(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "test",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 8,
Expand All @@ -159,6 +207,7 @@ func TestNewGitResource_Valid(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "test",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 0,
Expand All @@ -182,6 +231,7 @@ func TestNewGitResource_Valid(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "test",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 0,
Expand All @@ -205,6 +255,7 @@ func TestNewGitResource_Valid(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "test",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 0,
Expand All @@ -228,6 +279,7 @@ func TestNewGitResource_Valid(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "test",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 0,
Expand All @@ -251,6 +303,7 @@ func TestNewGitResource_Valid(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "test",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 0,
Expand Down Expand Up @@ -279,6 +332,8 @@ func TestGitResource_Replacements(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "",
Submodules: false,
Depth: 16,
SSLVerify: false,
HTTPProxy: "http-proxy.git.com",
Expand All @@ -291,6 +346,8 @@ func TestGitResource_Replacements(t *testing.T) {
"type": string(resourcev1alpha1.PipelineResourceTypeGit),
"url": "[email protected]:test/test.git",
"revision": "master",
"refspec": "",
"submodules": "false",
"depth": "16",
"sslVerify": "false",
"httpProxy": "http-proxy.git.com",
Expand Down Expand Up @@ -319,6 +376,7 @@ func TestGitResource_GetDownloadTaskModifier(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 1,
Expand Down Expand Up @@ -354,6 +412,7 @@ func TestGitResource_GetDownloadTaskModifier(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: false,
Depth: 1,
Expand Down Expand Up @@ -390,6 +449,7 @@ func TestGitResource_GetDownloadTaskModifier(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: true,
Depth: 8,
Expand Down Expand Up @@ -427,6 +487,7 @@ func TestGitResource_GetDownloadTaskModifier(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: false,
Depth: 1,
Expand Down Expand Up @@ -464,6 +525,7 @@ func TestGitResource_GetDownloadTaskModifier(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: false,
Depth: 1,
Expand Down Expand Up @@ -499,6 +561,7 @@ func TestGitResource_GetDownloadTaskModifier(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: false,
Depth: 1,
Expand Down Expand Up @@ -534,6 +597,7 @@ func TestGitResource_GetDownloadTaskModifier(t *testing.T) {
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "",
GitImage: "override-with-git:latest",
Submodules: false,
Depth: 1,
Expand Down Expand Up @@ -562,6 +626,45 @@ func TestGitResource_GetDownloadTaskModifier(t *testing.T) {
{Name: "HTTPS_PROXY", Value: "https-proxy.git.com"},
},
},
}, {
desc: "With Refspec",
gitResource: &git.Resource{
Name: "git-resource",
Type: resourcev1alpha1.PipelineResourceTypeGit,
URL: "[email protected]:test/test.git",
Revision: "master",
Refspec: "refs/tags/v1.0:refs/tags/v1.0 refs/heads/master:refs/heads/master",
GitImage: "override-with-git:latest",
Submodules: false,
Depth: 1,
SSLVerify: true,
HTTPProxy: "http-proxy.git.com",
HTTPSProxy: "https-proxy.git.com",
NOProxy: "no-proxy.git.com",
},
want: corev1.Container{
Name: "git-source-git-resource-l22wn",
Image: "override-with-git:latest",
Command: []string{"/ko-app/git-init"},
Args: []string{
"-url",
"[email protected]:test/test.git",
"-revision",
"master",
"-path",
"/test/test",
"-refspec",
"refs/tags/v1.0:refs/tags/v1.0 refs/heads/master:refs/heads/master",
"-submodules=false",
},
WorkingDir: "/workspace",
Env: []corev1.EnvVar{
{Name: "TEKTON_RESOURCE_NAME", Value: "git-resource"},
{Name: "HTTP_PROXY", Value: "http-proxy.git.com"},
{Name: "HTTPS_PROXY", Value: "https-proxy.git.com"},
{Name: "NO_PROXY", Value: "no-proxy.git.com"},
},
},
}} {
t.Run(tc.desc, func(t *testing.T) {
ts := pipelinev1alpha1.TaskSpec{}
Expand Down
Loading