pullRequest resource invalid upload path?

[castlemilk]

Expected Behavior

the pullRequest resource upload step should automating use the correct directory for uploading to a PR.

The input mounts to /workspace/<resource>/, would expect this folder to be used for the upload step?

Actual Behavior

upload step is looking in /workspace/output/<resource>/ which cannot be found because nothing has copied content there or this is an invalid directory to be using?

Steps to Reproduce the Problem

1. create a Pipeline which has a task that contains a pullRequest PipelineResource in both input and output resources as follows:

apiVersion: tekton.dev/v1alpha1
kind: Task
metadata:
  name: example
spec:
  inputs:
    params:
    - name: branch
      type: string
      default: master
    resources:
      - name: workspace
        type: git
      - name: pull-request
        type: pullRequest
  outputs:
    resources:
    - name: workspace
      type: git
    - name: pull-request
      type: pullRequest
  steps:
    - name: step-1
       ...

2. run task/pipeline and view upload step and you should see the following:

{"level":"info","ts":1575804730.2362704,"logger":"fallback-logger","caller":"logging/config.go:69","msg":"Fetch GitHub commit ID from kodata failed: open /var/run/ko/refs/heads/pr: no such file or directory"}
{"level":"info","ts":1575804730.2363453,"logger":"fallback-logger","caller":"pullrequest-init/main.go:56","msg":"RUNNING UPLOAD!"}
{"level":"fatal","ts":1575804730.2364097,"logger":"fallback-logger","caller":"pullrequest-init/main.go:59","msg":"open /workspace/output/pull-request/comments: no such file or directory","stacktrace":"main.main\n\t/workspace/go/src/github.com/tektoncd/pipeline/cmd/pullrequest-init/main.go:59\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:200"}
Step failed

Additional Info

tekton v0.9.1

[vincent-pli]

@castlemilk
The /workspace/output/<resource>/ is the default path of output resource, so it will try to find stuff there, that's correct.
In your case, you should copy stuff to that location in your steps before upload.
A workaround solution is set targetPath for output resource like this:

    - name: pull-request
      type: pullRequest
      targetPath: /workspace/pull-request

[vdemeester]

/kind question
/cc @wlynch

[castlemilk]

but the pullRequest resource is supposed to automatically manage the upload if used as an output resource, why should I have to copy it to /workspace/output/? seem like extra work.

[wlynch]

I don't think copying is necessary, as long as you set targetPath in your config to point to the correct directory.

@sbwsg Would this be solved/made easier by #1673?

[ghost]

We had a PR up for a while to pre-fill the output directory of a pull request resource but it was ultimately closed without merging: #1357

I think in general "output" resources are a bit confusing and the contract we've got now that requires their state to be placed in the outputs directory contributes to that.

I'm definitely hoping to alleviate some of the pain around read vs write with the work I'm doing now.

[bobcatfish]

but the pullRequest resource is supposed to automatically manage the upload if used as an output resource, why should I have to copy it to /workspace/output/? seem like extra work.

Some background: when the same resource was used as an input and an output, we used to make it so that instead of using /workspace/output, we'd use /workspace - this was strange b/c it meant (iirc) that the path the resources were at would change depending on what resources were bound at runtime, so we removed that functionality in #1122

In #1357 it was proposed to do what I think you're suggesting - to prefill the output dir with the contents of the corresponding input. At the time we thought the manifest based approach (#1357 (comment)) was at least enough to avoid accidentally wiping out PR data; however the issue you mentioned still exists and in fact if you look at recent examples by @dlorenc (e.g.https://medium.com/@dlorenc/building-a-chatops-bot-with-tekton-6d51c4f958f9) the first thing he does in every task that modifies a PR is copy all the data from the input to the output :

    - name: copy-pr
      image: ubuntu
      command: ["/bin/sh"]
      args:
      - -c
      - cp -r /workspace/pr/ /workspace/output/

Which come to think of it could be really weird if those were two different PullRequests - I think this is part of the reason why the latest PipelineResource redesign (#1673) suggests a read write (rw) mode for PipelineResources to cover this case. However this design is still in flux - long story short I think we have a couple options:

1. Detect when the same PipelineResource is used as an input and an output and copy data automatically (either at runtime, or assume that using the same name for both means they should be the same - maybe enforce at runtime?)
2. Manually do the copying for now (like in dan's example) until the redesign is done

Since this redesign is taking us a while, im totally fine with (1) personally - esp. since I find myself having to deal with the same issue :D

(+ @sbwsg )

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.