Populate the *Run.Status.Provenance.ConfigSource field

[chuangw6]

Changes

Before:
Prior, remote ResolutionRequest CRD supports recording the source information
in its Status that identifies where the remote resource came from.
Similarly, TaskRun/PipelineRun CRD supports receiving the source information
via a field in its status from remote ResolutionRequest. Specifically,
this field named ConfigSource is a subfield of the Provenance field in status.

Now:
In this PR, we are trying to pass the data from ResolutionRequest to pipeline
reconciler so that the data can be captured in TaskRun/PipelineRun's Status.Provenance.ConfigSource.
The implication of this change is that many functions' interface has been
changed because we are passing this extra source data alongside the remote resoure.

Note:

• The provenance field in Run.status is behind a feature flag named enable-provenance-in-status
  , which was introduced in Feature flag for provenance field in status #5670. The field will be populated iff
  the flag is set to true.
• If a pipeline yaml is from remote place A, and the individual tasks
  are from other remote places, pipelinerun status will only record the source
  for the pipeline, and the individual taskrun status will record the
  source for the corresponding task.

Related PRs/Issues:

• Add Provenance field in TaskRun&PipelineRun status #5580
• Add Source field in ResolutionRequest.status #5551
• Feature flag for provenance field in status #5670
• Set Source information in ResolutionRequest #5522

Signed-off-by: Chuang Wang [email protected]

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

• Has Docs included if any changes are user facing
• Has Tests included if any functionality added or changed
• Follows the commit message standard
• Meets the Tekton contributor standards (including
  functionality, content, code)
• Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
• Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
• Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

Populate the TaskRun/PipelineRun's Status.Provenance.ConfigSource field with the value from the remote ResolutionRequest Status. 

Note: the feature flag `enable-provenance-in-status` needs to be set to "true" to enable this provenance field to be populated & available in *Run.Status.

[tekton-robot]

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

[chuangw6]

/test all

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/remote/oci/resolver.go	72.3%	71.4%	-0.9

[chuangw6]

/test tekton-pipeline-unit-tests

[chuangw6]

/test all

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/taskrun/resources/taskref.go	85.2%	86.0%	0.8
pkg/remote/oci/resolver.go	72.3%	71.4%	-0.9

[chuangw6]

/retest

[abayer]

/retest

[abayer]

/assign

[abayer]

Oh, and this should be done for Pipelines, not just Tasks. I'd do all of that in one PR.

[chuangw6]

Oh, and this should be done for Pipelines, not just Tasks. I'd do all of that in one PR.

Yeah, that's what I planned to do in this PR. I just did for Tasks for now to gather some thoughts first. I've updated the description to reflect that. Thanks @abayer!

[chuangw6]

/test all

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/taskrun/resources/taskref.go	85.2%	86.0%	0.8
pkg/remote/oci/resolver.go	72.3%	71.4%	-0.9

[dibyom]

Good point! Agree. I've added the e2e test in my local but not pushed because the source value in all resolvers is still nil, which will lead us to compare if nil == nil. Happy to add this in another PR (or hold this pr) once the prs of setting value in resolver are merged. wdyt? @dibyom

Slight preference for merging the PRs setting the source value in resolver first and then merging this with the e2e test (since this pr enables the feature).

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	85.5%	85.6%	0.0
pkg/reconciler/pipelinerun/resources/pipelineref.go	92.2%	92.6%	0.4
pkg/reconciler/taskrun/resources/taskref.go	88.2%	88.7%	0.5
pkg/reconciler/taskrun/taskrun.go	81.4%	81.5%	0.1
pkg/remote/oci/resolver.go	72.3%	71.4%	-0.9

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	85.5%	85.6%	0.0
pkg/reconciler/pipelinerun/resources/pipelineref.go	92.2%	92.6%	0.4
pkg/reconciler/taskrun/resources/taskref.go	88.2%	88.7%	0.5
pkg/reconciler/taskrun/taskrun.go	81.4%	81.5%	0.1
pkg/remote/oci/resolver.go	72.3%	71.4%	-0.9
pkg/resolution/resolver/cluster/resolver.go	84.4%	84.8%	0.5

[chuangw6]

Slight preference for merging the PRs setting the source value in resolver first and then merging this with the e2e test (since this pr enables the feature).

That makes perfect sense to me! PRs for all resolvers were merged and e2e test is added. Please take a look! Thank you @dibyom .

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	85.5%	85.6%	0.0
pkg/reconciler/pipelinerun/resources/pipelineref.go	92.2%	92.6%	0.4
pkg/reconciler/taskrun/resources/taskref.go	88.2%	88.7%	0.5
pkg/reconciler/taskrun/taskrun.go	81.4%	81.5%	0.1
pkg/remote/oci/resolver.go	72.3%	71.4%	-0.9
pkg/resolution/resolver/cluster/resolver.go	84.4%	84.8%	0.5

[chuangw6]

/retest

[chuangw6]

/unhold

[dibyom]

Has Docs included if any changes are user facing

This seems to be already documented in https://github.com/tektoncd/pipeline/blob/main/docs/pipelineruns.md#the-status-field

[dibyom]

Thank you for all the work through the many iterations on this @chuangw6 🎉

/lgtm

[tekton-robot]

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
pkg/reconciler/pipelinerun/pipelinerun.go	85.5%	85.6%	0.0
pkg/reconciler/pipelinerun/resources/pipelineref.go	92.2%	92.6%	0.4
pkg/reconciler/taskrun/resources/taskref.go	88.2%	88.7%	0.5
pkg/reconciler/taskrun/taskrun.go	81.4%	81.5%	0.1
pkg/remote/oci/resolver.go	72.3%	71.4%	-0.9

[dibyom]

Coverage drop is oci/resolver is due to one line being split into two (thanks @chuangw6 ). So again:

/lgtm

[chuangw6]

Thank you so much everyone for the thorough review on this PR. @abayer @jagathprakash @Yongxuanzhang @alhuizenga @dibyom @wlynch!!

Started with the PoC which passed the data through unstructured annotations to the current approach which passes through the structured fields in status, this PR enables Chains to capture the important information in the provenance in a way that is extensible and easy to maintain. Special thanks to @wlynch @dibyom for the suggestion of introducing the structured fields into ResolutionRequest.Status and Run.Status. 👍