Add `Source` field in `ResolutionRequest.status` #5551

chuangw6 · 2022-09-23T19:07:56Z

/kind feature

Changes

resolve #5529

Before:
The ResolutionRequestStatusFields only has the Data field that is a string
representation of the resolved content.

Now:
A new field called Source is now introduced to the ResolutionRequestStatusFields
to record the source information of the remote data in a structured way
using the standard SLSA ConfigSource struct.

Why?
Recently there is a clear requirement that the remote source
information of the remote data should be recorded in the provenance to
link back to its origin including the resolved the commit sha when users
only provide the branch/tag name for the resolver. Without this PR, the
only way to achieve this is to pass the resolved source information through
annotations, which has a couple of the drawbacks i.e. unstructured data,
hard to maintain and to make changes in future etc. That's where this PR
comes in to solve the problem.

Once this PR is merged, #5397 should be able to pass data through this newly added field rather than annotations.

Signed-off-by: Chuang Wang [email protected]

Submitter Checklist

As the author of this PR, please check off the items in this checklist:

Has Docs included if any changes are user facing
Has Tests included if any functionality added or changed
Follows the commit message standard
Meets the Tekton contributor standards (including
functionality, content, code)
Has a kind label. You can add one by adding a comment on this PR that contains /kind <type>. Valid types are bug, cleanup, design, documentation, feature, flake, misc, question, tep
Release notes block below has been updated with any user facing changes (API changes, bug fixes, changes requiring upgrade notices or deprecation warnings)
Release notes contains the string "action required" if the change requires additional action from users switching to the new release

Release Notes

Add provenance-related field in ResolutionRequest.status.

tekton-robot · 2022-09-23T19:07:57Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

abayer · 2022-09-23T20:16:53Z

/test check-pr-has-kind-label

tekton-robot · 2022-09-23T20:16:54Z

@abayer: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

/test pull-tekton-pipeline-alpha-integration-tests
/test pull-tekton-pipeline-build-tests
/test pull-tekton-pipeline-integration-tests
/test tekton-pipeline-unit-tests

The following commands are available to trigger optional jobs:

/test pull-tekton-pipeline-go-coverage

Use /test all to run all jobs.

In response to this:

/test check-pr-has-kind-label

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

abayer · 2022-09-23T20:17:00Z

/test all

chuangw6 · 2022-09-27T21:17:05Z

Hey @abayer @wlynch @dibyom ,

This PR is ready for review. Please take a look when you get a chance. Thank you very much!!

Looking forward to your comments & feedback.

tekton-robot · 2022-09-27T21:20:01Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
docs/resolver-template/cmd/demoresolver/main.go	72.7%	75.0%	2.3
pkg/resolution/resolver/cluster/resolver.go	84.7%	84.9%	0.2
pkg/resolution/resolver/framework/fakeresolver.go	90.0%	90.3%	0.3
pkg/resolution/resolver/git/resolver.go	83.9%	84.0%	0.1
pkg/resolution/resolver/hub/resolver.go	70.6%	69.2%	-1.4

chuangw6 · 2022-09-27T21:24:26Z

/retest

chuangw6 · 2022-09-27T21:29:45Z

/retest

chuangw6 · 2022-09-28T14:49:56Z

docs/resolver-template/cmd/demoresolver/main.go

+
+// Source returns the source references of the data. None atm.
+func (*myResolvedResource) Source() *v1alpha1.ConfigSource {
+	return nil
+}


Hey @abayer

RE: "So there's the additional problem that we can't guarantee structured resolution source information - that's going to depend on the resolver implementation itself. We can control that for git, bundles, cluster, and hub, but not for any third-party resolvers that are written." from #5529 (comment)

I feel like the individual resolver implementation just needs to implement the Source() method to return a configsource, which is part of framework.ResolvedResource interface. This resolver template is an example. That said, I think we just need to update this section in how-to-write-a-resolver to mention this Source() method other than Data() and Annotations().

Thoughts on this?
Thank you!

Updated how-to-write-a-resolver doc in this PR. Please take a look.

tekton-robot · 2022-09-28T14:50:16Z

@chuangw6: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

/test pull-tekton-pipeline-alpha-integration-tests
/test pull-tekton-pipeline-build-tests
/test pull-tekton-pipeline-integration-tests
/test tekton-pipeline-unit-tests

The following commands are available to trigger optional jobs:

/test pull-tekton-pipeline-go-coverage

Use /test all to run all jobs.

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

chuangw6 · 2022-09-28T14:53:03Z

/retest

tekton-robot · 2022-09-28T15:00:40Z

The following is the coverage report on the affected files.
Say /test pull-tekton-pipeline-go-coverage to re-run this coverage report

File	Old Coverage	New Coverage	Delta
docs/resolver-template/cmd/demoresolver/main.go	72.7%	75.0%	2.3
pkg/resolution/resolver/cluster/resolver.go	84.7%	84.9%	0.2
pkg/resolution/resolver/framework/fakeresolver.go	90.0%	90.3%	0.3
pkg/resolution/resolver/git/resolver.go	83.9%	84.0%	0.1
pkg/resolution/resolver/hub/resolver.go	70.6%	69.2%	-1.4

Change 1: Add a Provenance field in TaskRun&PipelineRun status that currently only contains configsource data, but can be extended later to have more provenance-related fields. Change 2: Prior, tektoncd#5551 introduced the ConfigSource to api/resolution alpha & beta package. In this PR, we moved the ConfigSource to api/pipeline alpha & beta package for the provenance field to reuse that type (cannot import the api/resolution alpha because of import cycle). Why: See the motivation and discussions in tektoncd#5550. The tldr is that it helps pass provenance-related data in a more structured way ConfigSource is one example. Signed-off-by: Chuang Wang <[email protected]>

Change 1: Add a Provenance field in TaskRun&PipelineRun status. This field currently only contains a subfield named `ConfigSource`, but can be extended later to have more provenance-related fields. Change 2: Prior, tektoncd#5551 introduced the ConfigSource to api/resolution alpha & beta package. In this PR, we moved the ConfigSource to api/pipeline alpha & beta package for the provenance field to reuse that type (cannot import the api/resolution alpha because of import cycle). Why: See the motivation and discussions in tektoncd#5550. The tldr is that it helps pass provenance-related data in a more structured way ConfigSource is one example. Signed-off-by: Chuang Wang <[email protected]>

Change 1: Add a Provenance field in TaskRun&PipelineRun status. This field currently only contains a subfield named `ConfigSource`, but can be extended later to have more provenance-related fields. Change 2: Prior, #5551 introduced the ConfigSource to api/resolution alpha & beta package. In this PR, we moved the ConfigSource to api/pipeline alpha & beta package for the provenance field to reuse that type (cannot import the api/resolution alpha because of import cycle). Why: See the motivation and discussions in #5550. The tldr is that it helps pass provenance-related data in a more structured way ConfigSource is one example. Signed-off-by: Chuang Wang <[email protected]>

Change 1: Add a Provenance field in TaskRun&PipelineRun status. This field currently only contains a subfield named `ConfigSource`, but can be extended later to have more provenance-related fields. Change 2: Prior, tektoncd#5551 introduced the ConfigSource to api/resolution alpha & beta package. In this PR, we moved the ConfigSource to api/pipeline alpha & beta package for the provenance field to reuse that type (cannot import the api/resolution alpha because of import cycle). Why: See the motivation and discussions in tektoncd#5550. The tldr is that it helps pass provenance-related data in a more structured way ConfigSource is one example. Signed-off-by: Chuang Wang <[email protected]>

Prior, remote ResolutionRequest CRD supports **recording** the source information in its Status that identifies where the remote resource came from. Similarly, TaskRun/PipelineRun CRD supports **receiving** the source information via a field in its status from remote ResolutionRequest. Specifically, this field named `ConfigSource` is a subfield of the `Provenance` field in status. In this PR, we are trying to pass the data from ResolutionRequest to pipeline reconciler so that the data can be captured in PipelineRun.Status.Provenance.ConfigSource. Same for taskrun status. The implication of this change is that many functions' interface has been changed because we are passing this extra source data alongside the remote resoure. Note: - The `provenance` field in Run.status is behind a feature flag named `enable-provenance-in-status` , which was introduced in tektoncd#5670. The field will be populated iff the flag is set to `true`. - If a pipeline yaml is from remote place A, and the individual tasks are from other remote places, pipelinerun status will only record the source for the pipeline, and the individual taskrun status will record the source for the individual task. Related PRs: - tektoncd#5580 - tektoncd#5551 - tektoncd#5670 Signed-off-by: Chuang Wang <[email protected]>

Prior, remote `ResolutionRequest` CRD supports **recording** the source information in its Status that identifies where the remote resource came from. Similarly, `TaskRun/PipelineRun` CRD supports **receiving** the source information via a field in its status from remote ResolutionRequest. Specifically, this field named `ConfigSource` is a subfield of the `Provenance` field in status. In this PR, we are trying to pass the data from `ResolutionRequest` to pipeline reconciler so that the data can be captured in TaskRun/PipelineRun's `Status.Provenance.ConfigSource`. The implication of this change is that many functions' interface has been changed because we are passing this extra source data alongside the remote resoure. Note: - The `provenance` field in Run.status is behind a feature flag named `enable-provenance-in-status` , which was introduced in tektoncd#5670. The field will be populated iff the flag is set to `true`. - If a pipeline yaml is from remote place A, and the individual tasks are from other remote places, pipelinerun status will only record the source for the pipeline, and the individual taskrun status will record the source for the corresponding task. Related PRs: - tektoncd#5580 - tektoncd#5551 - tektoncd#5670 Signed-off-by: Chuang Wang <[email protected]>

Prior, remote `ResolutionRequest` CRD supports **recording** the source information in its Status that identifies where the remote resource came from. Similarly, `TaskRun/PipelineRun` CRD supports **receiving** the source information via a field in its status from remote ResolutionRequest. Specifically, this field named `ConfigSource` is a subfield of the `Provenance` field in status. In this PR, we are trying to pass the data from `ResolutionRequest` to pipeline reconciler so that the data can be captured in TaskRun/PipelineRun's `Status.Provenance.ConfigSource`. The implication of this change is that many functions' interface has been changed because we are passing this extra source data alongside the remote resoure. Note: - The `provenance` field in Run.status is behind a feature flag named `enable-provenance-in-status` , which was introduced in tektoncd#5670. The field will be populated iff the flag is set to `true`. - If a pipeline yaml is from remote place A, and the individual tasks are from other remote places, pipelinerun status will only record the source for the pipeline, and the individual taskrun status will record the source for the corresponding task. Related PRs/Issues: - tektoncd#5580 - tektoncd#5551 - tektoncd#5670 - tektoncd#5522 Signed-off-by: Chuang Wang <[email protected]>

Prior, remote `ResolutionRequest` CRD supports **recording** the source information in its Status that identifies where the remote resource came from. Similarly, `TaskRun/PipelineRun` CRD supports **receiving** the source information via a field in its status from remote ResolutionRequest. Specifically, this field named `ConfigSource` is a subfield of the `Provenance` field in status. In this PR, we are trying to pass the data from `ResolutionRequest` to pipeline reconciler so that the data can be captured in TaskRun/PipelineRun's `Status.Provenance.ConfigSource`. The implication of this change is that many functions' interface has been changed because we are passing this extra source data alongside the remote resoure. Note: - The `provenance` field in Run.status is behind a feature flag named `enable-provenance-in-status` , which was introduced in #5670. The field will be populated iff the flag is set to `true`. - If a pipeline yaml is from remote place A, and the individual tasks are from other remote places, pipelinerun status will only record the source for the pipeline, and the individual taskrun status will record the source for the corresponding task. Related PRs/Issues: - #5580 - #5551 - #5670 - #5522 Signed-off-by: Chuang Wang <[email protected]>

tekton-robot requested review from dibyom and vdemeester September 23, 2022 19:08

chuangw6 changed the title ~~Add provenance-related field in ResolutionRequest.status~~ Add provenance-related field in ResolutionRequest.status Sep 23, 2022

chuangw6 force-pushed the add-provenance-data-into-resolutionrequest branch 4 times, most recently from 7d7b903 to 4c66e94 Compare September 27, 2022 21:11

chuangw6 marked this pull request as ready for review September 27, 2022 21:15

tekton-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Sep 27, 2022

tekton-robot requested review from afrittoli and imjasonh September 27, 2022 21:15

chuangw6 changed the title ~~Add provenance-related field in ResolutionRequest.status~~ Add ConfigSource field in ResolutionRequest.status Sep 27, 2022

chuangw6 mentioned this pull request Sep 27, 2022

Populate the *Run.Status.Provenance.ConfigSource field #5397

Merged

7 tasks

chuangw6 commented Sep 28, 2022

View reviewed changes

chuangw6 force-pushed the add-provenance-data-into-resolutionrequest branch from 4c66e94 to 8dabb8c Compare September 28, 2022 14:55

tekton-robot merged commit b15c5de into tektoncd:main Sep 30, 2022

abayer mentioned this pull request Sep 30, 2022

Move remote resolution out of alpha #5515

Merged

7 tasks

chuangw6 mentioned this pull request Oct 3, 2022

Add Provenance field in TaskRun&PipelineRun status #5580

Merged

7 tasks

chuangw6 changed the title ~~Add ConfigSource field in ResolutionRequest.status~~ Add Source field in ResolutionRequest.status Nov 16, 2022

chuangw6 mentioned this pull request Nov 16, 2022

Chains needs to be able to "Identify entry point" - SLSA L3 requirement. tektoncd/chains#521

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add `Source` field in `ResolutionRequest.status` #5551

Add `Source` field in `ResolutionRequest.status` #5551

chuangw6 commented Sep 23, 2022 •

edited

Loading

tekton-robot commented Sep 23, 2022

abayer commented Sep 23, 2022

tekton-robot commented Sep 23, 2022

abayer commented Sep 23, 2022

chuangw6 commented Sep 27, 2022

tekton-robot commented Sep 27, 2022

chuangw6 commented Sep 27, 2022

chuangw6 commented Sep 27, 2022

chuangw6 Sep 28, 2022

chuangw6 Sep 28, 2022

tekton-robot commented Sep 28, 2022

chuangw6 commented Sep 28, 2022

tekton-robot commented Sep 28, 2022

Add Source field in ResolutionRequest.status #5551

Add Source field in ResolutionRequest.status #5551

Conversation

chuangw6 commented Sep 23, 2022 • edited Loading

Changes

Submitter Checklist

Release Notes

tekton-robot commented Sep 23, 2022

abayer commented Sep 23, 2022

tekton-robot commented Sep 23, 2022

abayer commented Sep 23, 2022

chuangw6 commented Sep 27, 2022

tekton-robot commented Sep 27, 2022

chuangw6 commented Sep 27, 2022

chuangw6 commented Sep 27, 2022

chuangw6 Sep 28, 2022

Choose a reason for hiding this comment

chuangw6 Sep 28, 2022

Choose a reason for hiding this comment

tekton-robot commented Sep 28, 2022

chuangw6 commented Sep 28, 2022

tekton-robot commented Sep 28, 2022

Add `Source` field in `ResolutionRequest.status` #5551

Add `Source` field in `ResolutionRequest.status` #5551

chuangw6 commented Sep 23, 2022 •

edited

Loading