Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ISV-5221][ISV-5222] Add a new step into the “build-index-image” to build and push index sbom. #1513

Merged
merged 1 commit into from
Oct 31, 2024
Merged

[ISV-5221][ISV-5222] Add a new step into the “build-index-image” to build and push index sbom. #1513

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions pipelines/docker-build-multi-platform-oci-ta/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|IMAGE_DIGEST| Digest of the image just built| deprecated-base-image-check:0.4:IMAGE_DIGEST ; clair-scan:0.2:image-digest ; sast-snyk-check:0.2:image-digest ; clamav-scan:0.1:image-digest ; push-dockerfile:0.1:IMAGE_DIGEST ; rpms-signature-scan:0.2:image-digest|
|IMAGE_REF| Image reference of the built image containing both the repository and the digest| |
|IMAGE_URL| Image repository and tag where the built image was pushed| show-sbom:0.1:IMAGE_URL ; deprecated-base-image-check:0.4:IMAGE_URL ; clair-scan:0.2:image-url ; ecosystem-cert-preflight-checks:0.1:image-url ; sast-snyk-check:0.2:image-url ; clamav-scan:0.1:image-url ; apply-tags:0.1:IMAGE ; push-dockerfile:0.1:IMAGE ; rpms-signature-scan:0.2:image-url|
|SBOM_BLOB_URL| Reference of SBOM blob digest to enable digest-based verification from provenance| |
### buildah-remote-oci-ta:0.2 task results
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
Expand Down
1 change: 1 addition & 0 deletions pipelines/docker-build-oci-ta/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|IMAGE_DIGEST| Digest of the image just built| deprecated-base-image-check:0.4:IMAGE_DIGEST ; clair-scan:0.2:image-digest ; sast-snyk-check:0.2:image-digest ; clamav-scan:0.1:image-digest ; push-dockerfile:0.1:IMAGE_DIGEST ; rpms-signature-scan:0.2:image-digest|
|IMAGE_REF| Image reference of the built image containing both the repository and the digest| |
|IMAGE_URL| Image repository and tag where the built image was pushed| show-sbom:0.1:IMAGE_URL ; deprecated-base-image-check:0.4:IMAGE_URL ; clair-scan:0.2:image-url ; ecosystem-cert-preflight-checks:0.1:image-url ; sast-snyk-check:0.2:image-url ; clamav-scan:0.1:image-url ; apply-tags:0.1:IMAGE ; push-dockerfile:0.1:IMAGE ; rpms-signature-scan:0.2:image-url|
|SBOM_BLOB_URL| Reference of SBOM blob digest to enable digest-based verification from provenance| |
### buildah-oci-ta:0.2 task results
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
Expand Down
1 change: 1 addition & 0 deletions pipelines/docker-build/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|IMAGE_DIGEST| Digest of the image just built| deprecated-base-image-check:0.4:IMAGE_DIGEST ; clair-scan:0.2:image-digest ; sast-snyk-check:0.2:image-digest ; clamav-scan:0.1:image-digest ; push-dockerfile:0.1:IMAGE_DIGEST ; rpms-signature-scan:0.2:image-digest|
|IMAGE_REF| Image reference of the built image containing both the repository and the digest| |
|IMAGE_URL| Image repository and tag where the built image was pushed| show-sbom:0.1:IMAGE_URL ; deprecated-base-image-check:0.4:IMAGE_URL ; clair-scan:0.2:image-url ; ecosystem-cert-preflight-checks:0.1:image-url ; sast-snyk-check:0.2:image-url ; clamav-scan:0.1:image-url ; apply-tags:0.1:IMAGE ; push-dockerfile:0.1:IMAGE ; rpms-signature-scan:0.2:image-url|
|SBOM_BLOB_URL| Reference of SBOM blob digest to enable digest-based verification from provenance| |
### buildah:0.2 task results
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
Expand Down
1 change: 1 addition & 0 deletions pipelines/fbc-builder/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,7 @@ This pipeline is pushed as a Tekton bundle to [quay.io](https://quay.io/reposito
|IMAGE_DIGEST| Digest of the image just built| deprecated-base-image-check:0.4:IMAGE_DIGEST ; inspect-image:0.1:IMAGE_DIGEST ; fbc-validate:0.1:IMAGE_DIGEST|
|IMAGE_REF| Image reference of the built image containing both the repository and the digest| |
|IMAGE_URL| Image repository and tag where the built image was pushed| show-sbom:0.1:IMAGE_URL ; deprecated-base-image-check:0.4:IMAGE_URL ; apply-tags:0.1:IMAGE ; inspect-image:0.1:IMAGE_URL ; fbc-validate:0.1:IMAGE_URL|
|SBOM_BLOB_URL| Reference of SBOM blob digest to enable digest-based verification from provenance| |
### buildah:0.2 task results
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
Expand Down
1 change: 1 addition & 0 deletions pipelines/tekton-bundle-builder/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,7 @@
|IMAGE_DIGEST| Digest of the image just built| |
|IMAGE_REF| Image reference of the built image containing both the repository and the digest| |
|IMAGE_URL| Image repository and tag where the built image was pushed| apply-tags:0.1:IMAGE|
|SBOM_BLOB_URL| Reference of SBOM blob digest to enable digest-based verification from provenance| |
### git-clone:0.1 task results
|name|description|used in params (taskname:taskrefversion:taskparam)
|---|---|---|
Expand Down
1 change: 1 addition & 0 deletions task/build-image-index/0.1/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,5 @@ This takes existing Image Manifests and combines them in an Image Index.
|IMAGE_URL|Image repository and tag where the built image was pushed|
|IMAGES|List of all referenced image manifests|
|IMAGE_REF|Image reference of the built image containing both the repository and the digest|
|SBOM_BLOB_URL|Reference of SBOM blob digest to enable digest-based verification from provenance|

70 changes: 70 additions & 0 deletions task/build-image-index/0.1/build-image-index.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,13 @@ spec:
name: IMAGES
- description: Image reference of the built image containing both the repository and the digest
name: IMAGE_REF
- name: SBOM_BLOB_URL
description: Reference of SBOM blob digest to enable digest-based verification from provenance
type: string
volumes:
- name: shared-dir
emptyDir: {}

stepTemplate:
env:
- name: BUILDAH_FORMAT
Expand All @@ -61,6 +68,9 @@ spec:
value: $(params.ALWAYS_BUILD_INDEX)
- name: STORAGE_DRIVER
value: $(params.STORAGE_DRIVER)
volumeMounts:
- name: shared-dir
mountPath: /index-build-data
steps:
- image: quay.io/konflux-ci/buildah-task:latest@sha256:5cbd487022fb7ac476cbfdea25513b810f7e343ec48f89dc6a4e8c3c39fa37a2
# per https://kubernetes.io/docs/concepts/containers/images/#imagepullpolicy-defaulting
Expand Down Expand Up @@ -143,7 +153,67 @@ spec:
cat "image-digest"
} > "$(results.IMAGE_REF.path)"
echo -n "${image_manifests:1:-1}" > "$(results.IMAGES.path)"

IMAGE_DIGEST=$(cat image-digest)

INDEX_IMAGE_PULLSPEC="${IMAGE}@${IMAGE_DIGEST}"
buildah manifest inspect "$INDEX_IMAGE_PULLSPEC" > /index-build-data/manifest_data.json
securityContext:
capabilities:
add:
- SETFCAP

- image: quay.io/redhat-appstudio/sbom-utility-scripts-image@sha256:3b219e0610c06401bb5bd355a4bdfeb7f6700f2bef66f89316739d4aae96c89d
name: create-sbom
computeResources:
limits:
memory: 512Mi
cpu: 200m
requests:
memory: 256Mi
cpu: 100m
script: |
#!/bin/bash
set -e

MANIFEST_DATA_FILE="/index-build-data/manifest_data.json"
if [ ! -f "$MANIFEST_DATA_FILE" ]; then
echo "The manifest_data.json file does not exist. Skipping the SBOM creation..."
exit 0
fi

IMAGE_URL="$(cat "$(results.IMAGE_URL.path)")"
IMAGE_DIGEST="$(cat "$(results.IMAGE_DIGEST.path)")"
echo "Creating SBOM result file..."
python3 index_image_sbom_script.py \
--image-index-url "$IMAGE_URL" \
--image-index-digest "$IMAGE_DIGEST" \
--inspect-input-file "$MANIFEST_DATA_FILE" \
--output-path /index-build-data/sbom-results.json

- name: upload-sbom
image: quay.io/konflux-ci/appstudio-utils:ab6b0b8e40e440158e7288c73aff1cf83a2cc8a9@sha256:24179f0efd06c65d16868c2d7eb82573cce8e43533de6cea14fec3b7446e0b14
script: |
#!/bin/bash
set -e

SBOM_RESULT_FILE="/index-build-data/sbom-results.json"
if [ ! -f "$SBOM_RESULT_FILE" ]; then
echo "The sbom_results.json file does not exists. Skipping the SBOM upload..."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need something like touch "$(results.SBOM_BLOB_URL.path)" here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah good catch. But I don't think it's actually possible to reach this line, @haripate maybe we can remove the if [ ! -f "$MANIFEST_DATA_FILE" ] and if [ ! -f "$SBOM_RESULT_FILE" ] conditions?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chmeliik, could it reach here if ALWAYS_BUILD_INDEX is true and only one Image Manifest was provided?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup, good point. I forgot this task is allowed not to create an image index

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that without the touch "$(results.SBOM_BLOB_URL.path)", the SBOM_BLOB_URL result will be missing entirely, as opposed to empty if we add the touch. Considering no other tasks reference the SBOM_BLOB_URL, that seems fine at the moment.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, seem reasonable.

We may need to revisit this in the future. If this Task does not produce an Image Index, then the SBOM_BLOB_URL should probably point to the SBOM of the Image Manifest.

TBH, I don't really understand why this Task has the ALWAYS_BUILD_INDEX parameter. If you don't want to build an Image Index, then maybe just don't include the Task? We're probably making things more difficult to use and manage with such parameters.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the idea was that we would have just one build pipeline able to handle single-arch and multi-arch builds without any customization on the user's part. I don't fully remember, @arewm was that it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were at least two reasons why I added this:

  • Single arch pipelines should still be able to easily generate an Image Index if it becomes a requirement. This option can provide consistency with the internal build pipeline where every image generated is behind a manifest list. It also better supports operator-based installations where images generally shouldn't be referenced by image manifest.
  • Adding the image index generation to the template improved the kustomization for the multi-arch builds (i.e. the image index and image manifest generating tasks can be next to each other).

Now that we support a matrix of size 1, we could potentially change the pipeline default to the multi-arch pipeline as these will build single-arch by default. We can then change the single-arch pipeline to not have the image index and use that only for builds requiring an Image Manifest.

exit 0
fi

cosign attach sbom --sbom "$SBOM_RESULT_FILE" --type spdx "$(cat "$(results.IMAGE_REF.path)")"

# Remove tag from IMAGE while allowing registry to contain a port number.
sbom_repo="${IMAGE%:*}"
sbom_digest="$(sha256sum "$SBOM_RESULT_FILE" | cut -d' ' -f1)"
# The SBOM_BLOB_URL is created by `cosign attach sbom`.
echo -n "${sbom_repo}@sha256:${sbom_digest}" | tee "$(results.SBOM_BLOB_URL.path)"
computeResources:
limits:
memory: 512Mi
cpu: 200m
requests:
memory: 256Mi
cpu: 100m
Loading