Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sast-snyk-check: increased version to 0.3 #1359

Merged
merged 1 commit into from
Oct 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions task/sast-snyk-check-oci-ta/0.3/MIGRATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Migration from 0.2 to 0.3

Version 0.3:

- The `IMP_FINDINGS_ONLY` parameter has been introduced and enabled by default with "true" value. Only high or critical vulnerabilities will be shown. This behavior can be disabled with "false" value.
- The scan results uploaded in the SARIF format now additionally contain source code snippets and `csdiff/v1` fingerprints for each finding.
- There are no default arguments as "--all-projects --exclude=test*,vendor,deps" are ignored by Snyk Code
- SARIF produced by Snyk Code is not included in the CI log.
- The `KFP_GIT_URL` parameter has been introduced to indicate the repository to filter false positives. If this variable is left empty, the results won't be filtered. At the same time, we can store all excluded findings in a file using the `RECORD_EXCLUDED` parameter and specify a name of project with the `PROJECT_NAME` to use specific filters.

## Action from users

Renovate bot PR will be created with warning icon for a sast-snyk-check which is expected, no action from users are required.
30 changes: 30 additions & 0 deletions task/sast-snyk-check-oci-ta/0.3/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# sast-snyk-check-oci-ta task

Scans source code for security vulnerabilities, including common issues such as SQL injection, cross-site scripting (XSS), and code injection attacks using Snyk Code, a Static Application Security Testing (SAST) tool.

Follow the steps given [here](https://redhat-appstudio.github.io/docs.appstudio.io/Documentation/main/how-to-guides/testing_applications/enable_snyk_check_for_a_product/) to obtain a snyk-token and to enable the snyk task in a Pipeline.

The snyk binary used in this Task comes from a container image defined in https://github.com/konflux-ci/konflux-test

See https://snyk.io/product/snyk-code/ and https://snyk.io/ for more information about the snyk tool.

## Parameters
|name|description|default value|required|
|---|---|---|---|
|ARGS|Append arguments.|""|false|
|CACHI2_ARTIFACT|The Trusted Artifact URI pointing to the artifact with the prefetched dependencies.|""|false|
|IMP_FINDINGS_ONLY|Report only important findings. Default is true. To report all findings, specify "false"|true|false|
|KFP_GIT_URL|URL from repository to download known false positives files|""|false|
|PROJECT_NAME|Name of the scanned project, used to find path exclusions. By default, the Konflux component name will be used.|""|false|
|RECORD_EXCLUDED|Write excluded records in file. Useful for auditing (defaults to false).|false|false|
|SNYK_SECRET|Name of secret which contains Snyk token.|snyk-secret|false|
|SOURCE_ARTIFACT|The Trusted Artifact URI pointing to the artifact with the application source code.||true|
|caTrustConfigMapKey|The name of the key in the ConfigMap that contains the CA bundle data.|ca-bundle.crt|false|
|caTrustConfigMapName|The name of the ConfigMap to read CA bundle data from.|trusted-ca|false|
|image-url|Image URL.|""|false|

## Results
|name|description|
|---|---|
|TEST_OUTPUT|Tekton task test output.|

12 changes: 12 additions & 0 deletions task/sast-snyk-check-oci-ta/0.3/recipe.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
---
base: ../../sast-snyk-check/0.3/sast-snyk-check.yaml
add:
- use-source
- use-cachi2
preferStepTemplate: true
removeWorkspaces:
- workspace
replacements:
workspaces.workspace.path: /var/workdir
regexReplacements:
hacbs/\$\(context.task.name\): source
262 changes: 262 additions & 0 deletions task/sast-snyk-check-oci-ta/0.3/sast-snyk-check-oci-ta.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,262 @@
---
apiVersion: tekton.dev/v1
kind: Task
metadata:
name: sast-snyk-check-oci-ta
annotations:
tekton.dev/pipelines.minVersion: 0.12.1
tekton.dev/tags: konflux
labels:
app.kubernetes.io/version: "0.3"
spec:
description: |-
Scans source code for security vulnerabilities, including common issues such as SQL injection, cross-site scripting (XSS), and code injection attacks using Snyk Code, a Static Application Security Testing (SAST) tool.

Follow the steps given [here](https://redhat-appstudio.github.io/docs.appstudio.io/Documentation/main/how-to-guides/testing_applications/enable_snyk_check_for_a_product/) to obtain a snyk-token and to enable the snyk task in a Pipeline.

The snyk binary used in this Task comes from a container image defined in https://github.com/konflux-ci/konflux-test

See https://snyk.io/product/snyk-code/ and https://snyk.io/ for more information about the snyk tool.
params:
- name: ARGS
description: Append arguments.
type: string
default: ""
- name: CACHI2_ARTIFACT
description: The Trusted Artifact URI pointing to the artifact with
the prefetched dependencies.
type: string
default: ""
- name: IMP_FINDINGS_ONLY
description: Report only important findings. Default is true. To report
all findings, specify "false"
type: string
default: "true"
- name: KFP_GIT_URL
description: URL from repository to download known false positives files
type: string
default: ""
- name: PROJECT_NAME
description: Name of the scanned project, used to find path exclusions.
By default, the Konflux component name will be used.
type: string
default: ""
- name: RECORD_EXCLUDED
description: Write excluded records in file. Useful for auditing (defaults
to false).
type: string
default: "false"
- name: SNYK_SECRET
description: Name of secret which contains Snyk token.
default: snyk-secret
- name: SOURCE_ARTIFACT
description: The Trusted Artifact URI pointing to the artifact with
the application source code.
type: string
- name: caTrustConfigMapKey
description: The name of the key in the ConfigMap that contains the
CA bundle data.
type: string
default: ca-bundle.crt
- name: caTrustConfigMapName
description: The name of the ConfigMap to read CA bundle data from.
type: string
default: trusted-ca
- name: image-url
description: Image URL.
type: string
default: ""
results:
- name: TEST_OUTPUT
description: Tekton task test output.
volumes:
- name: snyk-secret
secret:
optional: true
secretName: $(params.SNYK_SECRET)
- name: trusted-ca
configMap:
items:
- key: $(params.caTrustConfigMapKey)
path: ca-bundle.crt
name: $(params.caTrustConfigMapName)
optional: true
- name: workdir
emptyDir: {}
stepTemplate:
volumeMounts:
- mountPath: /var/workdir
name: workdir
steps:
- name: use-trusted-artifact
image: quay.io/redhat-appstudio/build-trusted-artifacts:latest@sha256:a83c92fc0a1e2c785937c6612dc8c8237818535543f00ecaf6b3b77a35f76259
args:
- use
- $(params.SOURCE_ARTIFACT)=/var/workdir/source
- $(params.CACHI2_ARTIFACT)=/var/workdir/cachi2
- name: sast-snyk-check
image: quay.io/redhat-appstudio/konflux-test:v1.4.7@sha256:cf6808a3bd605630a5d9f20595ff7c43f8645c00381219d32f5a11e88fe37072
workingDir: /var/workdir/source
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs to be checked. We are storing results in the same folder as the code, which doesn't seem like a good solution.
If we change this, that means that not all oci-ta tasks are similar

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be changed here. You can try removing that line.
Might be worth not using the hacbs term any more in the new version of the Task.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general, when using TA one can do whatever they want to the source code and the changes will not affect other Tasks, that's the point of using TA.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zregvart So according to the recipe file, the var/workdir/source should be replaced by $(workspaces.workspace.path)/hacbs/$(context.task.name), right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other way around hacbs/$(context.task.name) will be replaced with source.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't that contradicting what CI pipeline is suggesting: https://github.com/konflux-ci/build-definitions/actions/runs/11386335463/job/31682867224?pr=1359#step:3:107 ?
It is suggesting replacing source/ for /hacbs/"$(context.task.name)"/

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The line in the non-TA version of the Task is:

(cd "$SOURCE_CODE_DIR" && csgrep --mode=json --embed-context=3 "$(workspaces.workspace.path)"/hacbs/"$(context.task.name)"/sast_snyk_check_out.json) \

It does not match the regex hacbs/\$\(context.task.name\), there is an extra " after hacbs/. This means that no replacement is made and the line is taken as is from the non-TA version. This is what you see in the diff. You should not edit the generated files (sast-snyk-check-oci-ta.yaml) by hand, run hack/generate-ta-tasks.sh locally and tweak the recipe.yaml until you reach the desired state of sast-snyk-check-oci-ta.yaml.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@zregvart That's a very useful information we didn't have! Thank you very much for that!

volumeMounts:
- mountPath: /etc/secrets
name: snyk-secret
readOnly: true
- mountPath: /mnt/trusted-ca
name: trusted-ca
readOnly: true
env:
- name: SNYK_SECRET
value: $(params.SNYK_SECRET)
- name: ARGS
value: $(params.ARGS)
- name: IMP_FINDINGS_ONLY
value: $(params.IMP_FINDINGS_ONLY)
- name: KFP_GIT_URL
value: $(params.KFP_GIT_URL)
- name: PROJECT_NAME
value: $(params.PROJECT_NAME)
- name: RECORD_EXCLUDED
value: $(params.RECORD_EXCLUDED)
- name: COMPONENT_LABEL
valueFrom:
fieldRef:
fieldPath: metadata.labels['appstudio.openshift.io/component']
script: |
#!/usr/bin/env bash

set -euo pipefail
# shellcheck source=/dev/null
. /utils.sh
trap 'handle_error $(results.TEST_OUTPUT.path)' EXIT

if [[ -z "${PROJECT_NAME}" ]]; then
PROJECT_NAME=${COMPONENT_LABEL}
fi

echo "The PROJECT_NAME used is: ${PROJECT_NAME}"

# Installation of Red Hat certificates for cloning Red Hat internal repositories
ca_bundle=/mnt/trusted-ca/ca-bundle.crt
if [ -f "$ca_bundle" ]; then
echo "INFO: Using mounted CA bundle: $ca_bundle"
cp -vf $ca_bundle /etc/pki/ca-trust/source/anchors
update-ca-trust
fi

SNYK_TOKEN_PATH="/etc/secrets/snyk_token"
if [ -f "${SNYK_TOKEN_PATH}" ] && [ -s "${SNYK_TOKEN_PATH}" ]; then
# SNYK token is provided
SNYK_TOKEN="$(cat ${SNYK_TOKEN_PATH})"
export SNYK_TOKEN
else
# According to shellcheck documentation, the following error can be ignored as it is ignored through indirection: https://www.shellcheck.net/wiki/SC2034
# shellcheck disable=SC2034
to_enable_snyk='[here](https://redhat-appstudio.github.io/docs.appstudio.io/Documentation/main/how-to-guides/testing_applications/enable_snyk_check_for_a_product/)'
note="Task $(context.task.name) skipped: If you wish to use the Snyk code SAST task, please create a secret name snyk-secret with the key 'snyk_token' containing the Snyk token by following the steps given ${to_enable_snyk}"
TEST_OUTPUT=$(make_result_json -r SKIPPED -t "$note")
echo "${TEST_OUTPUT}" | tee "$(results.TEST_OUTPUT.path)"
exit 0
fi

SNYK_EXIT_CODE=0
SOURCE_CODE_DIR=/var/workdir
SEVERITY_THRESHOLD="high"
if [ "${IMP_FINDINGS_ONLY}" == "false" ]; then
SEVERITY_THRESHOLD="low"
fi
set +e
# We do want to expand ARGS (it can be multiple CLI flags, not just one)
# shellcheck disable=SC2086
snyk code test $ARGS --severity-threshold="$SEVERITY_THRESHOLD" "$SOURCE_CODE_DIR" --max-depth=1 --sarif-file-output="${SOURCE_CODE_DIR}"/sast_snyk_check_out.json 1>&2 >>stdout.txt
SNYK_EXIT_CODE=$?
set -e
test_not_skipped=0
SKIP_MSG="We found 0 supported files"
grep -q "$SKIP_MSG" stdout.txt || test_not_skipped=$?

# In order to generate csdiff/v1, we need to add the whole path of the source code as Snyk only provides an URI to embed the context
(cd "${SOURCE_CODE_DIR}" && csgrep --mode=json --embed-context=3 "${SOURCE_CODE_DIR}"/sast_snyk_check_out.json) |
csgrep --mode=json --strip-path-prefix="source/" \
>sast_snyk_check_out_all_findings.json
Copy link
Contributor Author

@jperezdealgaba jperezdealgaba Oct 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kdudka FYI: The missing space in this line is being introduced by the *-ta CI pipeline. It seems that some cases are not covered by the script.


echo "Results:"
(set -x && csgrep --mode=evtstat sast_snyk_check_out_all_findings.json)

# We check if the KFP_GIT_URL variable is set to apply the filters or not
if [[ -z "${KFP_GIT_URL}" ]]; then
echo "KFP_GIT_URL variable not defined. False positives won't be filtered"
mv sast_snyk_check_out_all_findings.json filtered_sast_snyk_check_out.json
else
echo "Filtering false positives in results files using csfilter-kfp..."

CMD=(
csfilter-kfp
--verbose
--kfp-git-url="${KFP_GIT_URL}"
--project-nvr="${PROJECT_NAME}"
)

if [ "${RECORD_EXCLUDED}" == "true" ]; then
CMD+=(--record-excluded="excluded-findings.json")
fi

set +e
"${CMD[@]}" sast_snyk_check_out_all_findings.json >filtered_sast_snyk_check_out.json
status=$?
set -e
if [ "$status" -ne 0 ]; then
echo "Error: failed to filter known false positives" >&2
return 1
else
echo "Message: Succeed to filter known false positives" >&2
fi
echo "Results after filtering:"
(set -x && csgrep --mode=evtstat filtered_sast_snyk_check_out.json)
fi

csgrep --mode=sarif filtered_sast_snyk_check_out.json >sast_snyk_check_out.sarif
if [[ "$SNYK_EXIT_CODE" -eq 0 ]] || [[ "$SNYK_EXIT_CODE" -eq 1 ]]; then
TEST_OUTPUT=
parse_test_output "$(context.task.name)" sarif sast_snyk_check_out.sarif || true

# When the test is skipped, the "SNYK_EXIT_CODE" is 3 and it can also be 3 in some other situation
elif [[ "$test_not_skipped" -eq 0 ]]; then
note="Task $(context.task.name) success: Snyk code test found zero supported files."
ERROR_OUTPUT=$(make_result_json -r SUCCESS -t "$note")
else
echo "sast-snyk-check test failed because of the following issues:"
cat stdout.txt
note="Task $(context.task.name) failed: For details, check Tekton task log."
ERROR_OUTPUT=$(make_result_json -r ERROR -t "$note")
fi
echo "${TEST_OUTPUT:-${ERROR_OUTPUT}}" | tee "$(results.TEST_OUTPUT.path)"
- name: upload
image: quay.io/konflux-ci/oras:latest@sha256:f4b891ee3038a5f13cd92ff4f473faad5601c2434d1c6b9bccdfc134d9d5f820
workingDir: /var/workdir/source
env:
- name: IMAGE_URL
value: $(params.image-url)
script: |
#!/usr/bin/env bash

if [ -z "${IMAGE_URL}" ]; then
echo 'No image-url provided. Skipping upload.'
exit 0
fi

UPLOAD_FILES="sast_snyk_check_out.sarif excluded-findings.json"
for UPLOAD_FILE in ${UPLOAD_FILES}; do
if [ ! -f "${UPLOAD_FILE}" ]; then
echo "No ${UPLOAD_FILE} exists. Skipping upload."
continue
fi
if [ "${UPLOAD_FILES}" == "excluded-findings.json" ]; then
MEDIA_TYPE=application/json
else
MEDIA_TYPE=application/sarif+json
fi
echo "Selecting auth"
select-oci-auth "${IMAGE_URL}" >"${HOME}/auth.json"
echo "Attaching to ${IMAGE_URL}"
oras attach --no-tty --registry-config "$HOME/auth.json" --artifact-type "${MEDIA_TYPE}" "${IMAGE_URL}" "${UPLOAD_FILE}:${MEDIA_TYPE}"
done
13 changes: 13 additions & 0 deletions task/sast-snyk-check/0.3/MIGRATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Migration from 0.2 to 0.3

Version 0.3:

- The `IMP_FINDINGS_ONLY` parameter has been introduced and enabled by default with "true" value. Only high or critical vulnerabilities will be shown. This behavior can be disabled with "false" value.
jperezdealgaba marked this conversation as resolved.
Show resolved Hide resolved
- The scan results uploaded in the SARIF format now additionally contain source code snippets and `csdiff/v1` fingerprints for each finding.
- There are no default arguments as "--all-projects --exclude=test*,vendor,deps" are ignored by Snyk Code
- SARIF produced by Snyk Code is not included in the CI log.
- The `KFP_GIT_URL` parameter has been introduced to indicate the repository to filter false positives. If this variable is left empty, the results won't be filtered. At the same time, we can store all excluded findings in a file using the `RECORD_EXCLUDED` parameter and specify a name of project with the `PROJECT_NAME` to use specific filters.

## Action from users

Renovate bot PR will be created with warning icon for a sast-snyk-check which is expected, no action from users are required.
39 changes: 39 additions & 0 deletions task/sast-snyk-check/0.3/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# sast-snyk-check task

## Description:

The sast-snyk-check task uses Snyk Code tool to perform Static Application Security Testing (SAST) for Snyk, a popular cloud-native application security platform.

Snyk's SAST tool uses a combination of static analysis and machine learning techniques to scan an application's source code for potential security vulnerabilities, including common issues such as SQL injection, cross-site scripting (XSS), and code injection attacks.

> NOTE: This task is executed only if the user provides a Snyk token stored in a secret in their namespace. The name of the secret then needs to be supplied in the `snyk-secret` pipeline parameter.

## Params:

| name | description | default value | required |
|-----------------------|--------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------|----------|
| SNYK_SECRET | Name of secret which contains Snyk token. | snyk-secret | true |
| ARGS | Append arguments. | "" | false |
| IMP_FINDINGS_ONLY | Report only important findings. To report all findings, specify "false" | true | true |
| KFP_GIT_URL | Link to the known-false-positives repository. If left blank, results won't be filtered | "" | false |
| PROJECT_NAME | Name of the scanned project, used to find path exclusions. By default, the Konflux component name will be used. | ${metadata.labels['appstudio.openshift.io/component']} | false |
| RECORD_EXCLUDED | Write excluded records in file. Useful for auditing. | false | false |

## How to obtain a snyk-token and enable snyk task on the pipeline:

Follow the steps given [here](https://redhat-appstudio.github.io/docs.appstudio.io/Documentation/main/how-to-guides/testing_applications/enable_snyk_check_for_a_product/)

## Results:

| name | description |
|---------------|----------------------------|
| TEST_OUTPUT | Tekton task test output. |

## Source repository for image:

https://github.com/konflux-ci/konflux-test

## Additional links:

* https://snyk.io/product/snyk-code/
* https://snyk.io/
Loading
Loading