using WhenExpressions, skipped task runs

[r0bj]

Expected Behavior

According to documentation https://tekton.dev/docs/pipelines/pipelines/#guard-task-execution-using-whenexpressions
"If any of the WhenExpressions evaluate to False, the Task is not run and the Task is listed in the Skipped Tasks section of the PipelineRunStatus."

Actual Behavior

WhenExpressions evaluate to False but Task runs.

Steps to Reproduce the Problem

Example from https://github.com/tektoncd/pipeline/blob/master/examples/v1beta1/pipelineruns/pipelinerun-with-when-expressions.yaml with one modification in line 63 ("exists" -> "some-value"):

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: guarded-pipeline
  namespace: tekton-execution
spec:
  params:
    - name: path
      type: string
      description: The path of the file to be created
  workspaces:
    - name: source
      description: |
        This workspace will receive the cloned git repo and be passed
        to the next Task to create a file
  tasks:
    - name: create-file
      when:
        - input: "$(params.path)"
          operator: in
          values: ["README.md"]
      workspaces:
        - name: source
          workspace: source
      taskSpec:
        workspaces:
          - name: source
            description: The workspace to create the readme file in
        steps:
          - name: write-new-stuff
            image: ubuntu
            script: 'touch $(workspaces.source.path)/README.md'
    - name: check-file
      params:
        - name: path
          value: "$(params.path)"
      workspaces:
        - name: source
          workspace: source
      runAfter:
        - create-file
      taskSpec:
        params:
          - name: path
        workspaces:
          - name: source
            description: The workspace to check for the file
        results:
          - name: status
            description: indicates whether the file exists or is missing
        steps:
          - name: check-file
            image: alpine
            script: |
              if test -f $(workspaces.source.path)/$(params.path); then
                printf exists | tee /tekton/results/status
              else
                printf missing | tee /tekton/results/status
              fi
    - name: echo-file-exists
      when:
        - input: "$(tasks.check-file.results.status)"
          operator: in
          values: ["some-value"]
      taskSpec:
        steps:
          - name: echo
            image: ubuntu
            script: 'echo file exists'
    - name: task-should-be-skipped
      when:
        - input: "$(params.path)"
          operator: notin
          values: ["README.md"]
      taskSpec:
        steps:
          - name: echo
            image: ubuntu
            script: exit 1
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: guarded-pr
  namespace: tekton-execution
spec:
  serviceAccountName: 'default'
  pipelineRef:
    name: guarded-pipeline
  params:
    - name: path
      value: README.md
  workspaces:
    - name: source
      volumeClaimTemplate:
        spec:
          accessModes:
          - ReadWriteOnce
          resources:
            requests:
              storage: 1Gi

2. Execute PipelineRun

guarded-pr-check-file-97n5x-pod-dl9bg         0/1     Completed   0          107s
guarded-pr-create-file-7wvkp-pod-67vbf        0/1     Completed   0          2m1s
guarded-pr-echo-file-exists-b9bk8-pod-vx4b6   0/1     Completed   0          100s

Note: Task echo-file-exists should be skipped due to WhenExpressions but was executed.

3. At the same time Task is listed in the Skipped Tasks section of the PipelineRunStatus:

  skippedTasks:
  - name: echo-file-exists
  - name: task-should-be-skipped

Additional Info

• Kubernetes version:

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-08-14T11:09:22Z", GoVersion:"go1.14.7", Compiler:"gc", Platform:"darwin/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.4", GitCommit:"c96aede7b5205121079932896c4ad89bb93260af", GitTreeState:"clean", BuildDate:"2020-06-17T11:33:59Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}

• Tekton Pipeline version:

Client version: 0.12.0
Pipeline version: v0.16.0
Triggers version: v0.8.0

[vdemeester]

@r0bj quick questions, what are the logs for pod guarded-pr-echo-file-exists-b9bk8-pod-vx4b6 ?

[r0bj]

@vdemeester Logs are:

+ echo file exists
file exists

My observation is that this issue is related to tasks that use other tasks as an input to WhenExpressions. Tasks which use params as an input seems work ok - they skipped and do not run, as expected.

[bobcatfish]

Quick update: @jerop is working on these and we'll make a 0.16.1 release (https://github.com/tektoncd/pipeline/milestone/34) once these are fixed. Thanks for the quick report @r0bj !

[jerop]

@r0bj thank you for reporting this issue!

we needed to do an additional check for skipping vs executing before creating tasks after result variable replacement --
#3197

it'll be fixed in 0.16.1 release

[bobcatfish]

Quick update: since there are a few issues, we're aiming to have fixes for all 3 in 0.16.2 on Monday

[bobcatfish]

Okay 0.16.2 is up now (https://github.com/tektoncd/pipeline/releases/tag/v0.16.2) please let us know if you still run into problems!