Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When onError is continue, but taskrun failed #6664

Closed
kycheng opened this issue May 16, 2023 · 3 comments · Fixed by #6675
Closed

When onError is continue, but taskrun failed #6664

kycheng opened this issue May 16, 2023 · 3 comments · Fixed by #6675
Assignees
Labels
kind/bug Categorizes issue or PR as related to a bug.

Comments

@kycheng
Copy link

kycheng commented May 16, 2023

I encountered an occasional problem. When I set onError=continue in the task, the pod may fail.

Expected Behavior

taskRun can work according to the onError setting.

Actual Behavior

An error was returned, see this is returned with an error code.

Steps to Reproduce the Problem

  1. Add a task containing onError to the pipeline

pipelineRun:

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  creationTimestamp: 2023-05-12T09:26:07Z
  generation: 1
  labels:
    tekton.dev/pipeline: apm-sqclr
  name: apm-sqclr
  namespace: dev
  resourceVersion: "514912236"
  uid: 28792e05-4c5f-419a-9cf0-fdf1e42bae3f
spec:
  params:
    - name: git-url
      value: https://gitlab-test.com/repo/project
    - name: git-revision
      value: refs/merge-requests/79/head
  pipelineSpec:
    params:
      - description: git repository url
        name: git-url
        type: string
      - description: git revision
        name: git-revision
        type: string
    results:
      - description: ""
        name: apm-version
        value: $(tasks.prepare.results.apm-version)
    tasks:
      - name: git-clone
        params:
          - name: url
            value: $(params.git-url)
          - name: revision
            value: $(params.git-revision)
          - name: depth
            value: "1"
        retries: 3
        taskSpec:
          metadata: {}
          params:
            - description: git url to clone
              name: url
              type: string
            - description: git revision to checkout (branch, tag, sha, ref…)
              name: revision
              type: string
            - default: ""
              description: (optional) git refspec to fetch before checking out revision
              name: refspec
              type: string
            - default: "1"
              description: performs a shallow clone where only the most recent commit(s) will
                be fetched
              name: depth
              type: string
            - default: "false"
              description: log the commands used during execution
              name: verbose
              type: string
          results:
            - description: The precise commit SHA that was fetched by this Task
              name: commit-id
              type: string
            - description: The precise commit short SHA that was fetched by this Task
              name: commit-short-id
              type: string
            - description: The precise URL that was fetched by this Task
              name: url
              type: string
          spec: null
          steps:
            - image: registry.com/devops/tektoncd/pipeline/cmd/git-init:v0.41.0-df84152d
              name: clone
              resources: {}
              script: >
                #!/bin/sh

                echo "some script"
          workspaces:
            - description: The git repo will be cloned onto the volume backing this workspace
              name: source
        timeout: 10m0s
        workspaces:
          - name: source
            workspace: source
      - name: generated-version
        params:
          - name: repo-url
            value: $(params.git-url)
          - name: repo-ref
            value: $(params.git-revision)
        runAfter:
          - git-clone
        taskRef:
          kind: ClusterTask
          name: generate-version
        timeout: 30m0s
        workspaces:
          - name: source
            workspace: source
        # some other task
    workspaces:
      - description: >
          This workspace is shared among all the pipeline tasks to read/write
          common resources
        name: source
      - description: golang cache
        name: cache
        optional: true
  podTemplate:
    securityContext:
      runAsUser: 0
  serviceAccountName: apm-sqclr-6b89v
  timeout: 1h0m0s
  workspaces:
    - name: source
      volumeClaimTemplate:
        metadata:
          creationTimestamp: null
        spec:
          accessModes:
            - ReadWriteMany
          resources:
            requests:
              storage: 2000Mi
        status: {}
    - name: cache
      persistentVolumeClaim:
        claimName: build-cache
      subPath: golang
status:
  completionTime: 2023-05-12T09:27:09Z
  conditions:
    - lastTransitionTime: 2023-05-12T09:27:09Z
      message: "Tasks Completed: 3 (Failed: 1, Cancelled 0), Skipped: 5"
      reason: Failed
      status: "False"
      type: Succeeded
  pipelineResults:
    - name: apm-version
      value: v0.1.20
  pipelineSpec:
    description: created by build run dev/apm-sqclr
    params:
      - description: git repository url
        name: git-url
        type: string
      - description: git revision
        name: git-revision
        type: string
    results:
      - description: ""
        name: apm-version
        value: $(tasks.prepare.results.apm-version)
    tasks:
      - name: git-clone
        params:
          - name: url
            value: https://gitlab-test.com/repo/project
          - name: revision
            value: refs/merge-requests/79/head
          - name: depth
            value: "1"
        retries: 3
        taskSpec:
          metadata: {}
          params:
            - description: git url to clone
              name: url
              type: string
            - description: git revision to checkout (branch, tag, sha, ref…)
              name: revision
              type: string
            - default: ""
              description: (optional) git refspec to fetch before checking out revision
              name: refspec
              type: string
            - default: "1"
              description: performs a shallow clone where only the most recent commit(s) will
                be fetched
              name: depth
              type: string
            - default: "false"
              description: log the commands used during execution
              name: verbose
              type: string
          results:
            - description: The precise commit SHA that was fetched by this Task
              name: commit-id
              type: string
            - description: The precise commit short SHA that was fetched by this Task
              name: commit-short-id
              type: string
            - description: The precise URL that was fetched by this Task
              name: url
              type: string
          spec: null
          steps:
            - image: registry.com/devops/tektoncd/pipeline/cmd/git-init:v0.41.0-df84152d
              name: clone
              resources: {}
              script: >
                #!/bin/sh

                echo "some script"
          workspaces:
            - description: The git repo will be cloned onto the volume backing this workspace
              name: source
        timeout: 10m0s
        workspaces:
          - name: source
            workspace: source
      - name: generated-version
        params:
          - name: repo-url
            value: https://gitlab-test.com/repo/project
          - name: repo-ref
            value: refs/merge-requests/79/head
        runAfter:
          - git-clone
        taskRef:
          kind: ClusterTask
          name: generate-version
        timeout: 30m0s
        workspaces:
          - name: source
            workspace: source
        # some other task
    workspaces:
      - description: >
          This workspace is shared among all the pipeline tasks to read/write
          common resources
        name: source
      - description: golang cache
        name: cache
        optional: true
  skippedTasks:
    - name: py-scan
      reason: PipelineRun was stopping
    - name: sonarqude-code-scan
      reason: PipelineRun was stopping
    - name: apm-build-image-amd64
      reason: PipelineRun was stopping
    - name: apm-build-image-arm64
      reason: PipelineRun was stopping
    - name: apm-build-image
      reason: PipelineRun was stopping
  startTime: 2023-05-12T09:26:07Z
  taskRuns:
    apm-sqclr-generated-version:
      pipelineTaskName: generated-version
      status:
        completionTime: 2023-05-12T09:26:46Z
        conditions:
          - lastTransitionTime: 2023-05-12T09:26:46Z
            message: >
              "step-deprecated-tips" exited with code 1 (image:
              "registry.com/devops/kubectl-devops@sha256:68e1f19c5962a92b755777c3b14be2d2b5a5a6af12979d968461a3ca88243267");
              for logs run: kubectl -n dev logs
              apm-sqclr-generated-version-pod -c step-deprecated-tips
            reason: Failed
            status: "False"
            type: Succeeded
        podName: apm-sqclr-generated-version-pod
        startTime: 2023-05-12T09:26:24Z
        steps:
          - container: step-deprecated-tips
            imageID: registry.com/devops/kubectl-devops@sha256:68e1f19c5962a92b755777c3b14be2d2b5a5a6af12979d968461a3ca88243267
            name: deprecated-tips
            terminated:
              containerID: containerd://388fe8ec0223882d8f37e4185a9de497de01f4aab507394a46076003751f5696
              exitCode: 1
              finishedAt: 2023-05-12T09:26:43Z
              reason: Completed
              startedAt: 2023-05-12T09:26:37Z
          - container: step-get-revision-info
            imageID: registry.com/devops/kubectl-devops@sha256:68e1f19c5962a92b755777c3b14be2d2b5a5a6af12979d968461a3ca88243267
            name: get-revision-info
            terminated:
              containerID: containerd://a09ac7a06094b7148033fa5aed359db0a577ede0c1d1486b8ce625184695b8c6
              exitCode: 0
              finishedAt: 2023-05-12T09:26:42Z
              message: '[{"key":"branch","value":"feat-32785","type":1},{"key":"branch-as-tag","value":"feat-32785","type":1},{"key":"commit-id","value":"224f016eef09fad1de26231d8ebca32cbdd858e3","type":1},{"key":"commit-short-id","value":"224f016e","type":1},{"key":"is-custom-release-branch","value":"false","type":1},{"key":"is-main-branch","value":"false","type":1},{"key":"is-release-branch","value":"false","type":1},{"key":"pr-id","value":"79","type":1},{"key":"target-branch","value":"master","type":1},{"key":"timestamp","value":"20230512092638","type":1}]'
              reason: Completed
              startedAt: 2023-05-12T09:26:38Z
          - container: step-generate-version
            imageID: registry.com/devops/kubectl-devops@sha256:68e1f19c5962a92b755777c3b14be2d2b5a5a6af12979d968461a3ca88243267
            name: generate-version
            terminated:
              containerID: containerd://305f8c9753559fb97536c3c5404ab5b4dd7e12b721f1453d79b0af6a6feb049f
              exitCode: 0
              finishedAt: 2023-05-12T09:26:45Z
              message: '[{"key":"VERSION","value":"feat-3.2305120926","type":1},{"key":"branch","value":"feat-32785","type":1},{"key":"branch-as-tag","value":"feat-32785","type":1},{"key":"commit-id","value":"224f016eef09fad1de26231d8ebca32cbdd858e3","type":1},{"key":"commit-short-id","value":"224f016e","type":1},{"key":"is-custom-release-branch","value":"false","type":1},{"key":"is-main-branch","value":"false","type":1},{"key":"is-release-branch","value":"false","type":1},{"key":"pr-id","value":"79","type":1},{"key":"semver-clean","value":"3.13.0-bfeat-3.2305120926","type":1},{"key":"semver-version","value":"v3.13.0-bfeat-3.2305120926","type":1},{"key":"target-branch","value":"master","type":1},{"key":"timestamp","value":"20230512092638","type":1},{"key":"version","value":"feat-3.2305120926","type":1}]'
              reason: Completed
              startedAt: 2023-05-12T09:26:42Z
        taskResults:
          - name: branch
            type: string
            value: feat-32785
          - name: branch-as-tag
            type: string
            value: feat-32785
          - name: commit-id
            type: string
            value: 224f016eef09fad1de26231d8ebca32cbdd858e3
          - name: commit-short-id
            type: string
            value: 224f016e
          - name: is-custom-release-branch
            type: string
            value: "false"
          - name: is-main-branch
            type: string
            value: "false"
          - name: is-release-branch
            type: string
            value: "false"
          - name: pr-id
            type: string
            value: "79"
          - name: target-branch
            type: string
            value: master
          - name: timestamp
            type: string
            value: "20230512092638"
          - name: VERSION
            type: string
            value: feat-3.2305120926
          - name: semver-clean
            type: string
            value: 3.13.0-bfeat-3.2305120926
          - name: semver-version
            type: string
            value: v3.13.0-bfeat-3.2305120926
          - name: version
            type: string
            value: feat-3.2305120926
        taskSpec:
          description: Generates versions for components
          params:
            - default: gitlab
              description: |-
                One of:
                 - gitlab
                 - github
              name: repo-type
              type: string
            - description: full qualified URL for code repository
              name: repo-url
              type: string
            - description: ref for repo
              name: repo-ref
              type: string
            - default: registry.com/devops/kubectl-devops:v3.3.0-beta.2.g525a9a02
              description: imaged used to execute task, defaults to
                registry.com/devops/kubectl-devops:v3.3.0-beta.2.g525a9a02
              name: tools-image
              type: string
            - default: "false"
              description: if true will print more debug related output
              name: verbose
              type: string
            - default: "true"
              description: if true will generate and validate semver, if invalid will fail, if
                false will not validate semver but will return same value as
                version
              name: semver-strict
              type: string
          results:
            - description: generated version
              name: VERSION
              type: string
            - description: generated version
              name: version
              type: string
            - description: generated semver version
              name: semver-version
              type: string
            - description: generated clean semver version
              name: semver-clean
              type: string
            - description: source branch of current revision. will return null for tag and
                commit revisions
              name: branch
              type: string
            - description: source branch of current revision in a format to be used as tag.
                will return null for tag and commit revisions
              name: branch-as-tag
              type: string
            - description: returns true if is a main branch
              name: is-main-branch
              type: string
            - description: returns true if is a release branch
              name: is-release-branch
              type: string
            - description: returns true if is a custom release branch
              name: is-custom-release-branch
              type: string
            - description: returns PR ID if revision is a pull request
              name: pr-id
              type: string
            - description: returns PR target branch if revision is a pull request
              name: target-branch
              type: string
            - description: The precise commit SHA that was fetched by this Task
              name: commit-id
              type: string
            - description: The precise commit short SHA that was fetched by this Task
              name: commit-short-id
              type: string
            - description: Runtime timestamp yyyyMMddHHmmss
              name: timestamp
              type: string
          steps:
            - image: registry.com/devops/kubectl-devops:v3.3.0-beta.2.g525a9a02
              name: deprecated-tips
              onError: continue
              resources:
                limits:
                  cpu: 50m
                  memory: 100Mi
                requests:
                  cpu: 50m
                  memory: 100Mi
              script: >
                #!/bin/sh

                echo "some tips"

                exit 1
            - image: registry.com/devops/kubectl-devops:v3.3.0-beta.2.g525a9a02
              imagePullPolicy: IfNotPresent
              name: get-revision-info
              resources:
                limits:
                  cpu: 300m
                  memory: 300Mi
                requests:
                  cpu: 150m
                  memory: 150Mi
              script: >
                #!/bin/sh

                set -e

                echo "some script, the step will return 0"
              workingDir: /workspace/source
            - image: registry.com/devops/kubectl-devops:v3.3.0-beta.2.g525a9a02
              imagePullPolicy: IfNotPresent
              name: generate-version
              resources:
                limits:
                  cpu: 800m
                  memory: 1200Mi
                requests:
                  cpu: 300m
                  memory: 500Mi
              script: >
                #!/bin/sh

                set -e

                echo "some script, the step will return 0"
              workingDir: /workspace/source
          workspaces:
            - description: code source
              name: source
    apm-sqclr-git-clone:
      pipelineTaskName: git-clone
      status:
        completionTime: 2023-05-12T09:26:24Z
        conditions:
          - lastTransitionTime: 2023-05-12T09:26:24Z
            message: All Steps have completed executing
            reason: Succeeded
            status: "True"
            type: Succeeded
        podName: apm-sqclr-git-clone-pod
        resourcesResult:
          - key: commit
            value: 224f016eef09fad1de26231d8ebca32cbdd858e3
        startTime: 2023-05-12T09:26:08Z
        steps:
          - container: step-clone
            imageID: registry.com/devops/tektoncd/pipeline/cmd/git-init@sha256:cedf4419e703835d12a5aaa95158ddc9efe0463efab0c761d02145a8323a963f
            name: clone
            terminated:
              containerID: containerd://001386d7074862fe9c90bc3af2f03b434060ebf837977574d5e70524be95de9c
              exitCode: 0
              finishedAt: 2023-05-12T09:26:23Z
              message: '[{"key":"commit","value":"224f016eef09fad1de26231d8ebca32cbdd858e3"},{"key":"commit-id","value":"224f016eef09fad1de26231d8ebca32cbdd858e3","type":1},{"key":"commit-short-id","value":"224f016e","type":1},{"key":"url","value":"https://gitlab-test.com/repo/project","type":1}]'
              reason: Completed
              startedAt: 2023-05-12T09:26:21Z
        taskResults:
          - name: commit-id
            type: string
            value: 224f016eef09fad1de26231d8ebca32cbdd858e3
          - name: commit-short-id
            type: string
            value: 224f016e
          - name: url
            type: string
            value: https://gitlab-test.com/repo/project
        taskSpec:
          params:
            - description: git url to clone
              name: url
              type: string
            - description: git revision to checkout (branch, tag, sha, ref…)
              name: revision
              type: string
            - default: ""
              description: (optional) git refspec to fetch before checking out revision
              name: refspec
              type: string
            - default: "1"
              description: performs a shallow clone where only the most recent commit(s) will
                be fetched
              name: depth
              type: string
            - default: "false"
              description: log the commands used during execution
              name: verbose
              type: string
          results:
            - description: The precise commit SHA that was fetched by this Task
              name: commit-id
              type: string
            - description: The precise commit short SHA that was fetched by this Task
              name: commit-short-id
              type: string
            - description: The precise URL that was fetched by this Task
              name: url
              type: string
          steps:
            - image: registry.com/devops/tektoncd/pipeline/cmd/git-init:v0.41.0-df84152d
              name: clone
              resources: {}
              script: >
                #!/bin/sh

                echo "some script"
          workspaces:
            - description: The git repo will be cloned onto the volume backing this workspace
              name: source

In the generated-version of pipelinerun, deprecated-tips set onError: continue, but the status of this task still fails.

The incident also contained some deliberate messages.

image

From some update audits of the platform we use, we can see that there are conflicts and failures when the operator updates taskruns.

image

Additional Info

  • Kubernetes version:
1.25.6
  • Tekton Pipeline version:
v0.41.0

I feel that this is a very strange problem. To view the place where the error is reported, terminated.exitCode needs to be non-zero.

pipeline/pkg/pod/status.go

Lines 374 to 379 in e38d112

if term.ExitCode != 0 {
// Newline required at end to prevent yaml parser from breaking the log help text at 80 chars
return fmt.Sprintf("%q exited with code %d (image: %q); for logs run: kubectl -n %s logs %s -c %s\n",
status.Name, term.ExitCode, status.ImageID,
pod.Namespace, pod.Name, status.Name)
}

But after setting OnError=continue, the pod's exitCode should be 0,

case e.OnError == ContinueOnError && errors.As(err, &ee):
// with continue on error and an ExitError, write non-zero exit code and a post file
exitCode := strconv.Itoa(ee.ExitCode())
output = append(output, v1beta1.PipelineResourceResult{
Key: "ExitCode",
Value: exitCode,
ResultType: v1beta1.InternalTektonResultType,
})
e.WritePostFile(e.PostFile, nil)
e.WriteExitCodeFile(e.StepMetadataDir, exitCode)

and only when the taskrun is in the end state will the taskRun's terminated.exitCode be set to the actual exitCode.
https://github.com/tektoncd/pipeline/blob/e38d112e752ebbe97e4e3eccb1c271bbe5ecbcf7/pkg/pod/status.go#L164C5-L187

Maybe there are some details that I don't know, can you help to look at the problem?

Thanks

@kycheng kycheng added the kind/bug Categorizes issue or PR as related to a bug. label May 16, 2023
@l-qing
Copy link
Contributor

l-qing commented May 17, 2023

Analysis:

  1. TaskRun.status.conditions[0].message contains for logs run: kubectl -n
  taskRuns:
    apm-sqclr-generated-version:
      statu
        conditions:
          - lastTransitionTime: 2023-05-12T09:26:46Z
            message: >
              "step-deprecated-tips" exited with code 1 (image:
              "registry.com/devops/kubectl-devops@sha256:68e1f19c5962a92b755777c3b14be2d2b5a5a6af12979d968461a3ca88243267");
              for logs run: kubectl -n dev logs
              apm-sqclr-generated-version-pod -c step-deprecated-tips
  1. The only source of the code: getFailureMessage
    This will only trigger if term.ExitCode != 0

    pipeline/pkg/pod/status.go

    Lines 359 to 379 in e38d112

    func getFailureMessage(logger *zap.SugaredLogger, pod *corev1.Pod) string {
    // First, try to surface an error about the actual build step that failed.
    for _, status := range pod.Status.ContainerStatuses {
    term := status.State.Terminated
    if term != nil {
    msg := status.State.Terminated.Message
    r, _ := termination.ParseMessage(logger, msg)
    for _, result := range r {
    if result.ResultType == v1beta1.InternalTektonResultType && result.Key == "Reason" && result.Value == "TimeoutExceeded" {
    // Newline required at end to prevent yaml parser from breaking the log help text at 80 chars
    return fmt.Sprintf("%q exited because the step exceeded the specified timeout limit; for logs run: kubectl -n %s logs %s -c %s\n",
    status.Name,
    pod.Namespace, pod.Name, status.Name)
    }
    }
    if term.ExitCode != 0 {
    // Newline required at end to prevent yaml parser from breaking the log help text at 80 chars
    return fmt.Sprintf("%q exited with code %d (image: %q); for logs run: kubectl -n %s logs %s -c %s\n",
    status.Name, term.ExitCode, status.ImageID,
    pod.Namespace, pod.Name, status.Name)
    }

  2. The only function call: updateCompletedTaskRunStatus

    pipeline/pkg/pod/status.go

    Lines 304 to 307 in e38d112

    func updateCompletedTaskRunStatus(logger *zap.SugaredLogger, trs *v1beta1.TaskRunStatus, pod *corev1.Pod) {
    if DidTaskRunFail(pod) {
    msg := getFailureMessage(logger, pod)
    markStatusFailure(trs, v1beta1.TaskRunReasonFailed.String(), msg)

  3. The only function call: MakeTaskRunStatus

    pipeline/pkg/pod/status.go

    Lines 102 to 115 in e38d112

    // MakeTaskRunStatus returns a TaskRunStatus based on the Pod's status.
    func MakeTaskRunStatus(logger *zap.SugaredLogger, tr v1beta1.TaskRun, pod *corev1.Pod) (v1beta1.TaskRunStatus, error) {
    trs := &tr.Status
    if trs.GetCondition(apis.ConditionSucceeded) == nil || trs.GetCondition(apis.ConditionSucceeded).Status == corev1.ConditionUnknown {
    // If the taskRunStatus doesn't exist yet, it's because we just started running
    markStatusRunning(trs, v1beta1.TaskRunReasonRunning.String(), "Not all Steps in the Task have finished executing")
    }
    sortPodContainerStatuses(pod.Status.ContainerStatuses, pod.Spec.Containers)
    complete := areStepsComplete(pod) || pod.Status.Phase == corev1.PodSucceeded || pod.Status.Phase == corev1.PodFailed
    if complete {
    updateCompletedTaskRunStatus(logger, trs, pod)

  4. The only function call: taskRun.Reconciler.reconcile

    // Convert the Pod's status to the equivalent TaskRun Status.
    tr.Status, err = podconvert.MakeTaskRunStatus(logger, *tr, pod)

  5. Source of pods:

    // Get the TaskRun's Pod if it should have one. Otherwise, create the Pod.
    var pod *corev1.Pod
    if tr.Status.PodName != "" {
    pod, err = c.podLister.Pods(tr.Namespace).Get(tr.Status.PodName)

  6. Events before the for logs run: kubectl -n: All Steps have completed executing
    It is indicates that this taskrun was already in a successful state.

    pipeline/pkg/pod/status.go

    Lines 497 to 505 in e38d112

    // markStatusSuccess sets taskrun status to success
    func markStatusSuccess(trs *v1beta1.TaskRunStatus) {
    trs.SetCondition(&apis.Condition{
    Type: apis.ConditionSucceeded,
    Status: corev1.ConditionTrue,
    Reason: v1beta1.TaskRunReasonSuccessful.String(),
    Message: "All Steps have completed executing",
    })
    }

  7. Thinking:

  • Under what circumstances would pod.Status.ContainerStatuses[x].State.Terminated != 0 and thus enter into the codeblock for the (1) condition?
  • Why would the status of a taskrun change from successful to failed?
  • Why are there conflict records in audit submissions?
  1. Places that may affect the ExitCode value:
  • Pod from podLister (6)
    • // Get the TaskRun's Pod if it should have one. Otherwise, create the Pod.
      var pod *corev1.Pod
      if tr.Status.PodName != "" {
      pod, err = c.podLister.Pods(tr.Namespace).Get(tr.Status.PodName)
  • setTaskRunStatusBasedOnStepStatus
    • Set the ExitCode of the Terminated based on the container's message.
    • pipeline/pkg/pod/status.go

      Lines 146 to 186 in e38d112

      func setTaskRunStatusBasedOnStepStatus(logger *zap.SugaredLogger, stepStatuses []corev1.ContainerStatus, tr *v1beta1.TaskRun) *multierror.Error {
      trs := &tr.Status
      var merr *multierror.Error
      for _, s := range stepStatuses {
      if s.State.Terminated != nil && len(s.State.Terminated.Message) != 0 {
      msg := s.State.Terminated.Message
      results, err := termination.ParseMessage(logger, msg)
      if err != nil {
      logger.Errorf("termination message could not be parsed as JSON: %v", err)
      merr = multierror.Append(merr, err)
      } else {
      time, err := extractStartedAtTimeFromResults(results)
      if err != nil {
      logger.Errorf("error setting the start time of step %q in taskrun %q: %v", s.Name, tr.Name, err)
      merr = multierror.Append(merr, err)
      }
      exitCode, err := extractExitCodeFromResults(results)
      if err != nil {
      logger.Errorf("error extracting the exit code of step %q in taskrun %q: %v", s.Name, tr.Name, err)
      merr = multierror.Append(merr, err)
      }
      taskResults, pipelineResourceResults, filteredResults := filterResultsAndResources(results)
      if tr.IsSuccessful() {
      trs.TaskRunResults = append(trs.TaskRunResults, taskResults...)
      trs.ResourcesResult = append(trs.ResourcesResult, pipelineResourceResults...)
      }
      msg, err = createMessageFromResults(filteredResults)
      if err != nil {
      logger.Errorf("%v", err)
      merr = multierror.Append(merr, err)
      } else {
      s.State.Terminated.Message = msg
      }
      if time != nil {
      s.State.Terminated.StartedAt = *time
      }
      if exitCode != nil {
      s.State.Terminated.ExitCode = *exitCode
      }
    • setTaskRunStatusBasedOnStepStatus will be called only after updateCompletedTaskRunStatus in MakeTaskRunStatus.
    • pipeline/pkg/pod/status.go

      Lines 102 to 135 in e38d112

      // MakeTaskRunStatus returns a TaskRunStatus based on the Pod's status.
      func MakeTaskRunStatus(logger *zap.SugaredLogger, tr v1beta1.TaskRun, pod *corev1.Pod) (v1beta1.TaskRunStatus, error) {
      trs := &tr.Status
      if trs.GetCondition(apis.ConditionSucceeded) == nil || trs.GetCondition(apis.ConditionSucceeded).Status == corev1.ConditionUnknown {
      // If the taskRunStatus doesn't exist yet, it's because we just started running
      markStatusRunning(trs, v1beta1.TaskRunReasonRunning.String(), "Not all Steps in the Task have finished executing")
      }
      sortPodContainerStatuses(pod.Status.ContainerStatuses, pod.Spec.Containers)
      complete := areStepsComplete(pod) || pod.Status.Phase == corev1.PodSucceeded || pod.Status.Phase == corev1.PodFailed
      if complete {
      updateCompletedTaskRunStatus(logger, trs, pod)
      } else {
      updateIncompleteTaskRunStatus(trs, pod)
      }
      trs.PodName = pod.Name
      trs.Steps = []v1beta1.StepState{}
      trs.Sidecars = []v1beta1.SidecarState{}
      var stepStatuses []corev1.ContainerStatus
      var sidecarStatuses []corev1.ContainerStatus
      for _, s := range pod.Status.ContainerStatuses {
      if IsContainerStep(s.Name) {
      stepStatuses = append(stepStatuses, s)
      } else if isContainerSidecar(s.Name) {
      sidecarStatuses = append(sidecarStatuses, s)
      }
      }
      var merr *multierror.Error
      if err := setTaskRunStatusBasedOnStepStatus(logger, stepStatuses, &tr); err != nil {
    • In other words, the ExitCode set in the setTaskRunStatusBasedOnStepStatus will not affect the judgment in getFailureMessage above.
  1. Thinking:
  • Why pod.Status.ContainerStatuses[x].State.Terminated != 0?
  • Who could possibly modify it?
    • It seems that there is only a function called setTaskRunStatusBasedOnStepStatus.
    • It modifies the reference value, which will affect the original value.
    • But the Pod that is obtained when calling getFailureMessage only comes from podLister.
      • // Get the TaskRun's Pod if it should have one. Otherwise, create the Pod.
        var pod *corev1.Pod
        if tr.Status.PodName != "" {
        pod, err = c.podLister.Pods(tr.Namespace).Get(tr.Status.PodName)
    • PodNamespaceLister
  1. Important: PodNamespaceLister.Get: Objects returned here must be treated as read-only
  • Is it possible that the previous reconciler modified the state of the Pod, and the second reconciler obtained the modified Pod value?
  • If this assumption is correct, do we need to avoid modifying the original value of the Pod here?

@l-qing
Copy link
Contributor

l-qing commented May 17, 2023

/assign

@l-qing
Copy link
Contributor

l-qing commented May 17, 2023

I modified the code of Tekton to verify a hypothesis.

  1. I printed the status information of the Pod before MakeTaskRunStatus in the taskrun reconciler.
  2. When there is an attempt to modify the exit code, I return an error to force a reconcile.
  3. I triggered a pipeline run and was able to reproduce the issue.
  4. Sometimes it can print out "exitCode":1
    The guess can be verified in the printed Pod status.
{"severity":"info","timestamp":"2023-05-17T11:22:48.727Z","logger":"tekton-pipelines-controller","caller":"taskrun/taskrun.go:537","message":"TEST: MakeTaskRunStatus","commit":"e38d112-dirty","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"e420e3df-72f4-43b6-a32d-a393399e35ec","knative.dev/key":"devops/build-c8gtb-task-21","pod":[{"name":"step-deprecated-tips","state":{"terminated":{"exitCode":0,"reason":"Completed","message":"[{\"key\":\"StartedAt\",\"value\":\"2023-05-17T11:22:45.316Z\",\"type\":3},{\"key\":\"ExitCode\",\"value\":\"1\",\"type\":3}]","startedAt":"2023-05-17T11:22:38Z","finishedAt":"2023-05-17T11:22:46Z","containerID":"containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a"}},"lastState":{},"ready":false,"restartCount":0,"image":"alpine","imageID":"alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7","containerID":"containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a","started":false},{"name":"step-deprecated-tips1","state":{"running":{"startedAt":"2023-05-17T11:22:41Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"alpine","imageID":"alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7","containerID":"containerd://b3d98cc4e284b2c19d063f0c3eb01b9c92e401c55dae7de64002b2b8057bcf14","started":true}]}
{"severity":"info","timestamp":"2023-05-17T11:22:48.727Z","logger":"tekton-pipelines-controller","caller":"pod/status.go:201","message":"TEST: Test update pod terminated exitcode","commit":"e38d112-dirty","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"e420e3df-72f4-43b6-a32d-a393399e35ec","knative.dev/key":"devops/build-c8gtb-task-21"}
{"severity":"info","timestamp":"2023-05-17T11:22:48.808Z","logger":"tekton-pipelines-controller","caller":"taskrun/taskrun.go:537","message":"TEST: MakeTaskRunStatus","commit":"e38d112-dirty","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"580a66d3-0575-444d-be00-8d23b6410159","knative.dev/key":"devops/build-c8gtb-task-21","pod":[{"name":"step-deprecated-tips","state":{"terminated":{"exitCode":1,"reason":"Completed","startedAt":"2023-05-17T11:22:45Z","finishedAt":"2023-05-17T11:22:46Z","containerID":"containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a"}},"lastState":{},"ready":false,"restartCount":0,"image":"alpine","imageID":"alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7","containerID":"containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a","started":false},{"name":"step-deprecated-tips1","state":{"running":{"startedAt":"2023-05-17T11:22:41Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"alpine","imageID":"alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7","containerID":"containerd://b3d98cc4e284b2c19d063f0c3eb01b9c92e401c55dae7de64002b2b8057bcf14","started":true}]}
{"severity":"info","timestamp":"2023-05-17T11:22:48.888Z","logger":"tekton-pipelines-controller","caller":"taskrun/taskrun.go:537","message":"TEST: MakeTaskRunStatus","commit":"e38d112-dirty","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"b17f1462-a137-4135-867c-8bfb0d36f15a","knative.dev/key":"devops/build-c8gtb-task-21","pod":[{"name":"step-deprecated-tips","state":{"terminated":{"exitCode":1,"reason":"Completed","startedAt":"2023-05-17T11:22:45Z","finishedAt":"2023-05-17T11:22:46Z","containerID":"containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a"}},"lastState":{},"ready":false,"restartCount":0,"image":"alpine","imageID":"alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7","containerID":"containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a","started":false},{"name":"step-deprecated-tips1","state":{"running":{"startedAt":"2023-05-17T11:22:41Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"alpine","imageID":"alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7","containerID":"containerd://b3d98cc4e284b2c19d063f0c3eb01b9c92e401c55dae7de64002b2b8057bcf14","started":true}]}
{"severity":"info","timestamp":"2023-05-17T11:22:50.810Z","logger":"tekton-pipelines-controller","caller":"taskrun/taskrun.go:537","message":"TEST: MakeTaskRunStatus","commit":"e38d112-dirty","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"f08c6e03-561f-495d-a025-5f1df3079058","knative.dev/key":"devops/build-c8gtb-task-21","pod":[{"name":"step-deprecated-tips","state":{"terminated":{"exitCode":0,"reason":"Completed","message":"[{\"key\":\"StartedAt\",\"value\":\"2023-05-17T11:22:45.316Z\",\"type\":3},{\"key\":\"ExitCode\",\"value\":\"1\",\"type\":3}]","startedAt":"2023-05-17T11:22:38Z","finishedAt":"2023-05-17T11:22:46Z","containerID":"containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a"}},"lastState":{},"ready":false,"restartCount":0,"image":"alpine","imageID":"alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7","containerID":"containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a","started":false},{"name":"step-deprecated-tips1","state":{"terminated":{"exitCode":0,"reason":"Completed","message":"[{\"key\":\"StartedAt\",\"value\":\"2023-05-17T11:22:46.108Z\",\"type\":3},{\"key\":\"ExitCode\",\"value\":\"1\",\"type\":3}]","startedAt":"2023-05-17T11:22:41Z","finishedAt":"2023-05-17T11:22:46Z","containerID":"containerd://b3d98cc4e284b2c19d063f0c3eb01b9c92e401c55dae7de64002b2b8057bcf14"}},"lastState":{},"ready":false,"restartCount":0,"image":"alpine","imageID":"alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7","containerID":"containerd://b3d98cc4e284b2c19d063f0c3eb01b9c92e401c55dae7de64002b2b8057bcf14","started":false}]}
{"severity":"info","timestamp":"2023-05-17T11:22:50.818Z","logger":"tekton-pipelines-controller","caller":"pod/status.go:201","message":"TEST: Test update pod terminated exitcode","commit":"e38d112-dirty","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"f08c6e03-561f-495d-a025-5f1df3079058","knative.dev/key":"devops/build-c8gtb-task-21"}
{"severity":"info","timestamp":"2023-05-17T11:22:50.970Z","logger":"tekton-pipelines-controller","caller":"taskrun/taskrun.go:537","message":"TEST: MakeTaskRunStatus","commit":"e38d112-dirty","knative.dev/controller":"github.com.tektoncd.pipeline.pkg.reconciler.taskrun.Reconciler","knative.dev/kind":"tekton.dev.TaskRun","knative.dev/traceid":"39102143-13d6-4f70-892e-55d41500e542","knative.dev/key":"devops/build-c8gtb-task-21","pod":[{"name":"step-deprecated-tips","state":{"terminated":{"exitCode":1,"reason":"Completed","startedAt":"2023-05-17T11:22:45Z","finishedAt":"2023-05-17T11:22:46Z","containerID":"containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a"}},"lastState":{},"ready":false,"restartCount":0,"image":"alpine","imageID":"alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7","containerID":"containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a","started":false},{"name":"step-deprecated-tips1","state":{"terminated":{"exitCode":1,"reason":"Completed","startedAt":"2023-05-17T11:22:46Z","finishedAt":"2023-05-17T11:22:46Z","containerID":"containerd://b3d98cc4e284b2c19d063f0c3eb01b9c92e401c55dae7de64002b2b8057bcf14"}},"lastState":{},"ready":false,"restartCount":0,"image":"alpine","imageID":"alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7","containerID":"containerd://b3d98cc4e284b2c19d063f0c3eb01b9c92e401c55dae7de64002b2b8057bcf14","started":false}]}

Changes

l-qing@1a12304
Shot 2023 05 17 at 19 41 12

Test PipelineRun

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: build-c8gtb
  namespace: devops
spec:
  pipelineSpec:
    tasks:
    - name: task-1
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-2
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-21
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-22
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-23
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-24
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-25
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-3
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: task
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            echo Hello
    workspaces:
    - description: |
        This workspace is shared among all the pipeline tasks to read/write common resources
      name: source
  podTemplate:
    securityContext:
      runAsUser: 0
  timeout: 1h0m0s
  workspaces:
  - name: source
    volumeClaimTemplate:
      metadata:
        creationTimestamp: null
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 500Mi
      status: {}
status:
  completionTime: "2023-05-17T11:23:57Z"
  conditions:
  - lastTransitionTime: "2023-05-17T11:23:57Z"
    message: 'Tasks Completed: 9 (Failed: 2, Cancelled 0), Skipped: 0'
    reason: Failed
    status: "False"
    type: Succeeded
  pipelineSpec:
    tasks:
    - name: task-1
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-2
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-21
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-22
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-23
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-24
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-25
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: deprecated-tips
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
        - image: alpine
          name: deprecated-tips1
          onError: continue
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            exit 1
    - name: task-3
      taskSpec:
        metadata: {}
        spec: null
        steps:
        - image: alpine
          name: task
          resources:
            limits:
              cpu: 50m
              memory: 100Mi
            requests:
              cpu: 50m
              memory: 100Mi
          script: |
            #!/bin/sh
            echo Hello
    workspaces:
    - description: |
        This workspace is shared among all the pipeline tasks to read/write common resources
      name: source
  startTime: "2023-05-17T11:20:55Z"
  taskRuns:
    build-c8gtb-task-1:
      pipelineTaskName: task-1
      status:
        completionTime: "2023-05-17T11:22:50Z"
        conditions:
        - lastTransitionTime: "2023-05-17T11:22:50Z"
          message: All Steps have completed executing
          reason: Succeeded
          status: "True"
          type: Succeeded
        podName: build-c8gtb-task-1-pod
        startTime: "2023-05-17T11:21:19Z"
        steps:
        - container: step-deprecated-tips
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips
          terminated:
            containerID: containerd://f4195c65f642f7eff222d4d52282d6415c03f6afd81c232e75a15103fa00f511
            exitCode: 1
            finishedAt: "2023-05-17T11:22:45Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:45Z"
        - container: step-deprecated-tips1
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips1
          terminated:
            containerID: containerd://fdad63826f4f4896b25b9575773db5e2b2fe00d1c042bbe42a9364cfdfa28190
            exitCode: 1
            finishedAt: "2023-05-17T11:22:46Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:46Z"
        taskSpec:
          steps:
          - image: alpine
            name: deprecated-tips
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
          - image: alpine
            name: deprecated-tips1
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
    build-c8gtb-task-2:
      pipelineTaskName: task-2
      status:
        completionTime: "2023-05-17T11:22:44Z"
        conditions:
        - lastTransitionTime: "2023-05-17T11:22:44Z"
          message: All Steps have completed executing
          reason: Succeeded
          status: "True"
          type: Succeeded
        podName: build-c8gtb-task-2-pod
        startTime: "2023-05-17T11:21:19Z"
        steps:
        - container: step-deprecated-tips
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips
          terminated:
            containerID: containerd://e0a5de5fa14455d9f99802f7b5f93018574bfbcefb9696345cd114e0a1b7e6c0
            exitCode: 1
            finishedAt: "2023-05-17T11:22:41Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:39Z"
        - container: step-deprecated-tips1
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips1
          terminated:
            containerID: containerd://4edefa3ff219f393847fdc576eff029af74e047c12813264626e47500e87510d
            exitCode: 1
            finishedAt: "2023-05-17T11:22:41Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:40Z"
        taskSpec:
          steps:
          - image: alpine
            name: deprecated-tips
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
          - image: alpine
            name: deprecated-tips1
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
    build-c8gtb-task-3:
      pipelineTaskName: task-3
      status:
        completionTime: "2023-05-17T11:22:44Z"
        conditions:
        - lastTransitionTime: "2023-05-17T11:22:44Z"
          message: All Steps have completed executing
          reason: Succeeded
          status: "True"
          type: Succeeded
        podName: build-c8gtb-task-3-pod
        startTime: "2023-05-17T11:21:21Z"
        steps:
        - container: step-task
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: task
          terminated:
            containerID: containerd://7415a8bfdb4d01b8afe1f9099e14679521f6978c36407777c959fa21d5f7e174
            exitCode: 0
            finishedAt: "2023-05-17T11:22:43Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:42Z"
        taskSpec:
          steps:
          - image: alpine
            name: task
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              echo Hello
    build-c8gtb-task-21:
      pipelineTaskName: task-21
      status:
        completionTime: "2023-05-17T11:22:50Z"
        conditions:
        - lastTransitionTime: "2023-05-17T11:22:50Z"
          message: |
            "step-deprecated-tips" exited with code 1 (image: "alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7"); for logs run: kubectl -n devops logs build-c8gtb-task-21-pod -c step-deprecated-tips
          reason: Failed
          status: "False"
          type: Succeeded
        podName: build-c8gtb-task-21-pod
        startTime: "2023-05-17T11:21:20Z"
        steps:
        - container: step-deprecated-tips
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips
          terminated:
            containerID: containerd://f8081dc2c9f07d3e40023e8fbe3528a985d1959663635a7bb162445d212cf12a
            exitCode: 1
            finishedAt: "2023-05-17T11:22:46Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:45Z"
        - container: step-deprecated-tips1
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips1
          terminated:
            containerID: containerd://b3d98cc4e284b2c19d063f0c3eb01b9c92e401c55dae7de64002b2b8057bcf14
            exitCode: 1
            finishedAt: "2023-05-17T11:22:46Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:46Z"
        taskSpec:
          steps:
          - image: alpine
            name: deprecated-tips
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
          - image: alpine
            name: deprecated-tips1
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
    build-c8gtb-task-22:
      pipelineTaskName: task-22
      status:
        completionTime: "2023-05-17T11:22:49Z"
        conditions:
        - lastTransitionTime: "2023-05-17T11:22:49Z"
          message: All Steps have completed executing
          reason: Succeeded
          status: "True"
          type: Succeeded
        podName: build-c8gtb-task-22-pod
        startTime: "2023-05-17T11:21:20Z"
        steps:
        - container: step-deprecated-tips
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips
          terminated:
            containerID: containerd://ba759ccb48da69b3f6f7a164e01f2604444fd52c92eaaa2a110ae41eab452e1e
            exitCode: 1
            finishedAt: "2023-05-17T11:22:44Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:43Z"
        - container: step-deprecated-tips1
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips1
          terminated:
            containerID: containerd://175764d99cdea547baff48045863c604474b6d0822a5454f259bf07b89d5d973
            exitCode: 1
            finishedAt: "2023-05-17T11:22:45Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:44Z"
        taskSpec:
          steps:
          - image: alpine
            name: deprecated-tips
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
          - image: alpine
            name: deprecated-tips1
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
    build-c8gtb-task-23:
      pipelineTaskName: task-23
      status:
        completionTime: "2023-05-17T11:22:47Z"
        conditions:
        - lastTransitionTime: "2023-05-17T11:22:47Z"
          message: |
            "step-deprecated-tips" exited with code 1 (image: "alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7"); for logs run: kubectl -n devops logs build-c8gtb-task-23-pod -c step-deprecated-tips
          reason: Failed
          status: "False"
          type: Succeeded
        podName: build-c8gtb-task-23-pod
        startTime: "2023-05-17T11:21:20Z"
        steps:
        - container: step-deprecated-tips
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips
          terminated:
            containerID: containerd://22132597c316003eafcd00d7d34fbcbd242062bb1966fae5b7fdc68156f7d860
            exitCode: 1
            finishedAt: "2023-05-17T11:22:44Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:44Z"
        - container: step-deprecated-tips1
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips1
          terminated:
            containerID: containerd://8806dacff8f7a4eb96dedbcbdb93fef67b9cadfaeacb25453ee62b6219945ea9
            exitCode: 1
            finishedAt: "2023-05-17T11:22:45Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:44Z"
        taskSpec:
          steps:
          - image: alpine
            name: deprecated-tips
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
          - image: alpine
            name: deprecated-tips1
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
    build-c8gtb-task-24:
      pipelineTaskName: task-24
      status:
        completionTime: "2023-05-17T11:23:57Z"
        conditions:
        - lastTransitionTime: "2023-05-17T11:23:57Z"
          message: All Steps have completed executing
          reason: Succeeded
          status: "True"
          type: Succeeded
        podName: build-c8gtb-task-24-pod
        startTime: "2023-05-17T11:21:20Z"
        steps:
        - container: step-deprecated-tips
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips
          terminated:
            containerID: containerd://780990efcd7ecc43c81e32926c41c549724cd0d061770a7bccd14d8129d0c843
            exitCode: 1
            finishedAt: "2023-05-17T11:23:56Z"
            reason: Completed
            startedAt: "2023-05-17T11:23:56Z"
        - container: step-deprecated-tips1
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips1
          terminated:
            containerID: containerd://9452bc8d3eea311d6a513298fc4e82050ef332daad05fbbb701724102082aed7
            exitCode: 1
            finishedAt: "2023-05-17T11:23:57Z"
            reason: Completed
            startedAt: "2023-05-17T11:23:56Z"
        taskSpec:
          steps:
          - image: alpine
            name: deprecated-tips
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
          - image: alpine
            name: deprecated-tips1
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
    build-c8gtb-task-25:
      pipelineTaskName: task-25
      status:
        completionTime: "2023-05-17T11:22:49Z"
        conditions:
        - lastTransitionTime: "2023-05-17T11:22:49Z"
          message: All Steps have completed executing
          reason: Succeeded
          status: "True"
          type: Succeeded
        podName: build-c8gtb-task-25-pod
        startTime: "2023-05-17T11:21:21Z"
        steps:
        - container: step-deprecated-tips
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips
          terminated:
            containerID: containerd://1644a6e541aff6226feb25ce2683d26bc32a1018e16fb2bdc31f7747d687409e
            exitCode: 1
            finishedAt: "2023-05-17T11:22:44Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:44Z"
        - container: step-deprecated-tips1
          imageID: alpine@sha256:3abbd87666664e68097ecb2482f54b65f0c6a533bb107ecd62011abbb2701ae7
          name: deprecated-tips1
          terminated:
            containerID: containerd://bbae4dbd843974ae7f1ee03c6509679605a566c14a39c7d94bd6d8d1c2800969
            exitCode: 1
            finishedAt: "2023-05-17T11:22:45Z"
            reason: Completed
            startedAt: "2023-05-17T11:22:45Z"
        taskSpec:
          steps:
          - image: alpine
            name: deprecated-tips
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1
          - image: alpine
            name: deprecated-tips1
            onError: continue
            resources:
              limits:
                cpu: 50m
                memory: 100Mi
              requests:
                cpu: 50m
                memory: 100Mi
            script: |
              #!/bin/sh
              exit 1

l-qing added a commit to l-qing/pipeline that referenced this issue May 17, 2023
fix tektoncd#6664

directly modifying the value returned by Lister may affect
the evaluation during the next reconciliation.
l-qing added a commit to l-qing/pipeline that referenced this issue May 22, 2023
fix tektoncd#6664

directly modifying the value returned by Lister may affect
the evaluation during the next reconciliation.
l-qing added a commit to l-qing/pipeline that referenced this issue May 24, 2023
fix tektoncd#6664

directly modifying the value returned by Lister may affect
the evaluation during the next reconciliation.
l-qing added a commit to l-qing/pipeline that referenced this issue May 25, 2023
fix tektoncd#6664

directly modifying the value returned by Lister may affect
the evaluation during the next reconciliation.
l-qing added a commit to l-qing/pipeline that referenced this issue May 26, 2023
fix tektoncd#6664

directly modifying the value returned by Lister may affect
the evaluation during the next reconciliation.
l-qing added a commit to l-qing/pipeline that referenced this issue May 28, 2023
fix tektoncd#6664

directly modifying the value returned by Lister may affect
the evaluation during the next reconciliation.
tekton-robot pushed a commit that referenced this issue May 29, 2023
fix #6664

directly modifying the value returned by Lister may affect
the evaluation during the next reconciliation.
NigelZHANG pushed a commit to katanomi/pipeline that referenced this issue Jun 7, 2023
fix tektoncd#6664

directly modifying the value returned by Lister may affect
the evaluation during the next reconciliation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants