ClusterTask creates CustomRun (v0.41.1 -> v0.44.2)

[rh-hemartin]

Expected Behavior

I get an error/warning when I create my ClusterTasks or they run as normal TaskRuns.

Actual Behavior

My ClusterTasks create CustomRuns instead of TaskRuns when I update from v0.41.1 to v0.44.2.

Steps to Reproduce the Problem

1. minikube start --driver=kvm2 --cpus=4 --memory=6g --disk-size=30GB --embed-certs
2. kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.41.1/release.yaml
3. kubect apply --filename dummy.yaml (file below)
4. kubect get pipelinerun dummy (this should complete after a while)
5. kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/previous/v0.44.2/release.yaml
6. kubect apply --filename dummy.yaml (file below)
7. kubect get pipelinerun dummy (this should hang)
8. kubectl get customruns (this should return a single one, called dummy-init)

dummy.yaml

# dummy.yaml
---
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: dummy-2
  namespace: default
spec:
  tasks:
  - name: init
    params:
    - name: resource
      value: resouce
    taskRef:
      apiVersion: tekton.dev/v1beta1
      kind: ClusterTask
      name: init
    workspaces:
    - name: output
      workspace: shared
  workspaces:
  - description: Main workspace that is shared across each task in the build pipeline
    name: shared
---
apiVersion: tekton.dev/v1beta1
kind: ClusterTask
metadata:
  name: init
spec:
  workspaces:
    - name: output
  params:
  - name: resource
    type: string
  steps:
  - name: hello
    image: registry.access.redhat.com/ubi9-micro:latest
    args: ["$(params.resource)"]
    command: ["echo"]
---
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: dummy
spec:
  pipelineRef:
    name: dummy-2
  workspaces:
  - name: shared
    volumeClaimTemplate:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 10Mi

Additional Info

• Kubernetes version:

  Output of kubectl version:

Client Version: v1.27.1
Kustomize Version: v5.0.1
Server Version: v1.26.3

• Tekton Pipeline version:

  Output of tkn version or kubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'

v0.41.1

v0.44.2

I am trying to understand why this is happening. Thanks!

[Yongxuanzhang]

I think this is the commit is the direct cause of the issue #5858. Which should be in the release of v0.43.0
If the taskref has apiversion and kind, it is considered as custom task and will create customrun.
To fix this issue, there are several options

1. if you're using v0.44.2, you could turn off the "enable-custom-tasks" feature flag.
2. remove the apiversion from the taskref

[Yongxuanzhang]

This is also related to this issue #6457
Maybe we should exclude task and clustertask from this function?
https://github.com/tektoncd/pipeline/blob/cec2422542b5f187e041ef9f96c55f3b2ab5380a/pkg/apis/pipeline/v1beta1/taskref_types.go#L58C1-L61
cc @jerop

[dibyom]

Yeah, this seems like a bug, we should exclude both Tasks and ClusterTasks

[ggallen]

@Yongxuanzhang , I set enable-custom-tasks: "false" in the feature-flags configmap, but there was no change in behavior. In fact, that option wasn't there at all in the configmap.

Is this option still recognized in 0.44.2?

I'd really prefer to use that to shut it off completely. Otherwise we'll have to make changes in multiple places.

[rh-hemartin]

Hi @Yongxuanzhang

The feature flag was removed on v0.44.0, and I also tried to make it work on v0.44.2, but it does not.

Removing the apiVersion from the taskRef worked properly, now a TaskRun is created instead.

Thank you very much.

[Yongxuanzhang]

@Yongxuanzhang , I set enable-custom-tasks: "false" in the feature-flags configmap, but there was no change in behavior. In fact, that option wasn't there at all in the configmap.

Is this option still recognized in 0.44.2?

I'd really prefer to use that to shut it off completely. Otherwise we'll have to make changes in multiple places.

Oh sorry, the feature flag was removed. If you don't want to make changes, another way is that we can patch a fix to previous versions.
I will open a fix today and keep you updated.

[ggallen]

@Yongxuanzhang , rather than a patch to add the enable-custom-tasks option back, why not just patch the real fix to exclude Tasks and ClusterTasks?

[Yongxuanzhang]

@Yongxuanzhang , rather than a patch to add the enable-custom-tasks option back, why not just patch the real fix to exclude Tasks and ClusterTasks?

I didn't mean to add that feature flag back, it will be a fix to exclude them as I mentioned above.
If you could use v0.43, I think the feature flag should be there?

[Yongxuanzhang]

/assign

[Yongxuanzhang]

@Yongxuanzhang , I set enable-custom-tasks: "false" in the feature-flags configmap, but there was no change in behavior. In fact, that option wasn't there at all in the configmap.

Is this option still recognized in 0.44.2?

I'd really prefer to use that to shut it off completely. Otherwise we'll have to make changes in multiple places.

Hi @ggallen, I wonder if you're ok with the option to remove the apiversion in your yamls? Based on the discussions in #6704 and docstrings apiversion is mainly used for custom tasks and it doesn't help when you use cluster tasks, tasks since the storage version for those resources are fixed in the cluster.

[ggallen]

@Yongxuanzhang, we are planning to remove apiversion from our YAMLs.