Investigate how to put custom task spec into pipelinerun

[Tomcli]

/kind feature

Description:
Related #412

Currently, custom task doesn't support pipeline.taskSpec, so we cannot generate the loop pipeline using just one single pipelinerun CR. If we are going to take custom task as a separate resource, we need to build a custom task registry that's just for a particular custom task type (like pipelineloop).

So ideally we want to make Tekton's pipeline.taskSpec to support custom task and it also can help minimize the numbers of CR we create.

Additional information:
[Miscellaneous information that will assist in solving the issue.]

[fenglixa]

Thanks @Tomcli
I investigated it yet, that custom-task doesn't support pipeline.taskSpec currently.

A potential solution is: create a resourceOps task as the first task in the pipelinerun CR. This solution could ennsure that's a single pipelinerun CR.

How do you think?

[Tomcli]

So based on the Tekton community comment, they will take some time to get the user feedbacks before adding the taskSpec support. So we can use resourceOps to deploy our custom task resource if that works for the datastage team.

[Tomcli]

Since we might have multiple pipelines that might use the same custom task name, we should make every custom task to append a unique string (like any sequencer). At the end, we also need to use resourceOps to clean up the custom task using finally because KFP doesn't have TTL for every custom task type.

[fenglixa]

Get the feedback from tekton tektoncd/pipeline#3682, It's good that Tekton support taskSpec defining for custom task, so rush to temp support via resourceOps is not required.

[Tomcli]

@fenglixa the resourceOps option probably won't work because the pipeline will not start until all the task references are present. I think our best option is to use the taskSpec if possible.

I think the Tekton community doesn't have a strong objection. If the community is not picking this up, we can start changing the Tekton reconciler and create do some experiments to provide feedbacks.
https://github.com/tektoncd/pipeline/blob/master/pkg/reconciler/pipelinerun/resources/pipelinerunresolution.go#L356-L366

[fenglixa]

@Tomcli the pipeline won't block if the task references are not present, that mean resourceOps option is working to us from functionally. The first task will be executed, the pipeline will be failed while the reference task starting to run but didn't find it.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[Tomcli]

/close
done with v0.8.0 release

[google-oss-robot]

@Tomcli: Closing this issue.

In response to this:

/close
done with v0.8.0 release

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.