Better tasks order logic in Pipeline - label selectors in runAfter

[ysaakpr]

As of today, Tekton has two methods to control the sequence of tasks in Pipeline. Which are basically by using runAfter and from keywords. In a complex CI CD pipeline, sometimes its difficult to maintain runAfter or even from.

Eg: In my Pipeline(Which is created dynamically, and programmatically) has 100s of tasks. Which performs actions in parallel. In the current scenario, in order to run final tasks, I have to put all the list of the above tasks to runAfter for all the remaining tasks.

I was thinking, a way to label the tasksruns, and run a task after all the tasksruns with a label specified have completed.

Eg Pipeline

- name: test-app
  labels:
     rungroup: loading
  taskRef:
    name: make-test
- name: test-app2
  labels:
     rungroup: loading
  taskRef:
    name: make-test
- name: build-app
  taskRef:
    name: kaniko-build
  runAfter:
    - labelSelector:
          rungroup: loading

In this example build-app must run after test-app and test-app since both have same label rungroup: loading

[jlpettersson]

This can potentially be solved also by runAfter: ['taskGroupName'] that I propose in Proposal: DAG within Pod - TaskGroup comment

[ysaakpr]

@jlpettersson, Awsome, but your proposal is more for running the same group tasks in one pod than for control the ordering I guess. I think we should track these two issues separately. Else if the above ticket is really focusing on taskGroup, we may close this in favour of your ticket.

[dibyom]

You might be interested in #2446

/kind feature

[ysaakpr]

@dibyom oh, that’s nice, finally is a good idea for running any tasks even pipeline failed. Cleanup jobs are better to do at finally. Though my use cases does not stand for finally, just because they are not the end. Means I want to run few other tasks after this tasks.

Implementing finally would be easier than this. Because it just have to check for status of pipeline. Grouping looks more complex but useful feature for many situations

[tekton-robot]

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

[tekton-robot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

[tekton-robot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.

/lifecycle stale

Send feedback to tektoncd/plumbing.

[tekton-robot]

@tekton-robot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

/close

Send feedback to tektoncd/plumbing.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.