Pipeline as code

[stainboy]

Feature request

I like the way that Jenkins works with Jenkinsfile. Normally, a Jenkinsfile is in the root directory of a Git repo, which will be dynamically located and loaded when a pipeline job is triggered by PR (or relevant events)

The changes of the PR can be either application code or Jenkinsfile. And Jenkins is always running the pipeline according to the "latest" behavior described in the Jenkinsfile.

This behavior is called "Pipeline as code".

With tekton, what I understood is we need to create the Pipeline and Trigger into Kubernetes before the PR event is triggered. Then how to dynamically alter the behavior of the Pipeline for each of the PR?

Use case

Mentioned above.

[jlpettersson]

There is also another definition of Pipeline as Code https://www.thoughtworks.com/radar/techniques/pipelines-as-code
Similar to Infrastructure as Code it means that you, keep stuff in version control, and use CI/CD for automated testing for every change, e.g. every change of a Pipeline is tested before it is used for an application, or for infrastructure test it in a isolated environment before it is used for real workload.

[vdemeester]

@stainboy I think this is more or less a duplicate of #859.
There is a bunch of experimentation work around this in tektoncd/plumbing and externally.

tekton-asa-code is one example of implementation (still wip).

/cc @chmouel @afrittoli @wlynch

[vdemeester]

In addition, this is definitely a use case (Pipeline as code) Tekton components should help provide, but this is more likely outside of tektoncd/pipeline scope and more on an higher level component, that brings a bunch of Tekton components (like pipeline and triggers) together in an opiniated way.

[bobcatfish]

. but this is more likely outside of tektoncd/pipeline scope and more on an higher level component, that brings a bunch of Tekton components (like pipeline and triggers) together in an opiniated way.

My opinion is that it will be much simpler to support a feature like this directly in tektoncd/pipeline, and at the same time I agree with @vdemeester that we should be clear about our use cases and requirements before we make any decisions, which we've been trying to hash out in #2298

[vdemeester]

My opinion is that it will be much simpler to support a feature like this directly in tektoncd/pipeline

I am very interested in that opinion because, for me, in a gist, this means that tektoncd/pipeline would have to be able to listen to events (which is currently tektoncd/triggers responsability), handle versions of resources or references to them (to not override other PRs, …) and execution. I'd rather make tektoncd/pipeline smaller — aka more less having to care of execution — and let the rest be handled by other components. And most likely provide an opiniated answer to this (as a higher level tekton component).

To re-make a parallel I did elsewhere, for me tektoncd/pipeline is like a Pods (spec, controller, …) and a Pipeline-as-Code system would be something higher level that uses tektoncd/pipeline, like Service and Deployment 🙃.

[bobcatfish]

I am very interested in that opinion because, for me, in a gist, this means that tektoncd/pipeline would have to be able to listen to events (which is currently tektoncd/triggers responsability), handle versions of resources or references to them (to not override other PRs, …) and execution.

Ah interesting! @vdemeester I think you're picturing something much much more complicated than I am. Maybe that explains some of why you're pushing back in #2298? I wouldn't want to see a solution that required tektoncd/pipeline to start listening for events either. I think if you look at the example I provided in #2298 (comment) in use cases this might give you a better idea of what im imagining: allowing Pipelines to consume a repo and commitish doesn't mean that Pipelines needs to understand how those values got to it (they can be provided by triggering or by anything else).

(That being said, I also don't think it would be terrible to combine triggers and pipelines into one project as I suggest in tektoncd/triggers#697 but that's another story)

[stainboy]

I am very happy to see that the use case has been accepted and tracked in #859. Thanks for the great work, maintainers and contributors!

[bobcatfish]

Going to close this as a duplicate of #859 - hopefully you should see some exciting things this year @stainboy :)