disabling pipelinerun/taskrun timeouts leads to overrun of reconcile requeues

[gabemontero]

Expected Behavior

If the pipelinerun/taskrun reconcilers are handling items whose timeouts have been disabled by setting the timeout field to 0, then if those reconcilers decide to return a controller.NewRequeueAfter(waitTime) generated error to the controller framework, then a reasonable delay before the next reconciliation event occurs.

Actual Behavior

A negative number is set to waitTime and the controller.NewRequeueAfter(waitTime) results in the item being immediately requeued, leading to many reconciiations per second for a relatively idle pipelinerun/taskrun

Steps to Reproduce the Problem

1. Set the pipelinerun or taskrun timeout to 0
2. Create a pipelinerun or taskrun whose state remains unchanged for even just a relatively small amount of time (30 seconds, a minute even)

Additional Info

• Kubernetes version:

  various versions

`

• Tekton Pipeline version:

0.53.x and forward, minimally

I'll be submitting a PR that strives to address this momentarily

@vdemeester @afrittoli @abayer @khrm FYI

I am attaching a pipeline controller log as well. Look for instances of the logs for the pipelinerun managed-release-9ln4j where you see the log pipelinerun/pipelinerun.go:769","msg":"PipelineRun managed-release-9ln4j status is being set to ... many times a second for long stretches.
tekton-pipelines-controller-requeue-overrun.log.gz