Specify agent delete grace period

[anbraten]

Currently unused agents are deleted as soon as the reconciliation loop runs again, instead it could be helpful to have some grace period before removing agents.

[xoxys]

This is exactly what WOODPECKER_AGENT_ALLOWED_STARTUP_TIME does but as you have renamed this for some reason from WOODPECKER_MIN_AGE in 51e6315 this var name is now misleading...

[anbraten]

Not really. That way you would just make sure an agent is alive for at least x minutes. Imagine an agent being actively doing tasks for 1 hour. After this it has nothing to do for a few seconds and gets immediately removed instead of waiting for x more minutes in which it might get some more tasks. So it has to be last task done + x minutes

[OrvilleQ]

I really hope this feature will be implemented in the near future.

I also had a problem with autoscaler deleting agents so fast that if I push a new commit and woodpecker cancels the last workflow, I have to wait another 4 or 5 minutes to create a new agent. This is really annoying.

[OrvilleQ]

I also had a problem with autoscaler deleting agents so fast that if I push a new commit and woodpecker cancels the last workflow, I have to wait another 4 or 5 minutes to create a new agent. This is really annoying.

And also, in order to achieve as much cost optimization as possible, there should probably have a smarter agent removal rule set.

Take Hetzner for example, if I understand correctly they calculate the cost of VPS and IPs on an hourly basis. If the associated resource is used for less than an hour, it is counted as an hour.

For maximum cost optimization, it might be a good idea to have an idle window and a deletion window. When an Agent is created, it enters an idle window of, say, 58 minutes. The Agent should not be deleted during this time, even if the CI/CD is not running, because the service provider will still charge an hourly rate even if the Agent is recycled. After the idle window ends, enter the deletion window, e.g., 2 minutes. During this time the Agent should be deleted if it meets the conditions and if it is still running then it should go to the next idle window.

[anbraten]

So we need somehow the following options, right?:

• inactivity / not connected to server time (could include the time a server needs to initially connect)
• idle time after which we plan to shutdown an agent
• time we wait before an agent scheduled for shutdown is actually shutdown