VPA should be able to handle native Sidecars

[voelzmo]

Which component are you using?:
vertical-pod-autoscaler

Is your feature request designed to solve a problem? If so describe the problem this feature should solve.:

Native Sidecars were introduced as part of https://github.com/kubernetes/enhancements/tree/master/keps/sig-node/753-sidecar-containers and released behind the SidecarContainers feature gate as alpha with k8s 1.28, promoted to beta in 1.29.
They are modeled as initContainers with restartPolicy=Always.

The VPA is currently designed to ignore initContainers, as this didn't make any sense: usually an initContainer lived off the resource requests that the other Containers in the same Pod summed up to. When the initContainer terminated, those resource were free for the long-running Containers and all was fine.
In many cases, it wasn't necessary to have good-fitting resource requests for initContainers, as this didn't influence scheduling decisions for most cases anyways – the formula Max ( Max(initContainers), Sum(Containers)) + pod overhead would only yield a different result, if the biggest initContainers requested more resources than the sum of all long-running containers.

Now, with some initContainers being Sidecars which keep running along the the regular containers of a Pod, it seems that their resource requests need to be considered.

• to make sure that capping recommendations according to a defined PodLimit still works
• to allow (optional?) source recommendations for native Sidecars, similar to what you can achieve today for non-native Sidecars by executing the VPA webhook last and with reinvocationPolicy: ifNecessary

Thoughts and more input welcome!

Additional Context
We currently also have a bug with native Sidecars leading to huge amounts of unnecessary logs: #6691

[voelzmo]

/area vertical-pod-autoscaler

[adrianmoisey]

Now, with some initContainers being Sidecars which keep running along the the regular containers of a Pod, it seems that their resource requests need to be considered

I agree that they do need to be considered.