Consider implementing "forced" MD foreground deletion

[sbueringer]

User story

As a user it would be nice if a deleting MD always stays around until all underlying MachineSets and Machines are deleted.

Some details

Today MD already support foreground and background deletion. Foreground deletion has to be explicitly set by users though (e.g. via kubectl ... --cascade=foreground or by setting propagationPolicy to Foreground in DeleteOptions via client-go).

I think a lot of our users are not familiar with the details of foreground vs background deletion and how to enable it (e.g. we didn't even consider setting propagationPolicy when deleting MachineDeployments in the Cluster topology controller).

In general I think it would be better if a MD stays around until all child objects (MS / Machines) are gone (it's easier to figure out if there are still "parts" of the MD that exist). On the other side it would be nice for us if we don't have to maintain both foreground and background deletion. It makes it easier to reason through the deletion process in other controller, e.g. like the MD+MS topology controllers doing template garbage collection (we have 0 test coverage for foreground deletion there).

Additional detail: Machine & Cluster deletion today only support foreground deletion independent of what users specify (basically by implementing their own finalizer + reconcileDelete logic)

[sbueringer]

cc @vincepri @enxebre @fabriziopandini @chrischdi

I think I got the most important points.

[vincepri]

/area machineset
/area machinedeployment
/triage accepted

[fabriziopandini]

+1
/help

[k8s-ci-robot]

@fabriziopandini:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

+1
/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[vincepri]

A couple more points:

• Having a familiar experience, the same way the cluster behaves on deletion is a better UX than what we have today.
• The MachineDeployment doesn't provide any further information after it gets deleted, if Machine(s) are stuck in deletion, there is no signal and objects may hang around
• It's not super intuitive, in general, to use foreground deletion with clients

[enxebre]

The scenarios 1 and 3 described by Stefan here where propagation after giving a deletion timestamp might come in handy would benefit from enforced foreground as well #10589 (comment)

[Dhairya-Arora01]

@sbueringer ..... in reconcileDelete MDs would check for descendant MS and would delete them.... similarly we need to have finalizer and reconcileDelete in MS as well that would look for descendant Machines and once all the owned Machines get deleted, the finalizer on MS would be removed and it would deleted.... then the same thing will happen with owner MD.....

Its like a chain of foreground deletions-> MD -> MS -> Machines......
Do you think I'm on the correct path? If yes, I can start implementing this.....

[sbueringer]

Do you think I'm on the correct path? If yes, I can start implementing this.....

Yup sounds perfect. Feel free to go ahead (& assign yourself the issue)

[Dhairya-Arora01]

/assign

[chrischdi]

assign

[chrischdi]

/assign

[sbueringer]

/close

#11174 was merged

[k8s-ci-robot]

@sbueringer: Closing this issue.

In response to this:

/close

#11174 was merged

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.