KEP-3476: Add Volume Group Snapshot KEP

[xing-yang]

This PR proposes a VolumeGroupSnapshot CRD.

• Enhancement issue: VolumeGroupSnapshot #3476

[saad-ali]

For creating a VolumeGroup I see three paths

1. Create an empty VolumeGroup and request new un-provisioned volume(s) to be provisioned and added one by one.
2. Create an empty VolumeGroup and request existing volume(s) to be added one by one.
3. Create an non-empty VolumeGroup with a fixed set of new un-provisioned volume(s) to be provisioned.
4. Create an non-empty VolumeGroup with a fixed existing set of volume(s).

Different storage vendors may support one more of these. We need to identify what is out there to figure out which of these cases to support.

To add to the complexity this may vary based on VolumeGroup feature: snapshot vs replication vs placement vs crash consistent writes.

Similarly, for snapshot restore there may be multiple possibilities:

1. Group Snapshot returns only a GroupSnapshotID not individual snapshot IDs, therefore restore must be via a GroupSnapshotID.
2. Group Snapshot returns only individual VolumeSnapshotID, therefore restore must be on a per snapshot basis.

I added an item to the SIG Storage Meeting Agenda tomorrow to ask for storage vendor input.

[rbo54]

I have left comments. Please feel free to contact me [email protected] if you wish.

[vishnuitta]

Consider a statefulset have 3 replicas, and having 2 claims in volumeClaimTemplates.. totally, it becomes 6 PVCs
With this KEP, would it be possible to tell the storage controller that 2 claims of each replica of statefulset are of one group, and there are 3 such groups so that controller can take care of placement of volumes

This is a common usecase where application is providing high availability by storing data in such a way that even if one pod goes down, application is running. But, if the underlying storage of those 6 PVCs comes from storage of single failure domain, application cannot run.

[xing-yang]

Consider a statefulset have 3 replicas, and having 2 claims in volumeClaimTemplates.. totally, it becomes 6 PVCs
With this KEP, would it be possible to tell the storage controller that 2 claims of each replica of statefulset are of one group, and there are 3 such groups so that controller can take care of placement of volumes

This is a common usecase where application is providing high availability by storing data in such a way that even if one pod goes down, application is running. But, if the underlying storage of those 6 PVCs comes from storage of single failure domain, application cannot run.

I think that should be possible. For the 2 PVCs on the 1st replica, we can add label replica-1, for the 2 PVCs on the 2nd replica, we can add label replica-2, and so on. We can create 3 VolumeGroups, one for PVCs on each replica. This is Immutable VolumeGroup with existing PVCs.

To support Mutable VolumeGroup with StatefulSet and place PVCs in each replica in a different group would require enhancement of the StatefulSet controller. If we always create a different group for PVCs in each replica, we can modify the StatefulSet controller to add the pod name as a suffix to the group name, but if we sometimes need to create a different group for PVCs in each replica, sometimes need to create just 1 group for all the PVCs in all replicas, and sometimes need to create a group for 2 out of 3 replicas, it will be tricky.

[xing-yang]

@msau42 I addressed your comments. PTAL. Thanks.

[msau42]

Since these can only be snapshot kinds, we could make our own SnapshotReference type that only includes name.

[xing-yang]

If we only need snapshot names, can it just be a list of strings?

[msau42]

We can follow up on this during a more detailed API review, but I believe the guidance is to define a type scoped to your supported sources.

[xing-yang]

Sure

[jsafrane]

/lgtm