vSphere in-tree to CSI driver migration

[davidz627]

Enhancement Description

• One-line enhancement description (can be used as a release note): Migrate the internals of in-tree VSphere Plugin to call out to the VSphere CSI Driver while maintaining the original API
• Design proposal link (community repo): https://github.com/kubernetes/community/blob/master/contributors/design-proposals/storage/csi-migration.md
• KEP: https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/20190129-csi-migration.md
• Primary contact (assignee): @divyenpatel @xing-yang
• Responsible SIGs: sig-storage
• Enhancement target (which target equals to which milestone):
  • Alpha release target: 1.18
  • Beta release target (off-by-default): 1.19
  • Beta release target (on-by-default): 1.25
  • Stable release target (x.y): 1.26
• GA
  • PRs
    • Code: Promote CSIMigrationvSphere to GA kubernetes#113336
    • Doc: CSIMigrationvSphere doc update website#37701

Parent enhancement: #625
Public Migration testing CI: LINK

TODO

• Replace design/KEP with specific one if needed
• Link public migration testing CI

[davidz627]

/sig storage

[jeremyrickard]

Hey @davidz627,

I see in the enhancement description you're targeting 1.18 for this. Can you confirm if this will be in 1.18? Enhancement Freeze is about a week away and we'll need to track it now if you'd like to be included? I think in #625 you indicated 1.19 for that one, are these enhancements to track at a more granular level?

Thanks!

[davidz627]

Yes, this is more granular. There is specific integration work required for each in-tree plugin - each of which is gated by a seperate feature flag and graduates at it's own rate based on set criteria. IIUC @divyenpatel and @xing-yang are leading the vSphere storage migration and AFAIK planned a 1.18 alpha so you should confirm with them.

[jeremyrickard]

@davidz627 , as a bit of a process issue, could we get the KEP updated with the actual test plan instead of the statement that just references the design proposal?

[xing-yang]

@jeremyrickard we are still planning 1.18 alpha for vSphere driver. Can you track this please? Thanks.

[jeremyrickard]

@xing-yang yep, we'll track this. The KEP will need to be updated with the actual test plan instead of the reference to the design proposal for us to get it past enhancement freeze though.

cc: @johnbelamaric @alejandrox1 @mrbobbytables

[xing-yang]

Thanks @jeremyrickard! Looks like @davidz627 will update the main KEP with test plan.

[davidz627]

PTAL #1499

[jeremyrickard]

THANK YOU SO MUCH @davidz627

[jeremyrickard]

/milestone v1.18

[palnabarun]

@davidz627 Just a friendly reminder, we are less than 2 days away from the Enhancements Freeze (3PM Pacific Time, Tuesday, January 28th).

I see that PR to update the KEP test plans is still open. Let us know if the release team can help in any way.

[davidz627]

@palnabarun the test plan has merged

[VineethReddy02]

Hello, @davidz627, I'm 1.18 docs lead.
Does this enhancement work planned for 1.18 require any new docs (or modifications to existing docs)? If not, can you please update the 1.18 Enhancement Tracker Sheet (or let me know and I'll do so)
If so, just a friendly reminder we're looking for a PR against k/website (branch dev-1.18) due by Friday, Feb 28th, it can just be a placeholder PR at this time. Let me know if you have any questions!

[davidz627]

This should have new docs much like the ones already existing for migration for GCE and AWS.
/cc @xing-yang @divyenpatel
who are owning this issue

[xing-yang]

@VineethReddy02 @davidz627 we'll submit a doc PR soon. Thanks!

[jeremyrickard]

Hey there @xing-yang @divyenpatel

cc: @davidz627,

Just a friendly reminder that code freeze for 1.18 is March 05, 2020.

As we track toward code freeze, please list out/link to any PRs you are working on toward graduating this enhancement!

[VineethReddy02]

Hello @xing-yang
We are close to the docs placeholder PR deadline against the dev-1.18 branch. Having a placeholder PR in place will definitely help us in tracking enhancements much better.

Thanks! :)

[xing-yang]

Hi @VineethReddy02, doc placeholder PR submitted here:
kubernetes/website#19227

[jeremyrickard]

Hey there @xing-yang, are there any k/k PRs that are open for this issue? As a reminder, we're getting close to code freeze next week 05 March 2020 and we'd like to track any work that is occurring for this issue.

[xing-yang]

Hi @jeremyrickard, we are not going to make the March 5th deadline. We will be targeting 1.19 instead. Thanks.

[jeremyrickard]

/milestone 1.19

[k8s-ci-robot]

@jeremyrickard: The provided milestone is not valid for this repository. Milestones in this repository: [keps-beta, keps-ga, v1.17, v1.18, v1.19, v1.20, v1.21]

Use /milestone clear to clear the milestone.

In response to this:

/milestone 1.19

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jeremyrickard]

/milestone v1.19

[xing-yang]

Hi @krol3, here's the doc PR: kubernetes/website#35551

[xing-yang]

/milestone clear

[xing-yang]

/milestone v1.26

[rhockenbury]

Hello @divyenpatel and @xing-yang 👋, 1.26 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00 PDT on Thursday 6th October 2022.

This enhancement is targeting for stage stable for 1.26 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.26
• KEP has a updated detailed test plan section filled out
• KEP has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, please plan to merge #3513 before enhancements freeze to meet all the criteria.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[rhockenbury]

With #3513 merged, this enhancement is now tracked for the v1.26 cycle. Thanks!

[rhockenbury]

Hi @divyenpatel 👋,

Checking in as we approach 1.26 code freeze at 17:00 PDT on Tuesday 8th November 2022.

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PRs are fully merged by the code freeze deadline.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze:

• Promote CSIMigrationvSphere to GA kubernetes#113336

As always, we are here to help should questions come up. Thanks!

[katmutua]

Hello @divyenpatel! 👋🏾,

@katmutua 1.26 Release Docs shadow here. This enhancement is marked as ‘Needs Docs’ for 1.26 release.

Please follow the steps detailed in the documentation to open a PR against dev-1.26 branch in the k/website repo. This PR can be just a placeholder at this time, and must be created by November 9.

Also, take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.
As a reminder, please link all of your docs PR to this issue so we can easily track it.

[msau42]

Hi @divyenpatel @xing-yang. I just became aware that there are some storage systems listed as supported in the vmware compatibility guide, however they seem to fail when used with the CSI driver (intree driver works). It appears to be a problem with FCD. Does csi migration convert the intree volumes to FCD? If so then I think this issue is concerning because once csi migration is GA, there is no way to turn it off. I would like to propose we delay GA until we can resolve this issue. Let me know your thoughts, thanks.

[xing-yang]

Hi @msau42, can you clarify what storage systems are failing when used with the CSI driver and provide logs? We can take a look and understand what is the problem.

[divyenpatel]

Does csi migration convert the intree volumes to FCD? If so then I think this issue is concerning because once csi migration is GA, there is no way to turn it off.

Yes CSI Migration converts vmdk (intree vSphere volumes) to FCD (First Class Disk or Improved Virtual Disk).
but FCDs are still VMDKs, and have the same backingDiskURL path. Even after vmdk is converted to FCD, we can still operate on it using in-tree vSphere volume plugin. We can continue to attach/detach/delete converted volumes using legacy APIs.

Even without we GA vSphere CSI Migration flags in the Kubernetes, migration flags are already enabled by default in the Kubernetes and vSphere CSI Driver. and Volumes are getting migrated to FCD.

[msau42]

We documented the various issues encountered here: kubernetes-sigs/vsphere-csi-driver#2083. It looks like they have all been resolved, so I am comfortable proceeding with the promotion to GA this release. Thanks everyone for helping to quickly resolve the issues!

[rhockenbury]

With kubernetes/kubernetes#113336 merged, this is now marked as tracked for code freeze.

[marosset]

Hi @divyenpatel and @xing-yang - since this enhancement has graduated to stable can you please update https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/1491-csi-migration-vsphere/kep.yaml setting the status to implemented and then close this issue.
Thanks!

[divyenpatel]

@marosset PR is out up change the status - #3691

[marosset]

/close

Thanks for all the work done for this enhancement!

[k8s-ci-robot]

@marosset: Closing this issue.

In response to this:

/close

Thanks for all the work done for this enhancement!

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.