Skip to content

Commit

Permalink
Add test plan from design doc to KEP for CSI storage migration
Browse files Browse the repository at this point in the history
  • Loading branch information
davidz627 committed Jan 23, 2020
1 parent 04ea4ee commit cdf166e
Showing 1 changed file with 55 additions and 0 deletions.
55 changes: 55 additions & 0 deletions keps/sig-storage/20190129-csi-migration.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,10 @@ see-also:
- [Graduation Criteria](#graduation-criteria)
- [Alpha -> Beta](#alpha---beta)
- [Beta -> GA](#beta---ga)
- [Test Plan](#test-plan)
- [Per-driver migration testing](#per-driver-migration-testing)
- [Upgrade/Downgrade/Skew Testing](#upgradedowngradeskew-testing)
- [CSI Driver Feature Parity Testing](#csi-driver-feature-parity-testing)
- [Implementation History](#implementation-history)
<!-- /toc -->

Expand Down Expand Up @@ -123,6 +127,57 @@ The detailed design was originally implemented as a [design proposal](https://gi

* All volume operation paths covered by Migration Shim in Beta for >= 1 quarter without significant issues

## Test Plan

### Per-driver migration testing

We will require each plugin/driver provider to set up public CI to run all existing
in-tree plugin driver tests for their migrated driver.

If migration is on for that plugin, we will add checks in the test framework
that inspects CSI Drivers metrics to make sure that the driver is servicing the
operations. We must also observe through metrics that none of the in-tree code
is being called Additionally, we must test that a PV created from migrated
dynamic provisioning is identical to the PV created from the in-tree plugin.

The above is done by checking that no in-tree plugin code is emitting metrics
when migration is on. We will also confirm that metrics are being emitted in
general by confirming the existence of an indicator metric.

This should cover all use cases of volume operations, including volume
reconstruction.

### Upgrade/Downgrade/Skew Testing

We will have test clusters brought up that have different feature flags
enabled on different components (ADC and Kubelet). Once these feature flag skew
configurations are brought up the test itself would have to know what
configuration it’s running in and validate the expected result.

Configurations to test:

| ADC | Kubelet | Expected Result |
|-------------------|----------------------------------------------------|--------------------------------------------------------------------------|
| ADC Migration On | Kubelet Migration On | Fully migrated - result should be same as “Migration Shim Testing” above |
| ADC Migration On | Kubelet Migration Off (or Kubelet version too low) | No calls made to driver. All operations serviced by in-tree plugin |
| ADC Migration Off | Kubelet Migration On | Not supported config - Undefined behavior |
| ADC Migration Off | Kubelet Migration Off | No calls made to driver. All operations service by in-tree plugin |

Additionally, we will craft a test where a cluster should be able to run through
all plugin tests, do a complete upgrade to a version with CSI Migration turned
on, then run through all the plugin tests again and verify that there is no
issue.

### CSI Driver Feature Parity Testing

We will need some way to automatically qualify drivers have feature parity
before promoting their migration features to Beta (on by default).

This is as simple as on the feature flags and run through our “Migration Shim
Testing” tests. If the driver passes all of them then they have parity. If not,
we need to revisit in-tree plugin tests and make sure they test the entire suite
of possible tests.

## Implementation History

Major milestones in the life cycle of a KEP should be tracked in `Implementation History`.
Expand Down

0 comments on commit cdf166e

Please sign in to comment.