Create volumes in outpost for outpost instances

[ayberk]

Is this a bug fix or adding new feature?
Bug Fix

What is this PR about? / Why do we need it?
Outposts Documentation for background.

Currently, we're running oblivious to outposts. Volumes are created in the parent region instead of the outpost itself. In order to fix it, we need to pass the outpostArn to CreateVolume.

I'm leveraging the outpost-arn instance metadata to determine if we're in an outpost instance. There are two possible responses from the endpoint:

• The outpost-arn if it's an outpost instance
• 404 if it's a regular non-outpost instance.

I've verified the behavior using this test code on a test outpost.

What testing is done?
Tested on an actual outpost instance and verified it works.

[k8s-ci-robot]

Hi @ayberk. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[coveralls]

Pull Request Test Coverage Report for Build 1231

• 87 of 92 (94.57%) changed or added relevant lines in 4 files are covered.
• No unchanged relevant lines lost coverage.
• Overall coverage increased (+0.9%) to 81.134%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
pkg/driver/controller.go	46	48	95.83%
pkg/cloud/cloud.go	7	10	70.0%

Totals
Change from base Build 1225:	0.9%
Covered Lines:	1574
Relevant Lines:	1940

---

💛 - Coveralls

[wongma7]

/ok-to-test

[ayberk]

/retest

[ayberk]

/retest

[ayberk]

/retest

[wongma7]

lgtm but i think there is an edgecase where GetDiskByName returns an existing volume that needs tobe handled.
/hold
for end to end test with real outpost

[wongma7]

/lgtm
nice work!!!
Please post testing result, like the output of kubectl get pv -o yaml <name of some outpost pv created by the driver> and/or kubectl get csinodeinfo -o yaml <name of some outpost node where the csi driver is installed> for sanity check then let's unhold and merge.. If you create a cluster with only outpost nodes, it should be possible to reuse the e2e tests, if there is a mix of outpost with non outpost nodes then I guess it will be random whether the test pods get scheduled to the outpost nodes or not...

[ayberk]

I made minor changes and was able to run the dynamic provisioning example on a test outpost. Volume was attached properly.

ubuntu@ip-10-0-1-194:~/aws-ebs-csi-driver/examples/kubernetes/dynamic-provisioning$ kubectl describe pv pvc-ecbd15ab-0a37-4910-972d-2f9a1487a8e6
Name:              pvc-ecbd15ab-0a37-4910-972d-2f9a1487a8e6
Labels:            <none>
Annotations:       pv.kubernetes.io/provisioned-by: ebs.csi.aws.com
Finalizers:        [kubernetes.io/pv-protection external-attacher/ebs-csi-aws-com]
StorageClass:      ebs-sc
Status:            Bound
Claim:             default/ebs-claim
Reclaim Policy:    Delete
Access Modes:      RWO
VolumeMode:        Filesystem
Capacity:          4Gi
Node Affinity:
  Required Terms:
    Term 0:        topology.ebs.csi.aws.com/partition in [aws]
                   topology.ebs.csi.aws.com/account-id in [foo]
                   topology.ebs.csi.aws.com/outpost-id in [bar]
                   topology.ebs.csi.aws.com/zone in [us-east-1d]
                   topology.ebs.csi.aws.com/region in [us-east-1]
Message:
Source:
    Type:              CSI (a Container Storage Interface (CSI) volume source)
    Driver:            ebs.csi.aws.com
    FSType:            ext4
    VolumeHandle:      vol-0372dc90eb689430d
    ReadOnly:          false
    VolumeAttributes:      storage.kubernetes.io/csiProvisionerIdentity=1601421491105-8081-ebs.csi.aws.com
Events:                <none>

ubuntu@ip-10-0-1-194:~/aws-ebs-csi-driver/examples/kubernetes/dynamic-provisioning$ kubectl describe node ip-10-0-1-194
Name:               ip-10-0-1-194
Roles:              master
Labels:             
                    beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=amd64
                    kubernetes.io/hostname=ip-10-0-1-194
                    kubernetes.io/os=linux
                    node-role.kubernetes.io/master=
                    topology.ebs.csi.aws.com/account-id=foo
                    topology.ebs.csi.aws.com/outpost-id=bar
                    topology.ebs.csi.aws.com/partition=aws
                    topology.ebs.csi.aws.com/region=us-east-1
                    topology.ebs.csi.aws.com/zone=us-east-1d
Annotations:        csi.volume.kubernetes.io/nodeid: {"ebs.csi.aws.com":"i-0954d4e23b3bf2da0"}
                    kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
                    node.alpha.kubernetes.io/ttl: 0
                    projectcalico.org/IPv4Address: 10.0.1.194/24
                    projectcalico.org/IPv4VXLANTunnelAddr: 192.168.245.0
                    volumes.kubernetes.io/controller-managed-attach-detach: true

ubuntu@ip-10-0-1-194:~$ kubectl get csinode -o yaml ip-10-0-1-194
apiVersion: storage.k8s.io/v1
kind: CSINode
metadata:
  creationTimestamp: "2020-09-29T21:33:12Z"
  managedFields:
  - apiVersion: storage.k8s.io/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:ownerReferences:
          .: {}
          k:{"uid":"64ba5bc7-45f6-494f-8e71-af9575184612"}:
            .: {}
            f:apiVersion: {}
            f:kind: {}
            f:name: {}
            f:uid: {}
      f:spec:
        f:drivers:
          k:{"name":"ebs.csi.aws.com"}:
            .: {}
            f:allocatable:
              .: {}
              f:count: {}
            f:name: {}
            f:nodeID: {}
            f:topologyKeys: {}
    manager: kubelet
    operation: Update
    time: "2020-09-30T00:29:58Z"
  name: ip-10-0-1-194
  ownerReferences:
  - apiVersion: v1
    kind: Node
    name: ip-10-0-1-194
    uid: 64ba5bc7-45f6-494f-8e71-af9575184612
  resourceVersion: "29252"
  selfLink: /apis/storage.k8s.io/v1/csinodes/ip-10-0-1-194
  uid: 41e397d9-eb3a-4376-b60c-96a2b6c81ba2
spec:
  drivers:
  - allocatable:
      count: 25
    name: ebs.csi.aws.com
    nodeID: i-0954d4e23b3bf2da0
    topologyKeys:
    - topology.ebs.csi.aws.com/account-id
    - topology.ebs.csi.aws.com/outpost-id
    - topology.ebs.csi.aws.com/partition
    - topology.ebs.csi.aws.com/region
    - topology.ebs.csi.aws.com/zone

[ayberk]

/retest

[wongma7]

/lgtm
/approve

[wongma7]

/hold cancel

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ayberk, wongma7

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [wongma7]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[sandeep173254]

@ayberk I am trying to do a poc with outpost but every time the volume is created in the parent region (no OutpostARN). my ec2 instance metadata returns OutpostARN. Any pointers would be helpful

Error message that i get because my instance is in Outpost but the volume is not:

AttachVolume.Attach failed for volume "pvc-1155bee4-0186-4a43-8047-a30b33d3bcdb" : rpc error: code = Internal desc = Could not attach volume "vol-0a41058da77cf512a" to node "i-09027f5cfb7d1455f": could not attach volume "vol-0a41058da77cf512a" to node "i-09027f5cfb7d1455f": operation error EC2: AttachVolume, https response error