Use mount flags in NodeStageVolume

[bertinatto]

/kind bug

What happened?

Mount flags are passed to both NodeStageVolume and NodePublishVolume, however only the latter use them.

How to reproduce it (as minimally and precisely as possible)?

1. Create a StorageClass with mountOptions
2. Create PVC using the StorageClass above
3. Create a pod that uses the PVC above
4. Check the mounts in the worker node (mount command)

[leakingtapan]

@bertinatto I understand NoeStageVolume is called for formatting the filesystem and mount it onto a global path. NodePublishVolume is used for mounting the volume from global path to container with the mount option specified. Why mount option needs to be applied to the NoeStageVolume call?

[bertinatto]

I understand NoeStageVolume is called for formatting the filesystem and mount it onto a global path. NodePublishVolume is used for mounting the volume from global path to container with the mount option specified.

Initially, that was my understanding as well. The mount flags were not being passed in to the NodeStageVolume call. However, this PR changed that: kubernetes/kubernetes#80191.

The GCEPD driver is already using the flags in both calls.

[leakingtapan]

Tested dynamic provisioning with ro mount option, the volume is not mounted as ro ...

[jsafrane]

Tested dynamic provisioning with ro mount option, the volume is not mounted as ro ...

I cannot confirm this with today's amazon/aws-ebs-csi-driver:latest. With these objects:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ro
mountOptions:
- ro
provisioner: ebs.csi.aws.com
reclaimPolicy: Delete
volumeBindingMode: Immediate

---

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myclaim
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 500Mi
  storageClassName: ro

---
apiVersion: v1
kind: Pod
metadata:
  name: testpod
  labels:
    name: test
spec:
  restartPolicy: Never
  terminationGracePeriodSeconds: 1
  tolerations:
    - key: node.kubernetes.io/network-unavailable
  containers:
    - image: gcr.io/google_containers/busybox
      command:
        - "/bin/sh"
        - "-c"
        - "while true; do date; sleep 1; done"
      name: busybox
      volumeMounts:
        - name: vol
          mountPath: /mnt/test
  volumes:
      - name: vol
        persistentVolumeClaim:
          claimName: myclaim

The volume is mounted as rw:

$ kubectl exec -ti testpod  mount | grep /mnt/test
/dev/xvdba on /mnt/test type ext4 (rw,seclabel,relatime)

From driver logs:

I1025 12:43:46.571344       1 node.go:93] NodeStageVolume: called with args {VolumeId:vol-00aec0d30b77438ef PublishContext:map[devicePath:/dev/xvdba] StagingTargetPath:/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/globalmount VolumeCapability:mount:<fs_type:"ext4" mount_flags:"ro" > access_mode:<mode:SINGLE_NODE_WRITER >  Secrets:map[] VolumeContext:map[storage.kubernetes.io/csiProvisionerIdentity:1572000744453-8081-ebs.csi.aws.com] XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
[...]
I1025 12:43:46.756065       1 mount_linux.go:493] Disk successfully formatted (mkfs): ext4 - /dev/xvdba /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/globalmount
I1025 12:43:46.756101       1 mount_linux.go:142] Mounting cmd (mount) with arguments ([-t ext4 -o defaults /dev/xvdba /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/globalmount])
I1025 12:43:46.766451       1 node.go:134] NodeStageVolume: volume="vol-00aec0d30b77438ef" operation finished

[...]

I1025 12:43:46.980489       1 node.go:269] NodePublishVolume: called with args {VolumeId:vol-00aec0d30b77438ef PublishContext:map[devicePath:/dev/xvdba] StagingTargetPath:/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/globalmount TargetPath:/var/lib/kubelet/pods/dfd2815f-93a4-4d70-a4a1-30f4edc6e89d/volumes/kubernetes.io~csi/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/mount VolumeCapability:mount:<fs_type:"ext4" mount_flags:"ro" > access_mode:<mode:SINGLE_NODE_WRITER >  Readonly:false Secrets:map[] VolumeContext:map[storage.kubernetes.io/csiProvisionerIdentity:1572000744453-8081-ebs.csi.aws.com] XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0}
I1025 12:43:46.980580       1 node.go:439] NodePublishVolume: creating dir /var/lib/kubelet/pods/dfd2815f-93a4-4d70-a4a1-30f4edc6e89d/volumes/kubernetes.io~csi/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/mount
I1025 12:43:46.980612       1 node.go:449] NodePublishVolume: mounting /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/globalmount at /var/lib/kubelet/pods/dfd2815f-93a4-4d70-a4a1-30f4edc6e89d/volumes/kubernetes.io~csi/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/mount with option [bind ro] as fstype ext4
I1025 12:43:46.980632       1 mount_linux.go:142] Mounting cmd (mount) with arguments ([-t ext4 -o bind /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/globalmount /var/lib/kubelet/pods/dfd2815f-93a4-4d70-a4a1-30f4edc6e89d/volumes/kubernetes.io~csi/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/mount])
I1025 12:43:46.982952       1 mount_linux.go:142] Mounting cmd (mount) with arguments ([-t ext4 -o bind,remount,ro /var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/globalmount /var/lib/kubelet/pods/dfd2815f-93a4-4d70-a4a1-30f4edc6e89d/volumes/kubernetes.io~csi/pvc-4924879e-8fe8-459e-a6b5-a2436e8a8756/mount])

IMO, you should pass mount options during NodeStage too. Passing it to bind-mount in NodePublish may work for ro, which is supported for bind mounts, however, other options will be silently ignored.