change 'Important Notes'->'Important Notes on Secrets and Credentials'

storage/Ceph/Ceph-CSI-CephFS.md

@@ -162,7 +162,7 @@ spec:
   persistentVolumeReclaimPolicy: Retain
   volumeMode: Filesystem
 ```
-> **NOTE**: there are some potential gotchas involved with creating the Secret (e.g. `csi-cephfs-secret`, above). Please refer to [this section of Ceph-CSI.md](./Ceph-CSI.md#important-notes)
+> **NOTE**: there are some potential gotchas involved with creating the Secret (e.g. `csi-cephfs-secret`, above). Please refer to [Ceph CSI - Important Notes on Secrets and Credentials](./Ceph-CSI.md#important-notes-on-secrets-and-credentials)
 
 
 This PV is created with the command:
@@ -278,7 +278,7 @@ status 32) occurred while running mount args: [-t ceph 10.0.3.197:6789,10.0.3.20
 mds_namespace=cephfs,_netdev] stderr: mount error: no mds server is up or the cluster is laggy
 ```
 
-...the message `no mds server is up or the cluster is laggy` is potentially misleading. It is more likely that the `userID` is missing or incorrect, in your `secret.yaml` file. See [Ceph CSI - Important Notes](https://github.com/DataONEorg/k8s-cluster/blob/main/storage/Ceph/Ceph-CSI.md#important-notes). 
+...the message `no mds server is up or the cluster is laggy` is potentially misleading. It is more likely that the `userID` is missing or incorrect, in your `secret.yaml` file. See [Ceph CSI - Important Notes on Secrets and Credentials](./Ceph-CSI.md#important-notes-on-secrets-and-credentials).
 
 
 

storage/Ceph/Ceph-CSI.md

@@ -19,7 +19,7 @@ Command line options to helm supply most of the information that is needed for t
 
 Here is an example `csi-config-map.yaml` file:
 
-```yaml
+```json
 ---
 # This is a sample configmap that helps define a Ceph cluster configuration
 # as required by the CSI plugins.
@@ -173,13 +173,7 @@ Command line options to helm supply most of the information that is needed for t
 
 The `secret.yaml` file contains the ceph storage cluster login credentials needed for ceph-csi to mount CephFS subvolumes that are statically provisioned. These CephFS subvolumes must be created manually with the Linux `ceph` utility before they can be accessed by ceph-csi.
 
-The `userId` and `userKey` values provide the needed authorization for this. 
-
-### Important Notes
-1. ceph-generated usernames are typically of the form: `client.k8s-dev-releasename-subvol-user`. Note that you must omit the `client.` prefix when adding to the `secret.yaml` file (i.e. use only: `k8s-dev-myreleasename-subvol-user`).
-    * (However, when mounting the volume via `fstab`, the `client.` prefix should be retained for the keyring file.)
-1. The example [`secret.yaml`](https://github.com/DataONEorg/k8s-cluster/blob/main/storage/Ceph/CephFS/secret.yaml) file contains plaintext credentials (listed under `stringData:`), that are automatically base64-encoded at runtime. If you prefer to base64-encode the userID and userKey before adding to the `secret.yaml` file, be sure to use the `-n` option with the `echo` command, (i.e.: `echo -n k8s-dev-myreleasename-subvol-user | base64`), to suppress the trailing newline character. Failure to do so will cause authentication to fail (see also: [CephFS Troubleshooting](https://github.com/DataONEorg/k8s-cluster/blob/main/storage/Ceph/Ceph-CSI-CephFS.md#troubleshooting)). If they are already base64 encoded in this way, values should be added to the `secret.yaml` file under `data:` instead of `stringData:`.
-1. for dynamically provisioned (ceph-csi provisions them) CephFS volumes and subvolumes, the `adminId` and `adminKey` values are required.
+The `userId` and `userKey` values provide the needed authorization for this.
 
 Some of the ceph-csi functionality is only in Alpha release state, so is not ready for production use. Please refer to the [Ceph-CSI Support Matrix](https://github.com/ceph/ceph-csi#support-matrix) for more information.
 
@@ -341,3 +335,10 @@ reclaimPolicy: Delete
 volumeBindingMode: Immediate
 ```
 
+### Important Notes on Secrets and Credentials
+
+1. In the Ceph client file configurations, the userid will likely contain a prefix; for example: `client.k8s-dev-releasename-subvol-user`. Note that you must omit the `client.` prefix when adding to the `secret.yaml` file (i.e. use only: `k8s-dev-myreleasename-subvol-user`).
+    * (However, when mounting the volume via `fstab`, the `client.` prefix should be retained for the keyring file.)
+1. In the Ceph user configuration files, the userKey is already base64 encoded, but ***it needs to be base64-encoded again*** when the kubernetes Secret is created. Put the Ceph-provided base64 string in the `stringData.userKey` field, and it will automatically be base64-encoded again, upon creation.
+1. If you prefer to manually base64-encode the userID and userKey before adding to the `secret.yaml` file, be sure to use the `-n` option with the `echo` command, (i.e.: `echo -n k8s-dev-myreleasename-subvol-user | base64`), to suppress the trailing newline character. Failure to do so will cause authentication to fail (see also: [CephFS Troubleshooting](https://github.com/DataONEorg/k8s-cluster/blob/main/storage/Ceph/Ceph-CSI-CephFS.md#troubleshooting)). If they are already (double-)base64 encoded in this way, values should be added to the `secret.yaml` file under `data:` instead of `stringData:`.
+1. for dynamically provisioned CephFS volumes and subvolumes (ceph-csi provisions them), the `adminId` and `adminKey` values are required.