Skip to content

Commit

Permalink
docs: update Mayastor deployment process
Browse files Browse the repository at this point in the history
Update Mayastor docs with latest requirements

Signed-off-by: Noel Georgi <[email protected]>
  • Loading branch information
frezbo committed Dec 23, 2021
1 parent cff1ff6 commit 4d1514a
Show file tree
Hide file tree
Showing 2 changed files with 28 additions and 182 deletions.
105 changes: 14 additions & 91 deletions website/content/docs/v0.14/Guides/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ It is easy and automatic.

## Storage Clusters

> **Talos** recommends having a separate disks (apart from the Talos install disk) to be used for storage.
Redundancy in storage is usually very important.
Scaling capabilities, reliability, speed, maintenance load, and ease of use are all factors you must consider when managing your own storage.

Expand Down Expand Up @@ -78,112 +80,33 @@ To see a live demo of this section, see the video below:
### Prep Nodes

Either during initial cluster creation or on running worker nodes, several machine config values should be edited.
This can be done with `talosctl edit machineconfig` or via config patches during `talosctl gen config`.

- Under `/machine/sysctls`, add `vm.nr_hugepages: "512"`
- Under `/machine/kubelet/extraMounts`, add `/var/local` like so:

```yaml
...
extraMounts:
- destination: /var/local
type: bind
source: /var/local
options:
- rbind
- rshared
- rw
...
```
This information is gathered from the Mayastor [documentation](https://mayastor.gitbook.io/introduction/quickstart/preparing-the-cluster).
We need to set the `vm.nr_hugepages` sysctl and add `openebs.io/engine=mayastor` labels to the nodes which are meant to be storage nodes
This can be done with `talosctl patch machineconfig` or via config patches during `talosctl gen config`.

- Either using `kubectl taint node` in a pre-existing cluster or by updating `/machine/kubelet/extraArgs` in machine config, add `openebs.io/engine=mayastor` as a node label.
If being done via machine config, `extraArgs` may look like:
Some examples are shown below, modify as needed.

```yaml
...
extraArgs:
node-labels: openebs.io/engine=mayastor
...
```

### Deploy Mayastor

Using the [Mayastor docs](https://mayastor.gitbook.io/introduction/quickstart/deploy-mayastor) as a reference, apply all YAML files necessary.
At the time of writing this looked like:
Using gen config

```bash
kubectl create namespace mayastor

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/moac-rbac.yaml

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/nats-deployment.yaml

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/csi/moac/crds/mayastorpool.yaml

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/csi-daemonset.yaml

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/moac-deployment.yaml

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/mayastor-daemonset.yaml
talosctl gen config my-cluster https://mycluster.local:6443 --config-patch '[{"op": "add", "path": "/machine/sysctls", "value": {"vm.nr_hugepages": "1024"}}, {"op": "add", "path": "/machine/kubelet/extraArgs", "value": {"node-labels": "openebs.io/engine=mayastor"}}]'
```

### Create Pools

Each "storage" node should have a "MayastorPool" that defines the local disks to use for storage.
These are later considered during scheduling and replication of data.
Create the pool by issuing the following, updating as necessary:
Patching an existing node

```bash
cat <<EOF | kubectl create -f -
apiVersion: "openebs.io/v1alpha1"
kind: MayastorPool
metadata:
name: pool-on-talos-xxx
namespace: mayastor
spec:
node: talos-xxx
disks: ["/dev/sdx"]
EOF
talosctl patch --immediate machineconfig -n <node ip> --config-patch '[{"op": "add", "path": "/machine/sysctls", "value": {"vm.nr_hugepages": "1024"}}, {"op": "add", "path": "/machine/kubelet/extraArgs", "value": {"node-labels": "openebs.io/engine=mayastor"}}]'
```

### Create StorageClass

With the pools created for each node, create a storage class that uses the `nvmf` protocol, updating the number of replicas as necessary:
> Note: If you are adding/updating the `vm.nr_hugepages` on a node which already had the `openebs.io/engine=mayastor` label set, you'd need to restart kubelet so that it picks up the new value, by issuing the following command
```bash
cat <<EOF | kubectl create -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: mayastor-nvmf
parameters:
repl: '1'
protocol: 'nvmf'
provisioner: io.openebs.csi-mayastor
EOF
talosctl -n <node ip> service kubelet restart
```

### Consume Storage

The storage can now be consumed by creating a PersistentVolumeClaim (PVC) that references the StorageClass.
The PVC can then be used by a Pod or Deployment.
An example of creating a PersistentVolumeClaim may look like:
### Deploy Mayastor

```bash
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mayastor-volume-claim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: mayastor-nvmf
EOF
```
Continue setting up [Mayastor](https://mayastor.gitbook.io/introduction/quickstart/deploy-mayastor) using the official documentation.

## NFS

Expand Down
105 changes: 14 additions & 91 deletions website/content/docs/v0.15/Guides/storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ It is easy and automatic.

## Storage Clusters

> **Talos** recommends having a separate disks (apart from the Talos install disk) to be used for storage.
Redundancy in storage is usually very important.
Scaling capabilities, reliability, speed, maintenance load, and ease of use are all factors you must consider when managing your own storage.

Expand Down Expand Up @@ -78,112 +80,33 @@ To see a live demo of this section, see the video below:
### Prep Nodes

Either during initial cluster creation or on running worker nodes, several machine config values should be edited.
This can be done with `talosctl edit machineconfig` or via config patches during `talosctl gen config`.

- Under `/machine/sysctls`, add `vm.nr_hugepages: "512"`
- Under `/machine/kubelet/extraMounts`, add `/var/local` like so:

```yaml
...
extraMounts:
- destination: /var/local
type: bind
source: /var/local
options:
- rbind
- rshared
- rw
...
```
This information is gathered from the Mayastor [documentation](https://mayastor.gitbook.io/introduction/quickstart/preparing-the-cluster).
We need to set the `vm.nr_hugepages` sysctl and add `openebs.io/engine=mayastor` labels to the nodes which are meant to be storage nodes
This can be done with `talosctl patch machineconfig` or via config patches during `talosctl gen config`.

- Either using `kubectl taint node` in a pre-existing cluster or by updating `/machine/kubelet/extraArgs` in machine config, add `openebs.io/engine=mayastor` as a node label.
If being done via machine config, `extraArgs` may look like:
Some examples are shown below, modify as needed.

```yaml
...
extraArgs:
node-labels: openebs.io/engine=mayastor
...
```

### Deploy Mayastor

Using the [Mayastor docs](https://mayastor.gitbook.io/introduction/quickstart/deploy-mayastor) as a reference, apply all YAML files necessary.
At the time of writing this looked like:
Using gen config

```bash
kubectl create namespace mayastor

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/moac-rbac.yaml

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/nats-deployment.yaml

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/csi/moac/crds/mayastorpool.yaml

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/csi-daemonset.yaml

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/moac-deployment.yaml

kubectl apply -f https://raw.githubusercontent.com/openebs/Mayastor/master/deploy/mayastor-daemonset.yaml
talosctl gen config my-cluster https://mycluster.local:6443 --config-patch '[{"op": "add", "path": "/machine/sysctls", "value": {"vm.nr_hugepages": "1024"}}, {"op": "add", "path": "/machine/kubelet/extraArgs", "value": {"node-labels": "openebs.io/engine=mayastor"}}]'
```

### Create Pools

Each "storage" node should have a "MayastorPool" that defines the local disks to use for storage.
These are later considered during scheduling and replication of data.
Create the pool by issuing the following, updating as necessary:
Patching an existing node

```bash
cat <<EOF | kubectl create -f -
apiVersion: "openebs.io/v1alpha1"
kind: MayastorPool
metadata:
name: pool-on-talos-xxx
namespace: mayastor
spec:
node: talos-xxx
disks: ["/dev/sdx"]
EOF
talosctl patch --immediate machineconfig -n <node ip> --config-patch '[{"op": "add", "path": "/machine/sysctls", "value": {"vm.nr_hugepages": "1024"}}, {"op": "add", "path": "/machine/kubelet/extraArgs", "value": {"node-labels": "openebs.io/engine=mayastor"}}]'
```

### Create StorageClass

With the pools created for each node, create a storage class that uses the `nvmf` protocol, updating the number of replicas as necessary:
> Note: If you are adding/updating the `vm.nr_hugepages` on a node which already had the `openebs.io/engine=mayastor` label set, you'd need to restart kubelet so that it picks up the new value, by issuing the following command
```bash
cat <<EOF | kubectl create -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
name: mayastor-nvmf
parameters:
repl: '1'
protocol: 'nvmf'
provisioner: io.openebs.csi-mayastor
EOF
talosctl -n <node ip> service kubelet restart
```

### Consume Storage

The storage can now be consumed by creating a PersistentVolumeClaim (PVC) that references the StorageClass.
The PVC can then be used by a Pod or Deployment.
An example of creating a PersistentVolumeClaim may look like:
### Deploy Mayastor

```bash
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: mayastor-volume-claim
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
storageClassName: mayastor-nvmf
EOF
```
Continue setting up [Mayastor](https://mayastor.gitbook.io/introduction/quickstart/deploy-mayastor) using the official documentation.

## NFS

Expand Down

0 comments on commit 4d1514a

Please sign in to comment.