-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: v.oleynikov <[email protected]>
- Loading branch information
Showing
9 changed files
with
494 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -21,6 +21,7 @@ git: | |
- .helmignore | ||
- charts | ||
- crds | ||
- docs | ||
- enabled | ||
- hooks | ||
- monitoring | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
title: "The sds-local-volume module: configuration" | ||
force_searchable: true | ||
description: The sds-local-volume Deckhouse module's configuration. | ||
--- |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,5 @@ | ||
--- | ||
title: "Модуль sds-local-volume: настройки" | ||
force_searchable: true | ||
description: Параметры настройки модуля sds-local-volume Deckhouse. | ||
--- |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
--- | ||
title: "The sds-local-volume module: Custom Resources" | ||
description: "The sds-local-volume module Custom Resources: LocalStorageClass." | ||
--- |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
--- | ||
title: "Модуль sds-local-volume: Custom Resources" | ||
description: "Модуль sds-local-volume Custom Resources: LocalStorageClass." | ||
--- |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
--- | ||
title: "The sds-local-volume module: FAQ" | ||
description: "The sds-local-volume module: FAQ" | ||
--- | ||
|
||
## What is difference between LVM and LVMThin? | ||
|
||
- LVM is simpler and has high performance that is similar to that of native disk drives, but it does not support snapshots; | ||
- LVMThin allows for snapshots and overprovisioning; however, it is slower than LVM. | ||
|
||
## How do I set the default StorageClass? | ||
|
||
Set the `spec.IsDefault` field to `true` in the corresponding [LocalStorageClass](./cr.html#localstorageclass) custom resource. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
--- | ||
title: "Модуль sds-local-volume: FAQ" | ||
description: "Модуль sds-local-volume: FAQ" | ||
--- | ||
|
||
## Когда следует использовать LVM, а когда LVMThin? | ||
|
||
- LVM проще и обладает высокой производительностью, сравнимой с производительностью накопителя, но не позволяет использовать snapshot'ы; | ||
- LVMThin позволяет использовать snapshot'ы и overprovisioning, но производительность ниже, чем у LVM. | ||
|
||
## Как назначить StorageClass по умолчанию? | ||
|
||
В соответствующем пользовательском ресурсе [LocalStorageClass](./cr.html#localstorageclass) в поле `spec.isDefault` указать `true`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,223 @@ | ||
--- | ||
title: "The sds-local-volume module" | ||
description: "The sds-local-volume module: General Concepts and Principles." | ||
moduleStatus: experimental | ||
--- | ||
|
||
This module manages local block storage based on `LVM`. The module allows you to create a `StorageClass` in `Kubernetes` by creating [Kubernetes custom resources](./cr.html). | ||
To create a `Storage Class`, you will need the `LVMVolumeGroup` configured on the cluster nodes. The `LVM` configuration is done by the [sds-node-configurator](../../sds-node-configurator/) module. | ||
> **Caution!** Before enabling the `sds-local-volume` module, you must enable the `sds-node-configurator` module. | ||
> | ||
After you enable the `sds-local-volume` module in the Deckhouse configuration, you have to create StorageClasses. | ||
|
||
> **Caution!** The user is not allowed to create a `StorageClass` for the local.csi.storage.deckhouse.io CSI driver. | ||
Two modes are supported: LVM and LVMThin. | ||
Each mode has its advantages and disadvantages. Read [FAQ](./faq.html#what-is-difference-between-lvm-and-lvmthin) to learn more and compare them. | ||
|
||
## Quickstart guide | ||
|
||
Note that all commands must be run on a machine that has administrator access to the Kubernetes API. | ||
|
||
### Enabling modules | ||
|
||
- Enable the sds-node-configurator module | ||
|
||
```yaml | ||
kubectl apply -f - <<EOF | ||
apiVersion: deckhouse.io/v1alpha1 | ||
kind: ModuleConfig | ||
metadata: | ||
name: sds-node-configurator | ||
spec: | ||
enabled: true | ||
version: 1 | ||
EOF | ||
|
||
``` | ||
|
||
- Wait for it to become `Ready`. At this stage, you do NOT need to check the pods in the `d8-sds-node-configurator` namespace. | ||
|
||
```shell | ||
kubectl get mc sds-node-configurator -w | ||
``` | ||
|
||
- Enable the `sds-local-volume` module. Refer to the [configuration](./configuration.html) to learn more about module settings. In the example below, the module is launched with the default settings. This will result in the following actions across all cluster nodes: | ||
- launch of service pods for the `sds-local-volume` components. | ||
|
||
```yaml | ||
kubectl apply -f - <<EOF | ||
apiVersion: deckhouse.io/v1alpha1 | ||
kind: ModuleConfig | ||
metadata: | ||
name: sds-local-volume | ||
spec: | ||
enabled: true | ||
version: 1 | ||
EOF | ||
``` | ||
|
||
- Wait for the module to become `Ready`. | ||
|
||
```shell | ||
kubectl get mc sds-local-volume -w | ||
``` | ||
|
||
- Make sure that all pods in `d8-sds-local-volume` and `d8-sds-node-configurator` namespaces are `Running` or `Completed` and are running on all nodes where `LVM` resources are intended to be used. | ||
|
||
```shell | ||
kubectl -n d8-sds-local-volume get pod -owide -w | ||
kubectl -n d8-sds-node-configurator get pod -o wide -w | ||
``` | ||
|
||
### Configuring storage on nodes | ||
|
||
You need to create `LVM` volume groups on the nodes using `LVMVolumeGroup` custom resources. As part of this quickstart guide, we will create a regular `Thin` storage. | ||
|
||
To configure the storage: | ||
|
||
- List all the [BlockDevice](../../sds-node-configurator/stable/cr.html#blockdevice) resources available in your cluster: | ||
|
||
```shell | ||
kubectl get bd | ||
|
||
NAME NODE CONSUMABLE SIZE PATH | ||
dev-ef4fb06b63d2c05fb6ee83008b55e486aa1161aa worker-0 false 976762584Ki /dev/nvme1n1 | ||
dev-0cfc0d07f353598e329d34f3821bed992c1ffbcd worker-0 false 894006140416 /dev/nvme0n1p6 | ||
dev-7e4df1ddf2a1b05a79f9481cdf56d29891a9f9d0 worker-1 false 976762584Ki /dev/nvme1n1 | ||
dev-b103062f879a2349a9c5f054e0366594568de68d worker-1 false 894006140416 /dev/nvme0n1p6 | ||
dev-53d904f18b912187ac82de29af06a34d9ae23199 worker-2 false 976762584Ki /dev/nvme1n1 | ||
dev-6c5abbd549100834c6b1668c8f89fb97872ee2b1 worker-2 false 894006140416 /dev/nvme0n1p6 | ||
``` | ||
|
||
|
||
- Create an [LVMVolumeGroup](../../sds-node-configurator/stable/cr.html#lvmvolumegroup) resource for `worker-0`: | ||
|
||
```yaml | ||
kubectl apply -f - <<EOF | ||
apiVersion: storage.deckhouse.io/v1alpha1 | ||
kind: LvmVolumeGroup | ||
metadata: | ||
name: "vg-1-on-worker-0" # The name can be any fully qualified resource name in Kubernetes. This LvmVolumeGroup resource name will be used to create ReplicatedStoragePool in the future | ||
spec: | ||
type: Local | ||
blockDeviceNames: # specify the names of the BlockDevice resources that are located on the target node and whose CONSUMABLE is set to true. Note that the node name is not specified anywhere since it is derived from BlockDevice resources. | ||
- dev-ef4fb06b63d2c05fb6ee83008b55e486aa1161aa | ||
- dev-0cfc0d07f353598e329d34f3821bed992c1ffbcd | ||
thinPools: | ||
- name: ssd-thin | ||
size: 50Gi | ||
actualVGNameOnTheNode: "vg-1" # the name of the LVM VG to be created from the above block devices on the node | ||
EOF | ||
``` | ||
|
||
- Wait for the created `LVMVolumeGroup` resource to become `Operational`: | ||
|
||
```shell | ||
kubectl get lvg vg-1-on-worker-0 -w | ||
``` | ||
|
||
- The resource becoming `Operational` means that an LVM VG named `vg-1` made up of the `/dev/nvme1n1` and `/dev/nvme0n1p6` block devices has been created on the `worker-0` node. | ||
|
||
- Next, create an [LVMVolumeGroup](../../sds-node-configurator/stable/cr.html#lvmvolumegroup) resource for `worker-1`: | ||
|
||
```yaml | ||
kubectl apply -f - <<EOF | ||
apiVersion: storage.deckhouse.io/v1alpha1 | ||
kind: LvmVolumeGroup | ||
metadata: | ||
name: "vg-1-on-worker-1" | ||
spec: | ||
type: Local | ||
blockDeviceNames: | ||
- dev-7e4df1ddf2a1b05a79f9481cdf56d29891a9f9d0 | ||
- dev-b103062f879a2349a9c5f054e0366594568de68d | ||
thinPools: | ||
- name: ssd-thin | ||
size: 50Gi | ||
actualVGNameOnTheNode: "vg-1" | ||
EOF | ||
``` | ||
|
||
- Wait for the created `LVMVolumeGroup` resource to become `Operational`: | ||
|
||
```shell | ||
kubectl get lvg vg-1-on-worker-1 -w | ||
``` | ||
|
||
- The resource becoming `Operational` means that an LVM VG named `vg-1` made up of the `/dev/nvme1n1` and `/dev/nvme0n1p6` block device has been created on the `worker-1` node. | ||
|
||
- Create an [LVMVolumeGroup](../../sds-node-configurator/stable/cr.html#lvmvolumegroup) resource for `worker-2`: | ||
|
||
```yaml | ||
kubectl apply -f - <<EOF | ||
apiVersion: storage.deckhouse.io/v1alpha1 | ||
kind: LvmVolumeGroup | ||
metadata: | ||
name: "vg-1-on-worker-2" | ||
spec: | ||
type: Local | ||
blockDeviceNames: | ||
- dev-53d904f18b912187ac82de29af06a34d9ae23199 | ||
- dev-6c5abbd549100834c6b1668c8f89fb97872ee2b1 | ||
thinPools: | ||
- name: ssd-thin | ||
size: 50Gi | ||
actualVGNameOnTheNode: "vg-1" | ||
EOF | ||
``` | ||
|
||
- Wait for the created `LVMVolumeGroup` resource to become `Operational`: | ||
|
||
```shell | ||
kubectl get lvg vg-1-on-worker-2 -w | ||
``` | ||
|
||
- The resource becoming `Operational` means that an LVM VG named `vg-1` made up of the `/dev/nvme1n1` and `/dev/nvme0n1p6` block device has been created on the `worker-2` node. | ||
|
||
- Create a [LocalStorageClass](./cr.html#localstorageclass) resource: | ||
|
||
```yaml | ||
kubectl apply -f -<<EOF | ||
apiVersion: storage.deckhouse.io/v1alpha1 | ||
kind: LocalStorageClass | ||
metadata: | ||
name: local-storage-class | ||
spec: | ||
isDefault: false | ||
lvm: | ||
lvmVolumeGroups: | ||
- name: vg-1-on-worker-0 | ||
thin: | ||
poolName: ssd-thin | ||
- name: vg-1-on-worker-1 | ||
thin: | ||
poolName: ssd-thin | ||
- name: vg-1-on-worker-2 | ||
thin: | ||
poolName: ssd-thin | ||
type: Thin | ||
reclaimPolicy: Delete | ||
volumeBindingMode: WaitForFirstConsumer | ||
EOF | ||
``` | ||
|
||
- Wait for the created `LocalStorageClass` resource to become `Created`: | ||
|
||
```shell | ||
kubectl get lsc local-storage-class -w | ||
``` | ||
|
||
- Confirm that the corresponding `StorageClass` has been created: | ||
|
||
```shell | ||
kubectl get sc local-storage-class | ||
``` | ||
|
||
- If `StorageClass` with the name `local-storage-class` is shown, then the configuration of the `sds-local-volume` module is complete. Now users can create PVs by specifying `StorageClass` with the name `local-storage-class`. | ||
|
||
## System requirements and recommendations | ||
|
||
### Requirements | ||
- Stock kernels shipped with the [supported distributions](https://deckhouse.io/documentation/v1/supported_versions.html#linux). | ||
- Do not use another SDS (Software defined storage) to provide disks to our SDS. |
Oops, something went wrong.