Skip to content

Latest commit

 

History

History
10404 lines (10404 loc) · 179 KB

api_doc.md

File metadata and controls

10404 lines (10404 loc) · 179 KB

API Reference

data.fluid.io/v1alpha1

Package v1alpha1 is the v1alpha1 version of the API.

Resource Types:

AlluxioRuntime

AlluxioRuntime is the Schema for the alluxioruntimes API

Field Description
apiVersion
string
data.fluid.io/v1alpha1
kind
string
AlluxioRuntime
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
AlluxioRuntimeSpec


alluxioVersion
VersionSpec

The version information that instructs fluid to orchestrate a particular version of Alluxio.

master
AlluxioCompTemplateSpec

The component spec of Alluxio master

jobMaster
AlluxioCompTemplateSpec

The component spec of Alluxio job master

worker
AlluxioCompTemplateSpec

The component spec of Alluxio worker

jobWorker
AlluxioCompTemplateSpec

The component spec of Alluxio job Worker

apiGateway
AlluxioCompTemplateSpec

The component spec of Alluxio API Gateway

initUsers
InitUsersSpec

The spec of init users

fuse
AlluxioFuseSpec

The component spec of Alluxio Fuse

properties
map[string]string

Configurable properties for Alluxio system.
Refer to Alluxio Configuration Properties for more info

jvmOptions
[]string

Options for JVM

tieredstore
TieredStore

Tiered storage used by Alluxio

data
Data

Management strategies for the dataset to which the runtime is bound

replicas
int32

The replicas of the worker, need to be specified

runAs
User

Manage the user to run Alluxio Runtime

disablePrometheus
bool
(Optional)

Disable monitoring for Alluxio Runtime Prometheus is enabled by default

hadoopConfig
string
(Optional)

Name of the configMap used to support HDFS configurations when using HDFS as Alluxio’s UFS. The configMap must be in the same namespace with the AlluxioRuntime. The configMap should contain user-specific HDFS conf files in it. For now, only “hdfs-site.xml” and “core-site.xml” are supported. It must take the filename of the conf file as the key and content of the file as the value.

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by the alluxio runtime components and/or fuses.

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to Alluxio’s pods

management
RuntimeManagement
(Optional)

RuntimeManagement defines policies when managing the runtime

status
RuntimeStatus

DataBackup

DataBackup is the Schema for the backup API

Field Description
apiVersion
string
data.fluid.io/v1alpha1
kind
string
DataBackup
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
DataBackupSpec


dataset
string

Dataset defines the target dataset of the DataBackup

backupPath
string

BackupPath defines the target path to save data of the DataBackup

runAs
User

Manage the user to run Alluxio DataBackup

runAfter
OperationRef
(Optional)

Specifies that the preceding operation in a workflow

ttlSecondsAfterFinished
int32
(Optional)

TTLSecondsAfterFinished is the time second to clean up data operations after finished or failed

status
OperationStatus

DataLoad

DataLoad is the Schema for the dataloads API

Field Description
apiVersion
string
data.fluid.io/v1alpha1
kind
string
DataLoad
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
DataLoadSpec


dataset
TargetDataset

Dataset defines the target dataset of the DataLoad

loadMetadata
bool

LoadMetadata specifies if the dataload job should load metadata

target
[]TargetPath

Target defines target paths that needs to be loaded

options
map[string]string

Options specifies the extra dataload properties for runtime

podMetadata
PodMetadata

PodMetadata defines labels and annotations that will be propagated to DataLoad pods

affinity
Kubernetes core/v1.Affinity
(Optional)

Affinity defines affinity for DataLoad pod

tolerations
[]Kubernetes core/v1.Toleration
(Optional)

Tolerations defines tolerations for DataLoad pod

nodeSelector
map[string]string
(Optional)

NodeSelector defiens node selector for DataLoad pod

schedulerName
string
(Optional)

SchedulerName sets the scheduler to be used for DataLoad pod

policy
Policy
(Optional)

including Once, Cron, OnEvent

schedule
string
(Optional)

The schedule in Cron format, only set when policy is cron, see https://en.wikipedia.org/wiki/Cron.

runAfter
OperationRef
(Optional)

Specifies that the preceding operation in a workflow

ttlSecondsAfterFinished
int32
(Optional)

TTLSecondsAfterFinished is the time second to clean up data operations after finished or failed

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by the DataLoad job.

status
OperationStatus

DataMigrate

DataMigrate is the Schema for the datamigrates API

Field Description
apiVersion
string
data.fluid.io/v1alpha1
kind
string
DataMigrate
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
DataMigrateSpec


VersionSpec
VersionSpec

(Members of VersionSpec are embedded into this type.)

(Optional)

The version information that instructs fluid to orchestrate a particular version for data migrate.

from
DataToMigrate

data to migrate source, including dataset and external storage

to
DataToMigrate

data to migrate destination, including dataset and external storage

block
bool
(Optional)

if dataMigrate blocked dataset usage, default is false

runtimeType
string
(Optional)

using which runtime to migrate data; if none, take dataset runtime as default

options
map[string]string
(Optional)

options for migrate, different for each runtime

policy
Policy
(Optional)

policy for migrate, including Once, Cron, OnEvent

schedule
string
(Optional)

The schedule in Cron format, only set when policy is cron, see https://en.wikipedia.org/wiki/Cron.

podMetadata
PodMetadata

PodMetadata defines labels and annotations that will be propagated to DataMigrate pods

affinity
Kubernetes core/v1.Affinity
(Optional)

Affinity defines affinity for DataMigrate pod

tolerations
[]Kubernetes core/v1.Toleration
(Optional)

Tolerations defines tolerations for DataMigrate pod

nodeSelector
map[string]string
(Optional)

NodeSelector defiens node selector for DataMigrate pod

schedulerName
string
(Optional)

SchedulerName sets the scheduler to be used for DataMigrate pod

runAfter
OperationRef
(Optional)

Specifies that the preceding operation in a workflow

ttlSecondsAfterFinished
int32
(Optional)

TTLSecondsAfterFinished is the time second to clean up data operations after finished or failed

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by the DataMigrate job.

parallelism
int32
(Optional)

Parallelism defines the parallelism tasks numbers for DataMigrate. If the value is greater than 1, the job acts as a launcher, and users should define the WorkerSpec.

parallelOptions
map[string]string
(Optional)

ParallelOptions defines options like ssh port and ssh secret name when parallelism is greater than 1.

status
OperationStatus

Dataset

Dataset is the Schema for the datasets API

Field Description
apiVersion
string
data.fluid.io/v1alpha1
kind
string
Dataset
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
DatasetSpec


mounts
[]Mount
(Optional)

Mount Points to be mounted on cache runtime.
This field can be empty because some runtimes don’t need to mount external storage (e.g. Vineyard).

owner
User
(Optional)

The owner of the dataset

nodeAffinity
CacheableNodeAffinity
(Optional)

NodeAffinity defines constraints that limit what nodes this dataset can be cached to. This field influences the scheduling of pods that use the cached dataset.

tolerations
[]Kubernetes core/v1.Toleration
(Optional)

If specified, the pod’s tolerations.

accessModes
[]Kubernetes core/v1.PersistentVolumeAccessMode
(Optional)

AccessModes contains all ways the volume backing the PVC can be mounted

runtimes
[]Runtime

Runtimes for supporting dataset (e.g. AlluxioRuntime)

placement
PlacementMode
(Optional)

Manage switch for opening Multiple datasets single node deployment or not TODO(xieydd) In future, evaluate node resources and runtime resources to decide whether to turn them on

dataRestoreLocation
DataRestoreLocation
(Optional)

DataRestoreLocation is the location to load data of dataset been backuped

sharedOptions
map[string]string
(Optional)

SharedOptions is the options to all mount

sharedEncryptOptions
[]EncryptOption
(Optional)

SharedEncryptOptions is the encryptOption to all mount

status
DatasetStatus

EFCRuntime

EFCRuntime is the Schema for the efcruntimes API

Field Description
apiVersion
string
data.fluid.io/v1alpha1
kind
string
EFCRuntime
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
EFCRuntimeSpec


master
EFCCompTemplateSpec

The component spec of EFC master

worker
EFCCompTemplateSpec

The component spec of EFC worker

initFuse
InitFuseSpec

The spec of init alifuse

fuse
EFCFuseSpec

The component spec of EFC Fuse

tieredstore
TieredStore

Tiered storage used by EFC worker

replicas
int32

The replicas of the worker, need to be specified

osAdvise
OSAdvise

Operating system optimization for EFC

cleanCachePolicy
CleanCachePolicy
(Optional)

CleanCachePolicy defines cleanCache Policy

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to all EFC’s pods

status
RuntimeStatus

GooseFSRuntime

GooseFSRuntime is the Schema for the goosefsruntimes API

Field Description
apiVersion
string
data.fluid.io/v1alpha1
kind
string
GooseFSRuntime
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
GooseFSRuntimeSpec


goosefsVersion
VersionSpec

The version information that instructs fluid to orchestrate a particular version of GooseFS.

master
GooseFSCompTemplateSpec

The component spec of GooseFS master

jobMaster
GooseFSCompTemplateSpec

The component spec of GooseFS job master

worker
GooseFSCompTemplateSpec

The component spec of GooseFS worker

jobWorker
GooseFSCompTemplateSpec

The component spec of GooseFS job Worker

apiGateway
GooseFSCompTemplateSpec

The component spec of GooseFS API Gateway

initUsers
InitUsersSpec

The spec of init users

fuse
GooseFSFuseSpec

The component spec of GooseFS Fuse

properties
map[string]string

Configurable properties for the GOOSEFS component.
Refer to GOOSEFS Configuration Properties for more info

jvmOptions
[]string

Options for JVM

tieredstore
TieredStore

Tiered storage used by GooseFS

data
Data

Management strategies for the dataset to which the runtime is bound

replicas
int32

The replicas of the worker, need to be specified

runAs
User

Manage the user to run GooseFS Runtime GooseFS support POSIX-ACL and Apache Ranger to manager authorization TODO([email protected]) Support Apache Ranger.

disablePrometheus
bool
(Optional)

Disable monitoring for GooseFS Runtime Prometheus is enabled by default

hadoopConfig
string
(Optional)

Name of the configMap used to support HDFS configurations when using HDFS as GooseFS’s UFS. The configMap must be in the same namespace with the GooseFSRuntime. The configMap should contain user-specific HDFS conf files in it. For now, only “hdfs-site.xml” and “core-site.xml” are supported. It must take the filename of the conf file as the key and content of the file as the value.

cleanCachePolicy
CleanCachePolicy
(Optional)

CleanCachePolicy defines cleanCache Policy

status
RuntimeStatus

JindoRuntime

JindoRuntime is the Schema for the jindoruntimes API

Field Description
apiVersion
string
data.fluid.io/v1alpha1
kind
string
JindoRuntime
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
JindoRuntimeSpec


jindoVersion
VersionSpec

The version information that instructs fluid to orchestrate a particular version of Jindo.

master
JindoCompTemplateSpec

The component spec of Jindo master

worker
JindoCompTemplateSpec

The component spec of Jindo worker

fuse
JindoFuseSpec

The component spec of Jindo Fuse

properties
map[string]string

Configurable properties for Jindo system.

tieredstore
TieredStore

Tiered storage used by Jindo

replicas
int32

The replicas of the worker, need to be specified

runAs
User

Manage the user to run Jindo Runtime

user
string
hadoopConfig
string
(Optional)

Name of the configMap used to support HDFS configurations when using HDFS as Jindo’s UFS. The configMap must be in the same namespace with the JindoRuntime. The configMap should contain user-specific HDFS conf files in it. For now, only “hdfs-site.xml” and “core-site.xml” are supported. It must take the filename of the conf file as the key and content of the file as the value.

secret
string
labels
map[string]string
(Optional)

Labels will be added on all the JindoFS pods. DEPRECATED: this is a deprecated field. Please use PodMetadata.Labels instead. Note: this field is set to be exclusive with PodMetadata.Labels

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to all Jindo’s fuse pods

logConfig
map[string]string
(Optional)
networkmode
NetworkMode
(Optional)

Whether to use hostnetwork or not

cleanCachePolicy
CleanCachePolicy
(Optional)

CleanCachePolicy defines cleanCache Policy

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by the jindo runtime components and/or fuses.

status
RuntimeStatus

JuiceFSRuntime

JuiceFSRuntime is the Schema for the juicefsruntimes API

Field Description
apiVersion
string
data.fluid.io/v1alpha1
kind
string
JuiceFSRuntime
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
JuiceFSRuntimeSpec


juicefsVersion
VersionSpec

The version information that instructs fluid to orchestrate a particular version of JuiceFS.

initUsers
InitUsersSpec

The spec of init users

master
JuiceFSCompTemplateSpec

The component spec of JuiceFS master

worker
JuiceFSCompTemplateSpec

The component spec of JuiceFS worker

jobWorker
JuiceFSCompTemplateSpec

The component spec of JuiceFS job Worker

fuse
JuiceFSFuseSpec

Desired state for JuiceFS Fuse

tieredstore
TieredStore

Tiered storage used by JuiceFS

configs
string

Configs of JuiceFS

replicas
int32

The replicas of the worker, need to be specified

runAs
User

Manage the user to run Juicefs Runtime

disablePrometheus
bool
(Optional)

Disable monitoring for JuiceFS Runtime Prometheus is enabled by default

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by the alluxio runtime components and/or fuses.

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to JuiceFs’s pods.

cleanCachePolicy
CleanCachePolicy
(Optional)

CleanCachePolicy defines cleanCache Policy

status
RuntimeStatus

VineyardRuntime

VineyardRuntime is the Schema for the VineyardRuntimes API

Field Description
apiVersion
string
data.fluid.io/v1alpha1
kind
string
VineyardRuntime
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
VineyardRuntimeSpec


master
MasterSpec
(Optional)

Master holds the configurations for Vineyard Master component Represents the Etcd component in Vineyard

worker
VineyardCompTemplateSpec
(Optional)

Worker holds the configurations for Vineyard Worker component Represents the Vineyardd component in Vineyard

replicas
int32

The replicas of the worker, need to be specified If worker.replicas and the field are both specified, the field will be respected

fuse
VineyardSockSpec
(Optional)

Fuse holds the configurations for Vineyard client socket. Note that the “Fuse” here is kept just for API consistency, VineyardRuntime mount a socket file instead of a FUSE filesystem to make data cache available. Applications can connect to the vineyard runtime components through IPC or RPC. IPC is the default way to connect to vineyard runtime components, which is more efficient than RPC. If the socket file is not mounted, the connection will fall back to RPC.

tieredstore
TieredStore
(Optional)

Tiered storage used by vineyardd The MediumType can only be MEM and SSD MEM actually represents the shared memory of vineyardd. SSD represents the external storage of vineyardd. Default is as follows. tieredstore: levels: - level: 0 mediumtype: MEM quota: 4Gi

Choose hostpath as the external storage of vineyardd. tieredstore: levels: - level: 0 mediumtype: MEM quota: 4Gi high: “0.8” low: “0.3” - level: 1 mediumtype: SSD quota: 10Gi volumeType: Hostpath path: /var/spill-path

disablePrometheus
bool
(Optional)

Disable monitoring metrics for Vineyard Runtime Default is false

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to Vineyard’s pods.

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by the vineyard components (Master and Worker). Default is null.

status
RuntimeStatus

APIGatewayStatus

(Appears on: RuntimeStatus)

API Gateway

Field Description
endpoint
string

Endpoint for accessing

AlluxioCompTemplateSpec

(Appears on: AlluxioRuntimeSpec)

AlluxioCompTemplateSpec is a description of the Alluxio commponents

Field Description
replicas
int32
(Optional)

Replicas is the desired number of replicas of the given template. If unspecified, defaults to 1. replicas is the min replicas of dataset in the cluster

jvmOptions
[]string

Options for JVM

properties
map[string]string
(Optional)

Configurable properties for the Alluxio component.
Refer to Alluxio Configuration Properties for more info

ports
map[string]int
(Optional)

Ports used by Alluxio(e.g. rpc: 19998 for master)

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by the Alluxio component.

Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.

env
map[string]string

Environment variables that will be used by Alluxio component.

enabled
bool
(Optional)

Enabled or Disabled for the components. For now, only API Gateway is enabled or disabled.

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector which must be true for the master to fit on a node

networkMode
NetworkMode
(Optional)

Whether to use hostnetwork or not

volumeMounts
[]Kubernetes core/v1.VolumeMount
(Optional)

VolumeMounts specifies the volumes listed in “.spec.volumes” to mount into the alluxio runtime component’s filesystem.

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to Alluxio’s pods

AlluxioFuseSpec

(Appears on: AlluxioRuntimeSpec)

AlluxioFuseSpec is a description of the Alluxio Fuse

Field Description
image
string

Image for Alluxio Fuse(e.g. alluxio/alluxio-fuse)

imageTag
string

Image Tag for Alluxio Fuse(e.g. 2.3.0-SNAPSHOT)

imagePullPolicy
string

One of the three policies: Always, IfNotPresent, Never

jvmOptions
[]string

Options for JVM

properties
map[string]string

Configurable properties for Alluxio System.
Refer to Alluxio Configuration Properties for more info

env
map[string]string

Environment variables that will be used by Alluxio Fuse

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by Alluxio Fuse.

Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.

args
[]string

Arguments that will be passed to Alluxio Fuse

global
bool
(Optional)

If the fuse client should be deployed in global mode, otherwise the affinity should be considered

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector which must be true for the fuse client to fit on a node, this option only effect when global is enabled

cleanPolicy
FuseCleanPolicy
(Optional)

CleanPolicy decides when to clean Alluxio Fuse pods. Currently Fluid supports two policies: OnDemand and OnRuntimeDeleted OnDemand cleans fuse pod once the fuse pod on some node is not needed OnRuntimeDeleted cleans fuse pod only when the cache runtime is deleted Defaults to OnRuntimeDeleted

networkMode
NetworkMode
(Optional)

Whether to use hostnetwork or not

volumeMounts
[]Kubernetes core/v1.VolumeMount
(Optional)

VolumeMounts specifies the volumes listed in “.spec.volumes” to mount into the alluxio runtime component’s filesystem.

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to Alluxio’s fuse pods

AlluxioRuntimeRole (string alias)

AlluxioRuntimeSpec

(Appears on: AlluxioRuntime)

AlluxioRuntimeSpec defines the desired state of AlluxioRuntime

Field Description
alluxioVersion
VersionSpec

The version information that instructs fluid to orchestrate a particular version of Alluxio.

master
AlluxioCompTemplateSpec

The component spec of Alluxio master

jobMaster
AlluxioCompTemplateSpec

The component spec of Alluxio job master

worker
AlluxioCompTemplateSpec

The component spec of Alluxio worker

jobWorker
AlluxioCompTemplateSpec

The component spec of Alluxio job Worker

apiGateway
AlluxioCompTemplateSpec

The component spec of Alluxio API Gateway

initUsers
InitUsersSpec

The spec of init users

fuse
AlluxioFuseSpec

The component spec of Alluxio Fuse

properties
map[string]string

Configurable properties for Alluxio system.
Refer to Alluxio Configuration Properties for more info

jvmOptions
[]string

Options for JVM

tieredstore
TieredStore

Tiered storage used by Alluxio

data
Data

Management strategies for the dataset to which the runtime is bound

replicas
int32

The replicas of the worker, need to be specified

runAs
User

Manage the user to run Alluxio Runtime

disablePrometheus
bool
(Optional)

Disable monitoring for Alluxio Runtime Prometheus is enabled by default

hadoopConfig
string
(Optional)

Name of the configMap used to support HDFS configurations when using HDFS as Alluxio’s UFS. The configMap must be in the same namespace with the AlluxioRuntime. The configMap should contain user-specific HDFS conf files in it. For now, only “hdfs-site.xml” and “core-site.xml” are supported. It must take the filename of the conf file as the key and content of the file as the value.

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by the alluxio runtime components and/or fuses.

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to Alluxio’s pods

management
RuntimeManagement
(Optional)

RuntimeManagement defines policies when managing the runtime

CacheableNodeAffinity

(Appears on: DatasetSpec)

CacheableNodeAffinity defines constraints that limit what nodes this dataset can be cached to.

Field Description
required
Kubernetes core/v1.NodeSelector

Required specifies hard node constraints that must be met.

CleanCachePolicy

(Appears on: EFCRuntimeSpec, GooseFSRuntimeSpec, JindoRuntimeSpec, JuiceFSRuntimeSpec, RuntimeManagement)

CleanCachePolicy defines policies when cleaning cache

Field Description
gracePeriodSeconds
int32
(Optional)

Optional duration in seconds the cache needs to clean gracefully. May be decreased in delete runtime request. Value must be non-negative integer. The value zero indicates clean immediately via the timeout command (no opportunity to shut down). If this value is nil, the default grace period will be used instead. The grace period is the duration in seconds after the processes running in the pod are sent a termination signal and the time when the processes are forcibly halted with timeout command. Set this value longer than the expected cleanup time for your process.

maxRetryAttempts
int32
(Optional)

Optional max retry Attempts when cleanCache function returns an error after execution, runtime attempts to run it three more times by default. With Maximum Retry Attempts, you can customize the maximum number of retries. This gives you the option to continue processing retries.

Condition

(Appears on: OperationStatus)

Condition explains the transitions on phase

Field Description
type
common.ConditionType

Type of condition, either Complete or Failed

status
Kubernetes core/v1.ConditionStatus

Status of the condition, one of True, False or Unknown

reason
string

Reason for the condition’s last transition

message
string

Message is a human-readable message indicating details about the transition

lastProbeTime
Kubernetes meta/v1.Time

LastProbeTime describes last time this condition was updated.

lastTransitionTime
Kubernetes meta/v1.Time

LastTransitionTime describes last time the condition transitioned from one status to another.

Data

(Appears on: AlluxioRuntimeSpec, GooseFSRuntimeSpec)

Data management strategies

Field Description
replicas
int32
(Optional)

The copies of the dataset

pin
bool
(Optional)

Pin the dataset or not. Refer to Alluxio User-CLI pin

DataBackupSpec

(Appears on: DataBackup)

DataBackupSpec defines the desired state of DataBackup

Field Description
dataset
string

Dataset defines the target dataset of the DataBackup

backupPath
string

BackupPath defines the target path to save data of the DataBackup

runAs
User

Manage the user to run Alluxio DataBackup

runAfter
OperationRef
(Optional)

Specifies that the preceding operation in a workflow

ttlSecondsAfterFinished
int32
(Optional)

TTLSecondsAfterFinished is the time second to clean up data operations after finished or failed

DataLoadSpec

(Appears on: DataLoad)

DataLoadSpec defines the desired state of DataLoad

Field Description
dataset
TargetDataset

Dataset defines the target dataset of the DataLoad

loadMetadata
bool

LoadMetadata specifies if the dataload job should load metadata

target
[]TargetPath

Target defines target paths that needs to be loaded

options
map[string]string

Options specifies the extra dataload properties for runtime

podMetadata
PodMetadata

PodMetadata defines labels and annotations that will be propagated to DataLoad pods

affinity
Kubernetes core/v1.Affinity
(Optional)

Affinity defines affinity for DataLoad pod

tolerations
[]Kubernetes core/v1.Toleration
(Optional)

Tolerations defines tolerations for DataLoad pod

nodeSelector
map[string]string
(Optional)

NodeSelector defiens node selector for DataLoad pod

schedulerName
string
(Optional)

SchedulerName sets the scheduler to be used for DataLoad pod

policy
Policy
(Optional)

including Once, Cron, OnEvent

schedule
string
(Optional)

The schedule in Cron format, only set when policy is cron, see https://en.wikipedia.org/wiki/Cron.

runAfter
OperationRef
(Optional)

Specifies that the preceding operation in a workflow

ttlSecondsAfterFinished
int32
(Optional)

TTLSecondsAfterFinished is the time second to clean up data operations after finished or failed

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by the DataLoad job.

DataMigrateSpec

(Appears on: DataMigrate)

DataMigrateSpec defines the desired state of DataMigrate

Field Description
VersionSpec
VersionSpec

(Members of VersionSpec are embedded into this type.)

(Optional)

The version information that instructs fluid to orchestrate a particular version for data migrate.

from
DataToMigrate

data to migrate source, including dataset and external storage

to
DataToMigrate

data to migrate destination, including dataset and external storage

block
bool
(Optional)

if dataMigrate blocked dataset usage, default is false

runtimeType
string
(Optional)

using which runtime to migrate data; if none, take dataset runtime as default

options
map[string]string
(Optional)

options for migrate, different for each runtime

policy
Policy
(Optional)

policy for migrate, including Once, Cron, OnEvent

schedule
string
(Optional)

The schedule in Cron format, only set when policy is cron, see https://en.wikipedia.org/wiki/Cron.

podMetadata
PodMetadata

PodMetadata defines labels and annotations that will be propagated to DataMigrate pods

affinity
Kubernetes core/v1.Affinity
(Optional)

Affinity defines affinity for DataMigrate pod

tolerations
[]Kubernetes core/v1.Toleration
(Optional)

Tolerations defines tolerations for DataMigrate pod

nodeSelector
map[string]string
(Optional)

NodeSelector defiens node selector for DataMigrate pod

schedulerName
string
(Optional)

SchedulerName sets the scheduler to be used for DataMigrate pod

runAfter
OperationRef
(Optional)

Specifies that the preceding operation in a workflow

ttlSecondsAfterFinished
int32
(Optional)

TTLSecondsAfterFinished is the time second to clean up data operations after finished or failed

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by the DataMigrate job.

parallelism
int32
(Optional)

Parallelism defines the parallelism tasks numbers for DataMigrate. If the value is greater than 1, the job acts as a launcher, and users should define the WorkerSpec.

parallelOptions
map[string]string
(Optional)

ParallelOptions defines options like ssh port and ssh secret name when parallelism is greater than 1.

DataProcess

DataProcess is the Schema for the dataprocesses API

Field Description
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
DataProcessSpec


dataset
TargetDatasetWithMountPath

Dataset specifies the target dataset and its mount path.

processor
Processor

Processor specify how to process data.

runAfter
OperationRef
(Optional)

Specifies that the preceding operation in a workflow

ttlSecondsAfterFinished
int32
(Optional)

TTLSecondsAfterFinished is the time second to clean up data operations after finished or failed

status
OperationStatus

DataProcessSpec

(Appears on: DataProcess)

DataProcessSpec defines the desired state of DataProcess

Field Description
dataset
TargetDatasetWithMountPath

Dataset specifies the target dataset and its mount path.

processor
Processor

Processor specify how to process data.

runAfter
OperationRef
(Optional)

Specifies that the preceding operation in a workflow

ttlSecondsAfterFinished
int32
(Optional)

TTLSecondsAfterFinished is the time second to clean up data operations after finished or failed

DataRestoreLocation

(Appears on: DatasetSpec)

DataRestoreLocation describes the spec restore location of Dataset

Field Description
path
string
(Optional)

Path describes the path of restore, in the form of local://subpath or pvc:///subpath

nodeName
string
(Optional)

NodeName describes the nodeName of restore if Path is in the form of local://subpath

DataToMigrate

(Appears on: DataMigrateSpec)

Field Description
dataset
DatasetToMigrate

dataset to migrate

externalStorage
ExternalStorage

external storage for data migrate

DatasetCondition

(Appears on: DatasetStatus)

Condition describes the state of the cache at a certain point.

Field Description
type
DatasetConditionType

Type of cache condition.

status
Kubernetes core/v1.ConditionStatus

Status of the condition, one of True, False, Unknown.

reason
string

The reason for the condition’s last transition.

message
string

A human readable message indicating details about the transition.

lastUpdateTime
Kubernetes meta/v1.Time

The last time this condition was updated.

lastTransitionTime
Kubernetes meta/v1.Time

Last time the condition transitioned from one status to another.

DatasetSpec

(Appears on: Dataset)

DatasetSpec defines the desired state of Dataset

Field Description
mounts
[]Mount
(Optional)

Mount Points to be mounted on cache runtime.
This field can be empty because some runtimes don’t need to mount external storage (e.g. Vineyard).

owner
User
(Optional)

The owner of the dataset

nodeAffinity
CacheableNodeAffinity
(Optional)

NodeAffinity defines constraints that limit what nodes this dataset can be cached to. This field influences the scheduling of pods that use the cached dataset.

tolerations
[]Kubernetes core/v1.Toleration
(Optional)

If specified, the pod’s tolerations.

accessModes
[]Kubernetes core/v1.PersistentVolumeAccessMode
(Optional)

AccessModes contains all ways the volume backing the PVC can be mounted

runtimes
[]Runtime

Runtimes for supporting dataset (e.g. AlluxioRuntime)

placement
PlacementMode
(Optional)

Manage switch for opening Multiple datasets single node deployment or not TODO(xieydd) In future, evaluate node resources and runtime resources to decide whether to turn them on

dataRestoreLocation
DataRestoreLocation
(Optional)

DataRestoreLocation is the location to load data of dataset been backuped

sharedOptions
map[string]string
(Optional)

SharedOptions is the options to all mount

sharedEncryptOptions
[]EncryptOption
(Optional)

SharedEncryptOptions is the encryptOption to all mount

DatasetStatus

(Appears on: Dataset)

DatasetStatus defines the observed state of Dataset

Field Description
mounts
[]Mount

the info of mount points have been mounted

ufsTotal
string

Total in GB of dataset in the cluster

phase
DatasetPhase

Dataset Phase. One of the four phases: Pending, Bound, NotBound and Failed

runtimes
[]Runtime

Runtimes for supporting dataset

conditions
[]DatasetCondition

Conditions is an array of current observed conditions.

cacheStates
common.CacheStateList

CacheStatus represents the total resources of the dataset.

hcfs
HCFSStatus

HCFSStatus represents hcfs info

fileNum
string

FileNum represents the file numbers of the dataset

dataLoadRef
string

DataLoadRef specifies the running DataLoad job that targets this Dataset. This is mainly used as a lock to prevent concurrent DataLoad jobs. Deprecated, use OperationRef instead

dataBackupRef
string

DataBackupRef specifies the running Backup job that targets this Dataset. This is mainly used as a lock to prevent concurrent DataBackup jobs. Deprecated, use OperationRef instead

operationRef
map[string]string

OperationRef specifies the Operation that targets this Dataset. This is mainly used as a lock to prevent concurrent same Operation jobs.

datasetRef
[]string

DatasetRef specifies the datasets namespaced name mounting this Dataset.

DatasetToMigrate

(Appears on: DataToMigrate)

Field Description
name
string

name of dataset

namespace
string

namespace of dataset

path
string

path to migrate

EFCCompTemplateSpec

(Appears on: EFCRuntimeSpec)

EFCCompTemplateSpec is a description of the EFC components

Field Description
replicas
int32
(Optional)

Replicas is the desired number of replicas of the given template. If unspecified, defaults to 1. replicas is the min replicas of dataset in the cluster

version
VersionSpec

The version information that instructs fluid to orchestrate a particular version of EFC Comp

properties
map[string]string
(Optional)

Configurable properties for the EFC component.

ports
map[string]int
(Optional)

Ports used by EFC(e.g. rpc: 19998 for master).

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by the EFC component.

Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.

disabled
bool
(Optional)

Enabled or Disabled for the components. Default enable.

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector which must be true for the component to fit on a node.

networkMode
NetworkMode
(Optional)

Whether to use host network or not.

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to EFC’s master and worker pods

EFCFuseSpec

(Appears on: EFCRuntimeSpec)

EFCFuseSpec is a description of the EFC Fuse

Field Description
version
VersionSpec

The version information that instructs fluid to orchestrate a particular version of EFC Fuse

properties
map[string]string
(Optional)

Configurable properties for EFC fuse

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by EFC Fuse.

Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector which must be true for the fuse client to fit on a node, this option only effect when global is enabled

cleanPolicy
FuseCleanPolicy
(Optional)

CleanPolicy decides when to clean EFC Fuse pods. Currently Fluid supports two policies: OnDemand and OnRuntimeDeleted OnDemand cleans fuse pod once th fuse pod on some node is not needed OnRuntimeDeleted cleans fuse pod only when the cache runtime is deleted Defaults to OnRuntimeDeleted

networkMode
NetworkMode
(Optional)

Whether to use hostnetwork or not

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to EFC’s fuse pods

EFCRuntimeSpec

(Appears on: EFCRuntime)

EFCRuntimeSpec defines the desired state of EFCRuntime

Field Description
master
EFCCompTemplateSpec

The component spec of EFC master

worker
EFCCompTemplateSpec

The component spec of EFC worker

initFuse
InitFuseSpec

The spec of init alifuse

fuse
EFCFuseSpec

The component spec of EFC Fuse

tieredstore
TieredStore

Tiered storage used by EFC worker

replicas
int32

The replicas of the worker, need to be specified

osAdvise
OSAdvise

Operating system optimization for EFC

cleanCachePolicy
CleanCachePolicy
(Optional)

CleanCachePolicy defines cleanCache Policy

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to all EFC’s pods

EncryptOption

(Appears on: DatasetSpec, ExternalEndpointSpec, ExternalStorage, Mount)

Field Description
name
string

The name of encryptOption

valueFrom
EncryptOptionSource
(Optional)

The valueFrom of encryptOption

EncryptOptionSource

(Appears on: EncryptOption)

Field Description
secretKeyRef
SecretKeySelector
(Optional)

The encryptInfo obtained from secret

ExternalEndpointSpec

(Appears on: MasterSpec)

ExternalEndpointSpec defines the configurations for external etcd cluster

Field Description
uri
string
(Optional)

URI specifies the endpoint of external Etcd cluster E,g. “etcd-svc.etcd-namespace.svc.cluster.local:2379” Default is not set and use http protocol to connect to external etcd cluster

encryptOptions
[]EncryptOption
(Optional)

encrypt info for accessing the external etcd cluster

options
map[string]string
(Optional)

Configurable options for External Etcd cluster.

ExternalStorage

(Appears on: DataToMigrate)

Field Description
uri
string

type of external storage, including s3, oss, gcs, ceph, nfs, pvc, etc. (related to runtime)

encryptOptions
[]EncryptOption
(Optional)

encrypt info for external storage

FuseCleanPolicy (string alias)

(Appears on: AlluxioFuseSpec, EFCFuseSpec, GooseFSFuseSpec, JindoFuseSpec, JuiceFSFuseSpec, ThinFuseSpec, VineyardSockSpec)

GooseFSCompTemplateSpec

(Appears on: GooseFSRuntimeSpec)

GooseFSCompTemplateSpec is a description of the GooseFS commponents

Field Description
replicas
int32
(Optional)

Replicas is the desired number of replicas of the given template. If unspecified, defaults to 1. replicas is the min replicas of dataset in the cluster

jvmOptions
[]string

Options for JVM

properties
map[string]string
(Optional)

Configurable properties for the GOOSEFS component.
Refer to GOOSEFS Configuration Properties for more info

ports
map[string]int
(Optional)

Ports used by GooseFS(e.g. rpc: 19998 for master)

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by the GooseFS component.

Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.

env
map[string]string

Environment variables that will be used by GooseFS component.

enabled
bool
(Optional)

Enabled or Disabled for the components. For now, only API Gateway is enabled or disabled.

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector which must be true for the master to fit on a node

annotations
map[string]string
(Optional)

Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations

GooseFSFuseSpec

(Appears on: GooseFSRuntimeSpec)

GooseFSFuseSpec is a description of the GooseFS Fuse

Field Description
image
string

Image for GooseFS Fuse(e.g. goosefs/goosefs-fuse)

imageTag
string

Image Tag for GooseFS Fuse(e.g. v1.0.1)

imagePullPolicy
string

One of the three policies: Always, IfNotPresent, Never

jvmOptions
[]string

Options for JVM

properties
map[string]string

Configurable properties for the GOOSEFS component.
Refer to GOOSEFS Configuration Properties for more info

env
map[string]string

Environment variables that will be used by GooseFS Fuse

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by GooseFS Fuse.

Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.

args
[]string

Arguments that will be passed to GooseFS Fuse

global
bool
(Optional)

If the fuse client should be deployed in global mode, otherwise the affinity should be considered

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector which must be true for the fuse client to fit on a node, this option only effect when global is enabled

cleanPolicy
FuseCleanPolicy
(Optional)

CleanPolicy decides when to clean GooseFS Fuse pods. Currently Fluid supports two policies: OnDemand and OnRuntimeDeleted OnDemand cleans fuse pod once th fuse pod on some node is not needed OnRuntimeDeleted cleans fuse pod only when the cache runtime is deleted Defaults to OnRuntimeDeleted

annotations
map[string]string
(Optional)

Annotations is an unstructured key value map stored with a resource that may be set by external tools to store and retrieve arbitrary metadata. They are not queryable and should be preserved when modifying objects. More info: http://kubernetes.io/docs/user-guide/annotations

GooseFSRuntimeSpec

(Appears on: GooseFSRuntime)

GooseFSRuntimeSpec defines the desired state of GooseFSRuntime

Field Description
goosefsVersion
VersionSpec

The version information that instructs fluid to orchestrate a particular version of GooseFS.

master
GooseFSCompTemplateSpec

The component spec of GooseFS master

jobMaster
GooseFSCompTemplateSpec

The component spec of GooseFS job master

worker
GooseFSCompTemplateSpec

The component spec of GooseFS worker

jobWorker
GooseFSCompTemplateSpec

The component spec of GooseFS job Worker

apiGateway
GooseFSCompTemplateSpec

The component spec of GooseFS API Gateway

initUsers
InitUsersSpec

The spec of init users

fuse
GooseFSFuseSpec

The component spec of GooseFS Fuse

properties
map[string]string

Configurable properties for the GOOSEFS component.
Refer to GOOSEFS Configuration Properties for more info

jvmOptions
[]string

Options for JVM

tieredstore
TieredStore

Tiered storage used by GooseFS

data
Data

Management strategies for the dataset to which the runtime is bound

replicas
int32

The replicas of the worker, need to be specified

runAs
User

Manage the user to run GooseFS Runtime GooseFS support POSIX-ACL and Apache Ranger to manager authorization TODO([email protected]) Support Apache Ranger.

disablePrometheus
bool
(Optional)

Disable monitoring for GooseFS Runtime Prometheus is enabled by default

hadoopConfig
string
(Optional)

Name of the configMap used to support HDFS configurations when using HDFS as GooseFS’s UFS. The configMap must be in the same namespace with the GooseFSRuntime. The configMap should contain user-specific HDFS conf files in it. For now, only “hdfs-site.xml” and “core-site.xml” are supported. It must take the filename of the conf file as the key and content of the file as the value.

cleanCachePolicy
CleanCachePolicy
(Optional)

CleanCachePolicy defines cleanCache Policy

HCFSStatus

(Appears on: DatasetStatus)

HCFS Endpoint info

Field Description
endpoint
string

Endpoint for accessing

underlayerFileSystemVersion
string

Underlayer HCFS Compatible Version

InitFuseSpec

(Appears on: EFCRuntimeSpec)

InitFuseSpec is a description of initialize the fuse kernel module for runtime

Field Description
version
VersionSpec

The version information that instructs fluid to orchestrate a particular version of Alifuse

InitUsersSpec

(Appears on: AlluxioRuntimeSpec, GooseFSRuntimeSpec, JuiceFSRuntimeSpec)

InitUsersSpec is a description of the initialize the users for runtime

Field Description
image
string

Image for initialize the users for runtime(e.g. alluxio/alluxio-User init)

imageTag
string

Image Tag for initialize the users for runtime(e.g. 2.3.0-SNAPSHOT)

imagePullPolicy
string

One of the three policies: Always, IfNotPresent, Never

env
map[string]string

Environment variables that will be used by initialize the users for runtime

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by initialize the users for runtime.

Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.

JindoCompTemplateSpec

(Appears on: JindoRuntimeSpec)

JindoCompTemplateSpec is a description of the Jindo commponents

Field Description
replicas
int32
(Optional)

Replicas is the desired number of replicas of the given template. If unspecified, defaults to 1. replicas is the min replicas of dataset in the cluster

properties
map[string]string
(Optional)

Configurable properties for the Jindo component.

ports
map[string]int
(Optional)
resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by the Jindo component.

Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.

env
map[string]string

Environment variables that will be used by Jindo component.

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector which must be true for the master to fit on a node

tolerations
[]Kubernetes core/v1.Toleration
(Optional)

If specified, the pod’s tolerations.

labels
map[string]string
(Optional)

Labels will be added on JindoFS Master or Worker pods. DEPRECATED: This is a deprecated field. Please use PodMetadata instead. Note: this field is set to be exclusive with PodMetadata.Labels

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to Jindo’s pods

disabled
bool
(Optional)

If disable JindoFS master or worker

volumeMounts
[]Kubernetes core/v1.VolumeMount
(Optional)

VolumeMounts specifies the volumes listed in “.spec.volumes” to mount into the jindo runtime component’s filesystem.

JindoFuseSpec

(Appears on: JindoRuntimeSpec)

JindoFuseSpec is a description of the Jindo Fuse

Field Description
image
string

Image for Jindo Fuse(e.g. jindo/jindo-fuse)

imageTag
string

Image Tag for Jindo Fuse(e.g. 2.3.0-SNAPSHOT)

imagePullPolicy
string

One of the three policies: Always, IfNotPresent, Never

properties
map[string]string

Configurable properties for Jindo System.

env
map[string]string

Environment variables that will be used by Jindo Fuse

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by Jindo Fuse.

Resources are not allowed for ephemeral containers. Ephemeral containers use spare resources already allocated to the pod.

args
[]string

Arguments that will be passed to Jindo Fuse

global
bool
(Optional)

If the fuse client should be deployed in global mode, otherwise the affinity should be considered

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector which must be true for the fuse client to fit on a node, this option only effect when global is enabled

tolerations
[]Kubernetes core/v1.Toleration
(Optional)

If specified, the pod’s tolerations.

labels
map[string]string
(Optional)

Labels will be added on all the JindoFS pods. DEPRECATED: this is a deprecated field. Please use PodMetadata.Labels instead. Note: this field is set to be exclusive with PodMetadata.Labels

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to Jindo’s fuse pods

cleanPolicy
FuseCleanPolicy
(Optional)

CleanPolicy decides when to clean JindoFS Fuse pods. Currently Fluid supports two policies: OnDemand and OnRuntimeDeleted OnDemand cleans fuse pod once th fuse pod on some node is not needed OnRuntimeDeleted cleans fuse pod only when the cache runtime is deleted Defaults to OnRuntimeDeleted

disabled
bool
(Optional)

If disable JindoFS fuse

logConfig
map[string]string
(Optional)

JindoRuntimeSpec

(Appears on: JindoRuntime)

JindoRuntimeSpec defines the desired state of JindoRuntime

Field Description
jindoVersion
VersionSpec

The version information that instructs fluid to orchestrate a particular version of Jindo.

master
JindoCompTemplateSpec

The component spec of Jindo master

worker
JindoCompTemplateSpec

The component spec of Jindo worker

fuse
JindoFuseSpec

The component spec of Jindo Fuse

properties
map[string]string

Configurable properties for Jindo system.

tieredstore
TieredStore

Tiered storage used by Jindo

replicas
int32

The replicas of the worker, need to be specified

runAs
User

Manage the user to run Jindo Runtime

user
string
hadoopConfig
string
(Optional)

Name of the configMap used to support HDFS configurations when using HDFS as Jindo’s UFS. The configMap must be in the same namespace with the JindoRuntime. The configMap should contain user-specific HDFS conf files in it. For now, only “hdfs-site.xml” and “core-site.xml” are supported. It must take the filename of the conf file as the key and content of the file as the value.

secret
string
labels
map[string]string
(Optional)

Labels will be added on all the JindoFS pods. DEPRECATED: this is a deprecated field. Please use PodMetadata.Labels instead. Note: this field is set to be exclusive with PodMetadata.Labels

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to all Jindo’s fuse pods

logConfig
map[string]string
(Optional)
networkmode
NetworkMode
(Optional)

Whether to use hostnetwork or not

cleanCachePolicy
CleanCachePolicy
(Optional)

CleanCachePolicy defines cleanCache Policy

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by the jindo runtime components and/or fuses.

JobProcessor

(Appears on: Processor)

Field Description
podSpec
Kubernetes core/v1.PodSpec
(Optional)

PodSpec defines Pod specification of the DataProcess job.

JuiceFSCompTemplateSpec

(Appears on: JuiceFSRuntimeSpec)

JuiceFSCompTemplateSpec is a description of the JuiceFS components

Field Description
replicas
int32
(Optional)

Replicas is the desired number of replicas of the given template. If unspecified, defaults to 1. replicas is the min replicas of dataset in the cluster

ports
[]Kubernetes core/v1.ContainerPort
(Optional)

Ports used by JuiceFS

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by the JuiceFS component.

options
map[string]string

Options

env
[]Kubernetes core/v1.EnvVar

Environment variables that will be used by JuiceFS component.

enabled
bool
(Optional)

Enabled or Disabled for the components.

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector

volumeMounts
[]Kubernetes core/v1.VolumeMount
(Optional)

VolumeMounts specifies the volumes listed in “.spec.volumes” to mount into runtime component’s filesystem.

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to JuiceFs’s pods.

networkMode
NetworkMode
(Optional)

Whether to use hostnetwork or not

JuiceFSFuseSpec

(Appears on: JuiceFSRuntimeSpec)

Field Description
image
string

Image for JuiceFS fuse

imageTag
string

Image for JuiceFS fuse

imagePullPolicy
string

One of the three policies: Always, IfNotPresent, Never

env
[]Kubernetes core/v1.EnvVar

Environment variables that will be used by JuiceFS Fuse

resources
Kubernetes core/v1.ResourceRequirements

Resources that will be requested by JuiceFS Fuse.

options
map[string]string
(Optional)

Options mount options that fuse pod will use

global
bool
(Optional)

If the fuse client should be deployed in global mode, otherwise the affinity should be considered

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector which must be true for the fuse client to fit on a node, this option only effect when global is enabled

volumeMounts
[]Kubernetes core/v1.VolumeMount
(Optional)

VolumeMounts specifies the volumes listed in “.spec.volumes” to mount into runtime component’s filesystem.

cleanPolicy
FuseCleanPolicy
(Optional)

CleanPolicy decides when to clean Juicefs Fuse pods. Currently Fluid supports two policies: OnDemand and OnRuntimeDeleted OnDemand cleans fuse pod once th fuse pod on some node is not needed OnRuntimeDeleted cleans fuse pod only when the cache runtime is deleted Defaults to OnDemand

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to JuiceFs’s pods.

networkMode
NetworkMode
(Optional)

Whether to use hostnetwork or not

JuiceFSRuntimeSpec

(Appears on: JuiceFSRuntime)

JuiceFSRuntimeSpec defines the desired state of JuiceFSRuntime

Field Description
juicefsVersion
VersionSpec

The version information that instructs fluid to orchestrate a particular version of JuiceFS.

initUsers
InitUsersSpec

The spec of init users

master
JuiceFSCompTemplateSpec

The component spec of JuiceFS master

worker
JuiceFSCompTemplateSpec

The component spec of JuiceFS worker

jobWorker
JuiceFSCompTemplateSpec

The component spec of JuiceFS job Worker

fuse
JuiceFSFuseSpec

Desired state for JuiceFS Fuse

tieredstore
TieredStore

Tiered storage used by JuiceFS

configs
string

Configs of JuiceFS

replicas
int32

The replicas of the worker, need to be specified

runAs
User

Manage the user to run Juicefs Runtime

disablePrometheus
bool
(Optional)

Disable monitoring for JuiceFS Runtime Prometheus is enabled by default

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by the alluxio runtime components and/or fuses.

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to JuiceFs’s pods.

cleanCachePolicy
CleanCachePolicy
(Optional)

CleanCachePolicy defines cleanCache Policy

Level

(Appears on: TieredStore)

Level describes configurations a tier needs.
Refer to Configuring Tiered Storage for more info

Field Description
mediumtype
common.MediumType

Medium Type of the tier. One of the three types: MEM, SSD, HDD

volumeType
common.VolumeType
(Optional)

VolumeType is the volume type of the tier. Should be one of the three types: hostPath, emptyDir and volumeTemplate. If not set, defaults to hostPath.

volumeSource
VolumeSource

VolumeSource is the volume source of the tier. It follows the form of corev1.VolumeSource. For now, users should only specify VolumeSource when VolumeType is set to emptyDir.

path
string
(Optional)

File paths to be used for the tier. Multiple paths are supported. Multiple paths should be separated with comma. For example: “/mnt/cache1,/mnt/cache2”.

quota
Kubernetes resource.Quantity
(Optional)

Quota for the whole tier. (e.g. 100Gi) Please note that if there’re multiple paths used for this tierstore, the quota will be equally divided into these paths. If you’d like to set quota for each, path, see QuotaList for more information.

quotaList
string
(Optional)

QuotaList are quotas used to set quota on multiple paths. Quotas should be separated with comma. Quotas in this list will be set to paths with the same order in Path. For example, with Path defined with “/mnt/cache1,/mnt/cache2” and QuotaList set to “100Gi, 50Gi”, then we get 100GiB cache storage under “/mnt/cache1” and 50GiB under “/mnt/cache2”. Also note that num of quotas must be consistent with the num of paths defined in Path.

high
string

Ratio of high watermark of the tier (e.g. 0.9)

low
string

Ratio of low watermark of the tier (e.g. 0.7)

MasterSpec

(Appears on: VineyardRuntimeSpec)

MasterSpec defines the configurations for Vineyard Master component which is also regarded as the Etcd component in Vineyard. For more info about Vineyard, refer to Vineyard

Field Description
VineyardCompTemplateSpec
VineyardCompTemplateSpec

(Members of VineyardCompTemplateSpec are embedded into this type.)

(Optional)

The component configurations for Vineyard Master

endpoint
ExternalEndpointSpec
(Optional)

ExternalEndpoint defines the configurations for external etcd cluster Default is not set If set, the Vineyard Master component will not be deployed, which means the Vineyard Worker component will use an external Etcd cluster. E,g. endpoint: uri: “etcd-svc.etcd-namespace.svc.cluster.local:2379” encryptOptions: - name: access-key valueFrom: secretKeyRef: name: etcd-secret key: accesskey

Metadata

Metadata defines subgroup properties of metav1.ObjectMeta

Field Description
PodMetadata
PodMetadata

(Members of PodMetadata are embedded into this type.)

selector
Kubernetes meta/v1.GroupKind

MetadataSyncPolicy

(Appears on: RuntimeManagement)

MetadataSyncPolicy defines policies when syncing metadata

Field Description
autoSync
bool
(Optional)

AutoSync enables automatic metadata sync when setting up a runtime. If not set, it defaults to true.

Mount

(Appears on: DatasetSpec, DatasetStatus, RuntimeStatus)

Mount describes a mounting.
Refer to Alluxio Storage Integrations for more info

Field Description
mountPoint
string

MountPoint is the mount point of source.

options
map[string]string
(Optional)

The Mount Options.
Refer to Mount Options.
The option has Prefix ‘fs.’ And you can Learn more from The Storage Integrations

name
string
(Optional)

The name of mount

path
string
(Optional)

The path of mount, if not set will be /{Name}

readOnly
bool
(Optional)

Optional: Defaults to false (read-write).

shared
bool
(Optional)

Optional: Defaults to false (shared).

encryptOptions
[]EncryptOption
(Optional)

The secret information

NetworkMode (string alias)

(Appears on: AlluxioCompTemplateSpec, AlluxioFuseSpec, EFCCompTemplateSpec, EFCFuseSpec, JindoRuntimeSpec, JuiceFSCompTemplateSpec, JuiceFSFuseSpec, ThinCompTemplateSpec, ThinFuseSpec, VineyardCompTemplateSpec, VineyardSockSpec)

NodePublishSecretPolicy (string alias)

(Appears on: ThinRuntimeProfileSpec)

OSAdvise

(Appears on: EFCRuntimeSpec)

OSAdvise is a description of choices to have optimization on specific operating system

Field Description
osVersion
string
(Optional)

Specific operating system version that can have optimization.

enabled
bool
(Optional)

Enable operating system optimization not enabled by default.

OperationRef

(Appears on: DataBackupSpec, DataLoadSpec, DataMigrateSpec, DataProcessSpec)

Field Description
apiVersion
string
(Optional)

API version of the referent operation

kind
string

Kind specifies the type of the referent operation

name
string

Name specifies the name of the referent operation

namespace
string
(Optional)

Namespace specifies the namespace of the referent operation.

OperationStatus

(Appears on: DataBackup, DataLoad, DataMigrate, DataProcess)

OperationStatus defines the observed state of operation

Field Description
phase
common.Phase

Phase describes current phase of operation

duration
string

Duration tell user how much time was spent to operation

conditions
[]Condition

Conditions consists of transition information on operation’s Phase

infos
map[string]string

Infos operation customized name-value

lastScheduleTime
Kubernetes meta/v1.Time

LastScheduleTime is the last time the cron operation was scheduled

lastSuccessfulTime
Kubernetes meta/v1.Time

LastSuccessfulTime is the last time the cron operation successfully completed

waitingFor
WaitingStatus

WaitingStatus stores information about waiting operation.

OperationType (string alias)

PlacementMode (string alias)

(Appears on: DatasetSpec)

PodMetadata

(Appears on: AlluxioCompTemplateSpec, AlluxioFuseSpec, AlluxioRuntimeSpec, DataLoadSpec, DataMigrateSpec, EFCCompTemplateSpec, EFCFuseSpec, EFCRuntimeSpec, JindoCompTemplateSpec, JindoFuseSpec, JindoRuntimeSpec, JuiceFSCompTemplateSpec, JuiceFSFuseSpec, JuiceFSRuntimeSpec, Metadata, Processor, VineyardCompTemplateSpec, VineyardRuntimeSpec, VineyardSockSpec)

PodMetadata defines subgroup properties of metav1.ObjectMeta

Field Description
labels
map[string]string

Labels are labels of pod specification

annotations
map[string]string

Annotations are annotations of pod specification

Policy (string alias)

(Appears on: DataLoadSpec, DataMigrateSpec)

Processor

(Appears on: DataProcessSpec)

Processor defines the actual processor for DataProcess. Processor can be either of a Job or a Shell script.

Field Description
serviceAccountName
string
(Optional)

ServiceAccountName defiens the serviceAccountName of the container

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations on the processor pod.

job
JobProcessor
(Optional)

Job represents a processor which runs DataProcess as a job.

script
ScriptProcessor

Shell represents a processor which executes shell script

Runtime

(Appears on: DatasetSpec, DatasetStatus)

Runtime describes a runtime to be used to support dataset

Field Description
name
string

Name of the runtime object

namespace
string

Namespace of the runtime object

category
common.Category

Category the runtime object belongs to (e.g. Accelerate)

type
string

Runtime object’s type (e.g. Alluxio)

masterReplicas
int32

Runtime master replicas

RuntimeCondition

(Appears on: RuntimeStatus)

Condition describes the state of the cache at a certain point.

Field Description
type
RuntimeConditionType

Type of cache condition.

status
Kubernetes core/v1.ConditionStatus

Status of the condition, one of True, False, Unknown.

reason
string

The reason for the condition’s last transition.

message
string

A human readable message indicating details about the transition.

lastProbeTime
Kubernetes meta/v1.Time

The last time this condition was updated.

lastTransitionTime
Kubernetes meta/v1.Time

Last time the condition transitioned from one status to another.

RuntimeManagement

(Appears on: AlluxioRuntimeSpec)

RuntimeManagement defines suggestions for runtime controllers to manage the runtime

Field Description
cleanCachePolicy
CleanCachePolicy
(Optional)

CleanCachePolicy defines the policy of cleaning cache when shutting down the runtime

metadataSyncPolicy
MetadataSyncPolicy
(Optional)

MetadataSyncPolicy defines the policy of syncing metadata when setting up the runtime. If not set,

RuntimeStatus

(Appears on: AlluxioRuntime, EFCRuntime, GooseFSRuntime, JindoRuntime, JuiceFSRuntime, VineyardRuntime, ThinRuntime)

RuntimeStatus defines the observed state of Runtime

Field Description
valueFile
string

config map used to set configurations

masterPhase
RuntimePhase

MasterPhase is the master running phase

masterReason
string

Reason for Master’s condition transition

workerPhase
RuntimePhase

WorkerPhase is the worker running phase

workerReason
string

Reason for Worker’s condition transition

desiredWorkerNumberScheduled
int32

The total number of nodes that should be running the runtime worker pod (including nodes correctly running the runtime worker pod).

currentWorkerNumberScheduled
int32

The total number of nodes that can be running the runtime worker pod (including nodes correctly running the runtime worker pod).

workerNumberReady
int32

The number of nodes that should be running the runtime worker pod and have one or more of the runtime worker pod running and ready.

workerNumberAvailable
int32
(Optional)

The number of nodes that should be running the runtime worker pod and have one or more of the runtime worker pod running and available (ready for at least spec.minReadySeconds)

workerNumberUnavailable
int32
(Optional)

The number of nodes that should be running the runtime worker pod and have none of the runtime worker pod running and available (ready for at least spec.minReadySeconds)

desiredMasterNumberScheduled
int32

The total number of nodes that should be running the runtime pod (including nodes correctly running the runtime master pod).

currentMasterNumberScheduled
int32

The total number of nodes that should be running the runtime pod (including nodes correctly running the runtime master pod).

masterNumberReady
int32

The number of nodes that should be running the runtime worker pod and have zero or more of the runtime master pod running and ready.

fusePhase
RuntimePhase

FusePhase is the Fuse running phase

fuseReason
string

Reason for the condition’s last transition.

currentFuseNumberScheduled
int32

The total number of nodes that can be running the runtime Fuse pod (including nodes correctly running the runtime Fuse pod).

desiredFuseNumberScheduled
int32

The total number of nodes that should be running the runtime Fuse pod (including nodes correctly running the runtime Fuse pod).

fuseNumberReady
int32

The number of nodes that should be running the runtime Fuse pod and have one or more of the runtime Fuse pod running and ready.

fuseNumberUnavailable
int32
(Optional)

The number of nodes that should be running the runtime fuse pod and have none of the runtime fuse pod running and available (ready for at least spec.minReadySeconds)

fuseNumberAvailable
int32
(Optional)

The number of nodes that should be running the runtime Fuse pod and have one or more of the runtime Fuse pod running and available (ready for at least spec.minReadySeconds)

setupDuration
string

Duration tell user how much time was spent to setup the runtime

conditions
[]RuntimeCondition

Represents the latest available observations of a ddc runtime’s current state.

cacheStates
common.CacheStateList

CacheStatus represents the total resources of the dataset.

selector
string

Selector is used for auto-scaling

apiGateway
APIGatewayStatus

APIGatewayStatus represents rest api gateway status

mountTime
Kubernetes meta/v1.Time

MountTime represents time last mount happened if Mounttime is earlier than master starting time, remount will be required

mounts
[]Mount

MountPoints represents the mount points specified in the bounded dataset

cacheAffinity
Kubernetes core/v1.NodeAffinity

CacheAffinity represents the runtime worker pods node affinity including node selector

ScriptProcessor

(Appears on: Processor)

Field Description
VersionSpec
VersionSpec

(Members of VersionSpec are embedded into this type.)

VersionSpec specifies the container’s image info.

restartPolicy
Kubernetes core/v1.RestartPolicy
(Optional)

RestartPolicy specifies the processor job’s restart policy. Only “Never”, “OnFailure” is allowed.

command
[]string
(Optional)

Entrypoint command for ScriptProcessor.

source
string

Script source for ScriptProcessor

env
[]Kubernetes core/v1.EnvVar
(Optional)

List of environment variables to set in the container.

volumeMounts
[]Kubernetes core/v1.VolumeMount
(Optional)

Pod volumes to mount into the container’s filesystem.

volumes
[]Kubernetes core/v1.Volume
(Optional)

List of volumes that can be mounted by containers belonging to the pod.

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by the DataProcess job.

SecretKeySelector

(Appears on: EncryptOptionSource)

Field Description
name
string

The name of required secret

key
string
(Optional)

The required key in the secret

TargetDataset

(Appears on: DataLoadSpec, TargetDatasetWithMountPath)

TargetDataset defines the target dataset of the DataLoad

Field Description
name
string

Name defines name of the target dataset

namespace
string

Namespace defines namespace of the target dataset

TargetDatasetWithMountPath

(Appears on: DataProcessSpec)

TargetDataset defines which dataset will be processed by DataProcess. Under the hood, the dataset’s pvc will be mounted to the given mountPath of the DataProcess’s containers.

Field Description
TargetDataset
TargetDataset

(Members of TargetDataset are embedded into this type.)

mountPath
string

MountPath defines where the Dataset should be mounted in DataProcess’s containers.

subPath
string
(Optional)

SubPath defines subpath of the target dataset to mount.

TargetPath

(Appears on: DataLoadSpec)

TargetPath defines the target path of the DataLoad

Field Description
path
string

Path defines path to be load

replicas
int32

Replicas defines how many replicas will be loaded

ThinCompTemplateSpec

(Appears on: ThinRuntimeProfileSpec, ThinRuntimeSpec)

ThinCompTemplateSpec is a description of the thinRuntime components

Field Description
image
string

Image for thinRuntime fuse

imageTag
string

Image for thinRuntime fuse

imagePullPolicy
string

One of the three policies: Always, IfNotPresent, Never

replicas
int32
(Optional)

Replicas is the desired number of replicas of the given template. If unspecified, defaults to 1. replicas is the min replicas of dataset in the cluster

ports
[]Kubernetes core/v1.ContainerPort
(Optional)

Ports used thinRuntime

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources that will be requested by thinRuntime component.

env
[]Kubernetes core/v1.EnvVar

Environment variables that will be used by thinRuntime component.

enabled
bool
(Optional)

Enabled or Disabled for the components.

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector

volumeMounts
[]Kubernetes core/v1.VolumeMount
(Optional)

VolumeMounts specifies the volumes listed in “.spec.volumes” to mount into runtime component’s filesystem.

livenessProbe
Kubernetes core/v1.Probe
(Optional)

livenessProbe of thin fuse pod

readinessProbe
Kubernetes core/v1.Probe
(Optional)

readinessProbe of thin fuse pod

networkMode
NetworkMode
(Optional)

Whether to use hostnetwork or not

ThinFuseSpec

(Appears on: ThinRuntimeProfileSpec, ThinRuntimeSpec)

Field Description
image
string

Image for thinRuntime fuse

imageTag
string

Image for thinRuntime fuse

imagePullPolicy
string

One of the three policies: Always, IfNotPresent, Never

ports
[]Kubernetes core/v1.ContainerPort
(Optional)

Ports used thinRuntime

env
[]Kubernetes core/v1.EnvVar

Environment variables that will be used by thinRuntime Fuse

command
[]string

Command that will be passed to thinRuntime Fuse

args
[]string

Arguments that will be passed to thinRuntime Fuse

options
map[string]string

Options configurable options of FUSE client, performance parameters usually. will be merged with Dataset.spec.mounts.options into fuse pod.

resources
Kubernetes core/v1.ResourceRequirements

Resources that will be requested by thinRuntime Fuse.

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector which must be true for the fuse client to fit on a node, this option only effect when global is enabled

cleanPolicy
FuseCleanPolicy
(Optional)

CleanPolicy decides when to clean thinRuntime Fuse pods. Currently Fluid supports two policies: OnDemand and OnRuntimeDeleted OnDemand cleans fuse pod once the fuse pod on some node is not needed OnRuntimeDeleted cleans fuse pod only when the cache runtime is deleted Defaults to OnDemand

networkMode
NetworkMode
(Optional)

Whether to use hostnetwork or not

livenessProbe
Kubernetes core/v1.Probe
(Optional)

livenessProbe of thin fuse pod

readinessProbe
Kubernetes core/v1.Probe
(Optional)

readinessProbe of thin fuse pod

volumeMounts
[]Kubernetes core/v1.VolumeMount
(Optional)

VolumeMounts specifies the volumes listed in “.spec.volumes” to mount into the thinruntime component’s filesystem.

ThinRuntime

ThinRuntime is the Schema for the thinruntimes API

Field Description
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
ThinRuntimeSpec


profileName
string

The specific runtime profile name, empty value is used for handling datasets which mount another dataset

worker
ThinCompTemplateSpec

The component spec of worker

fuse
ThinFuseSpec

The component spec of thinRuntime

tieredstore
TieredStore

Tiered storage

replicas
int32

The replicas of the worker, need to be specified

runAs
User

Manage the user to run Runtime

disablePrometheus
bool
(Optional)

Disable monitoring for Runtime Prometheus is enabled by default

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by runtime components and/or fuses.

status
RuntimeStatus

ThinRuntimeProfile

ThinRuntimeProfile is the Schema for the ThinRuntimeProfiles API

Field Description
metadata
Kubernetes meta/v1.ObjectMeta
Refer to the Kubernetes API documentation for the fields of the metadata field.
spec
ThinRuntimeProfileSpec


fileSystemType
string

file system of thinRuntime

worker
ThinCompTemplateSpec

The component spec of worker

fuse
ThinFuseSpec

The component spec of thinRuntime

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by runtime components and/or fuses.

nodePublishSecretPolicy
NodePublishSecretPolicy

NodePublishSecretPolicy describes the policy to decide which to do with node publish secret when mounting an existing persistent volume.

status
ThinRuntimeProfileStatus

ThinRuntimeProfileSpec

(Appears on: ThinRuntimeProfile)

ThinRuntimeProfileSpec defines the desired state of ThinRuntimeProfile

Field Description
fileSystemType
string

file system of thinRuntime

worker
ThinCompTemplateSpec

The component spec of worker

fuse
ThinFuseSpec

The component spec of thinRuntime

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by runtime components and/or fuses.

nodePublishSecretPolicy
NodePublishSecretPolicy

NodePublishSecretPolicy describes the policy to decide which to do with node publish secret when mounting an existing persistent volume.

ThinRuntimeProfileStatus

(Appears on: ThinRuntimeProfile)

ThinRuntimeProfileStatus defines the observed state of ThinRuntimeProfile

ThinRuntimeSpec

(Appears on: ThinRuntime)

ThinRuntimeSpec defines the desired state of ThinRuntime

Field Description
profileName
string

The specific runtime profile name, empty value is used for handling datasets which mount another dataset

worker
ThinCompTemplateSpec

The component spec of worker

fuse
ThinFuseSpec

The component spec of thinRuntime

tieredstore
TieredStore

Tiered storage

replicas
int32

The replicas of the worker, need to be specified

runAs
User

Manage the user to run Runtime

disablePrometheus
bool
(Optional)

Disable monitoring for Runtime Prometheus is enabled by default

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by runtime components and/or fuses.

TieredStore

(Appears on: AlluxioRuntimeSpec, EFCRuntimeSpec, GooseFSRuntimeSpec, JindoRuntimeSpec, JuiceFSRuntimeSpec, ThinRuntimeSpec, VineyardRuntimeSpec)

TieredStore is a description of the tiered store

Field Description
levels
[]Level

configurations for multiple tiers

User

(Appears on: AlluxioRuntimeSpec, DataBackupSpec, DatasetSpec, GooseFSRuntimeSpec, JindoRuntimeSpec, JuiceFSRuntimeSpec, ThinRuntimeSpec)

User explains the user and group to run a Container

Field Description
uid
int64

The uid to run the alluxio runtime

gid
int64

The gid to run the alluxio runtime

user
string

The user name to run the alluxio runtime

group
string

The group name to run the alluxio runtime

VersionSpec

(Appears on: AlluxioRuntimeSpec, DataMigrateSpec, EFCCompTemplateSpec, EFCFuseSpec, GooseFSRuntimeSpec, InitFuseSpec, JindoRuntimeSpec, JuiceFSRuntimeSpec, ScriptProcessor)

VersionSpec represents the settings for the version that fluid is orchestrating.

Field Description
image
string

Image (e.g. alluxio/alluxio)

imageTag
string

Image tag (e.g. 2.3.0-SNAPSHOT)

imagePullPolicy
string

One of the three policies: Always, IfNotPresent, Never

VineyardCompTemplateSpec

(Appears on: MasterSpec, VineyardRuntimeSpec)

VineyardCompTemplateSpec is the common configurations for vineyard components including Master and Worker.

Field Description
replicas
int32
(Optional)

The replicas of Vineyard component. If not specified, defaults to 1. For worker, the replicas should not be greater than the number of nodes in the cluster

image
string
(Optional)

The image of Vineyard component. For Master, the default image is bitnami/etcd For Worker, the default image is vineyardcloudnative/vineyardd The default container registry is docker.io, you can change it by setting the image field

imageTag
string
(Optional)

The image tag of Vineyard component. For Master, the default image tag is 3.5.10. For Worker, the default image tag is v0.21.5.

imagePullPolicy
string
(Optional)

The image pull policy of Vineyard component. Default is IfNotPresent.

nodeSelector
map[string]string
(Optional)

NodeSelector is a selector to choose which nodes to launch the Vineyard component. E,g. {“disktype”: “ssd”}

ports
map[string]int
(Optional)

Ports used by Vineyard component. For Master, the default client port is 2379 and peer port is 2380. For Worker, the default rpc port is 9600 and the default exporter port is 9144.

env
map[string]string
(Optional)

Environment variables that will be used by Vineyard component. For Master, refer to Etcd Configuration for more info Default is not set.

options
map[string]string
(Optional)

Configurable options for Vineyard component. For Master, there is no configurable options. For Worker, support the following options.

vineyardd.reserve.memory: (Bool) where to reserve memory for vineyardd If set to true, the memory quota will be counted to the vineyardd rather than the application. etcd.prefix: (String) the prefix of etcd key for vineyard objects wait.etcd.timeout: (String) the timeout period before waiting the etcd to be ready, in seconds

Default value is as follows.

vineyardd.reserve.memory: "true"
etcd.prefix: "/vineyard"
wait.etcd.timeout: "120"
resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources contains the resource requirements and limits for the Vineyard component. Default is not set. For Worker, when the options contains vineyardd.reserve.memory=true, the resources.request.memory for worker should be greater than tieredstore.levels[0].quota(aka vineyardd shared memory)

volumeMounts
[]Kubernetes core/v1.VolumeMount
(Optional)

VolumeMounts specifies the volumes listed in “.spec.volumes” to mount into the vineyard runtime component’s filesystem. It is useful for specifying a persistent storage. Default is not set.

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to Vineyard’s pods including Master and Worker. Default is not set.

networkMode
NetworkMode
(Optional)

Whether to use hostnetwork or not Default is HostNetwork

VineyardRuntimeSpec

(Appears on: VineyardRuntime)

VineyardRuntimeSpec defines the desired state of VineyardRuntime

Field Description
master
MasterSpec
(Optional)

Master holds the configurations for Vineyard Master component Represents the Etcd component in Vineyard

worker
VineyardCompTemplateSpec
(Optional)

Worker holds the configurations for Vineyard Worker component Represents the Vineyardd component in Vineyard

replicas
int32

The replicas of the worker, need to be specified If worker.replicas and the field are both specified, the field will be respected

fuse
VineyardSockSpec
(Optional)

Fuse holds the configurations for Vineyard client socket. Note that the “Fuse” here is kept just for API consistency, VineyardRuntime mount a socket file instead of a FUSE filesystem to make data cache available. Applications can connect to the vineyard runtime components through IPC or RPC. IPC is the default way to connect to vineyard runtime components, which is more efficient than RPC. If the socket file is not mounted, the connection will fall back to RPC.

tieredstore
TieredStore
(Optional)

Tiered storage used by vineyardd The MediumType can only be MEM and SSD MEM actually represents the shared memory of vineyardd. SSD represents the external storage of vineyardd. Default is as follows. tieredstore: levels: - level: 0 mediumtype: MEM quota: 4Gi

Choose hostpath as the external storage of vineyardd. tieredstore: levels: - level: 0 mediumtype: MEM quota: 4Gi high: “0.8” low: “0.3” - level: 1 mediumtype: SSD quota: 10Gi volumeType: Hostpath path: /var/spill-path

disablePrometheus
bool
(Optional)

Disable monitoring metrics for Vineyard Runtime Default is false

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to Vineyard’s pods.

volumes
[]Kubernetes core/v1.Volume
(Optional)

Volumes is the list of Kubernetes volumes that can be mounted by the vineyard components (Master and Worker). Default is null.

VineyardSockSpec

(Appears on: VineyardRuntimeSpec)

VineyardSockSpec holds the configurations for vineyard client socket

Field Description
image
string
(Optional)

Image for Vineyard Fuse Default is vineyardcloudnative/vineyard-fluid-fuse

imageTag
string
(Optional)

Image Tag for Vineyard Fuse Default is v0.21.5

imagePullPolicy
string
(Optional)

Image pull policy for Vineyard Fuse Default is IfNotPresent Available values are Always, IfNotPresent, Never

env
map[string]string
(Optional)

Environment variables that will be used by Vineyard Fuse. Default is not set.

cleanPolicy
FuseCleanPolicy
(Optional)

CleanPolicy decides when to clean Vineyard Fuse pods. Currently Fluid supports two policies: OnDemand and OnRuntimeDeleted OnDemand cleans fuse pod once th fuse pod on some node is not needed OnRuntimeDeleted cleans fuse pod only when the cache runtime is deleted Defaults to OnRuntimeDeleted

resources
Kubernetes core/v1.ResourceRequirements
(Optional)

Resources contains the resource requirements and limits for the Vineyard Fuse. Default is not set.

networkMode
NetworkMode
(Optional)

Whether to use hostnetwork or not Default is HostNetwork

podMetadata
PodMetadata
(Optional)

PodMetadata defines labels and annotations that will be propagated to Vineyard’s pods.

VolumeSource

(Appears on: Level)

VolumeSource defines volume source and volume claim template.

Field Description
VolumeSource
Kubernetes core/v1.VolumeSource

(Members of VolumeSource are embedded into this type.)

WaitingStatus

(Appears on: OperationStatus)

Field Description
operationComplete
bool

OperationComplete indicates if the preceding operation is complete


Generated with gen-crd-api-reference-docs on git commit acb0aec73.