diff --git a/docs/en/administration/fault_diagnosis_and_analysis.md b/docs/en/administration/fault_diagnosis_and_analysis.md index c7ab6bb0821c..2dfae017d7f9 100644 --- a/docs/en/administration/fault_diagnosis_and_analysis.md +++ b/docs/en/administration/fault_diagnosis_and_analysis.md @@ -168,7 +168,7 @@ $ tree ./debug │ ├── stats.5s.txt │ ├── stats.txt │ └── system-info.log -└── tmp-test1-20230609104324.zip +└── tmp-test1-20230609104324.zip ``` ## Real-time performance monitoring {#performance-monitor} diff --git a/docs/en/guide/cache_management.md b/docs/en/guide/cache_management.md index eec9535d73d3..9baf6c163dfe 100644 --- a/docs/en/guide/cache_management.md +++ b/docs/en/guide/cache_management.md @@ -28,7 +28,7 @@ As for object storage, JuiceFS clients split files into data blocks (default 4Mi As a userspace filesystem, JuiceFS metadata cache is both managed as kernel cache (via FUSE API), and maintained in client memory space. -### Metadata Cache in Kernel {#kernel-metadata-cache} +### Metadata cache in kernel {#kernel-metadata-cache} JuiceFS Client controls these kinds of metadata as kernel cache: attribute (file name, size, permission, mtime, etc.), entry (inode, name, and type. The word "entry" and "dir-entry" is used in parameter names, to further distinguish between file and directory). Use the following parameters to control TTL through FUSE: @@ -49,7 +49,7 @@ Do note that `entry` cache is gradually built upon file access and may not conta Real world scenarios scarcely require setting different values for `--entry-cache` and `--dir-entry-cache`, these options exist for theoretical possibilities like when directories seldomly change while files change a lot, in that situation, you can use a higher `--dir-entry-cache` than `--entry-cache`. -### Metadata Cache in Client Memory {#client-memory-metadata-cache} +### Metadata cache in client memory {#client-memory-metadata-cache} When JuiceFS Client `open` a file, its file attributes are cached in client memory, this attribute cache includes not only the kernel cached file attributes like size, mtime, but also information specific to JuiceFS like [the relationship between file and chunks and slices](../introduction/architecture.md#how-juicefs-store-files). @@ -61,7 +61,7 @@ With `--open-cache` enabled, JuiceFS no longer operates under close-to-open cons In comparison, JuiceFS Enterprise Edition provides richer functionalities around memory metadata cache (supports active invalidation). Read [Enterprise Edition documentation](https://juicefs.com/docs/cloud/guide/cache/#client-memory-metadata-cache) for more. -## Data cache +## Data cache {#data-cache} To improve performance, JuiceFS also provides various caching mechanisms for data, including page cache in the kernel, local file system cache in client host, and read/write buffer in client process itself. Read requests will try the kernel page cache, the client process buffer, and the local disk cache in turn. If the data requested is not found in any level of the cache, it will be read from the object storage, and also be written into every level of the cache asynchronously to improve the performance of the next access. @@ -75,7 +75,7 @@ If you wish to improve write speed, and have already increased [`--max-uploads`] The `--buffer-size` also controls the data upload size for each `flush` operation, this means for clients working in a low bandwidth environment, you may need to use a lower `--buffer-size` to avoid `flush` timeouts. Refer to ["Connection problems with object storage"](../administration/troubleshooting.md#io-error-object-storage) for troubleshooting under low internet speed. -### Kernel page cache +### Kernel page cache {#kernel-data-cache} Kernel will build page cache for opened files. If this file is not updated (i.e. `mtime` doesn't change) afterwards, it will be read directly from the page cache to achieve the best performance. @@ -83,13 +83,13 @@ JuiceFS Client tracks a list of recently opened files. If file is opened again, Repeated reads of the same file in JuiceFS can be extremely fast, with latencies as low as a few microseconds and throughput up to several GiBs per second. -### Kernel writeback-cache mode +### Kernel writeback-cache mode {#fuse-writeback-cache} Starting from Linux kernel 3.15, FUSE supports [writeback-cache](https://www.kernel.org/doc/Documentation/filesystems/fuse-io.txt) mode, the kernel will consolidate high-frequency random small (10-100 bytes) write requests to significantly improve its performance, but this comes with a side effect: sequential writes are also turned into random writes, hence sequential write performance is hindered, so only use it on intensive random write scenarios. To enable writeback-cache mode, use the [`-o writeback_cache`](../reference/fuse_mount_options.md#writeback_cache) option when you [mount JuiceFS](../reference/command_reference.md#mount). Note that writeback-cache mode is not the same as [Client write data cache](#writeback), the former is a kernel implementation while the latter happens inside the JuiceFS Client, read the corresponding section to learn their intended scenarios. -### Read Cache in Client {#client-read-cache} +### Read cache in client {#client-read-cache} The client will perform prefetch and cache automatically to improve sequence read performance according to the read mode in the application. Data will be cached in local file system, which can be any local storage device like HDD, SSD or even memory. diff --git a/docs/en/guide/how_to_set_up_object_storage.md b/docs/en/guide/how_to_set_up_object_storage.md index 2ee26a2cc32f..b71419f45566 100644 --- a/docs/en/guide/how_to_set_up_object_storage.md +++ b/docs/en/guide/how_to_set_up_object_storage.md @@ -55,7 +55,7 @@ juicefs format --storage s3 \ After executing the above command, the JuiceFS client will create 4 buckets named `myjfs-0`, `myjfs-1`, `myjfs-2`, and `myjfs-3`. -## Access Key and Secret Key +## Access Key and Secret Key {#aksk} In general, object storages are authenticated with Access Key ID and Access Key Secret. For JuiceFS file system, they are provided by options `--access-key` and `--secret-key` (or AK, SK for short). @@ -72,7 +72,7 @@ juicefs format --storage s3 \ Public clouds typically allow users to create IAM (Identity and Access Management) roles, such as [AWS IAM role](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles.html) or [Alibaba Cloud RAM role](https://www.alibabacloud.com/help/doc-detail/110376.htm), which can be assigned to VM instances. If the cloud server instance already has read and write access to the object storage, there is no need to specify `--access-key` and `--secret-key`. -## Use temporary access credentials +## Use temporary access credentials {#session-token} Permanent access credentials generally have two parts, Access Key, Secret Key, while temporary access credentials generally include three parts, Access Key, Secret Key and token, and temporary access credentials have an expiration time, usually between a few minutes and a few hours. diff --git a/docs/en/reference/command_reference.md b/docs/en/reference/command_reference.md index 8e032a0c6e1f..00fdf8477be5 100644 --- a/docs/en/reference/command_reference.md +++ b/docs/en/reference/command_reference.md @@ -2,17 +2,15 @@ title: Command Reference sidebar_position: 1 slug: /command_reference -description: This article provides descriptions, usage and examples of all commands and options included in JuiceFS. +description: Descriptions, usage and examples of all commands and options included in JuiceFS Client. --- import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -## Overview +Running `juicefs` by itself and it will print all available commands. In addition, you can add `-h/--help` flag after each command to get more information, e.g., `juicefs format -h`. -If you run `juicefs` by itself, it will print all available commands. In addition, you can add `-h/--help` flag after each command to get more information of it, e.g., `juicefs format -h`. - -```bash +``` NAME: juicefs - A POSIX file system built on Redis and object storage. @@ -20,7 +18,7 @@ USAGE: juicefs [global options] command [command options] [arguments...] VERSION: - 1.1.0-beta1+2023-06-08.5ef17ba0 + 1.1.0 COMMANDS: ADMIN: @@ -68,11 +66,7 @@ COPYRIGHT: Apache License 2.0 ``` -:::note -If the command option is of boolean type, such as `--debug`, there is no need to set any value, just add `--debug` to the command to enable the function; this function is disabled if `--debug` is not added. -::: - -## Auto Completion +## Auto completion {#auto-completion} To enable commands completion, simply source the script provided within [`hack/autocomplete`](https://github.com/juicedata/juicefs/tree/main/hack/autocomplete) directory. For example: @@ -115,373 +109,259 @@ echo "source path/to/zsh_autocomplete" >> ~/.zshrc Alternatively, if you are using bash on a Linux system, you may just copy the script to `/etc/bash_completion.d` and rename it to `juicefs`: ```shell -sudo cp hack/autocomplete/bash_autocomplete /etc/bash_completion.d/juicefs +cp hack/autocomplete/bash_autocomplete /etc/bash_completion.d/juicefs source /etc/bash_completion.d/juicefs ``` -## Commands - -### ADMIN +## Admin {#admin} -#### `juicefs format` {#format} +### `juicefs format` {#format} -Create a file system, if a volume already exists with the same `META-URL`, this command will skip the creation step. To adjust volume settings afterwards, use [`juicefs config`](#config). +Create and format a file system, if a volume already exists with the same `META-URL`, this command will skip the format step. To adjust configurations for existing volumes, use [`juicefs config`](#config). -##### Synopsis +#### Synopsis -``` +```shell juicefs format [command options] META-URL NAME -``` - -- **META-URL**: Database URL for metadata storage, see "[JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md)" for details. -- **NAME**: the name of the file system - -##### Options - -###### General - -`--force`
-overwrite existing format (default: false) - -`--no-update`
-don't update existing volume (default: false) - -###### Data Storage - -`--storage value`
-Object storage type (e.g. `s3`, `gcs`, `oss`, `cos`) (default: `"file"`, please refer to [documentation](../guide/how_to_set_up_object_storage.md#supported-object-storage) for all supported object storage types) - -`--bucket value`
-A bucket URL to store data (default: `"$HOME/.juicefs/local"` or `"/var/jfs"`) - -`--access-key value`
-Access Key for object storage (can also be set via the environment variable `ACCESS_KEY`) - -`--secret-key value`
-Secret Key for object storage (can also be set via the environment variable `SECRET_KEY`) - -`--session-token value`
-session token for object storage - -`--storage-class value`
-the default storage class - -###### Data Format - -`--block-size value`
-size of block in KiB (default: 4096). 4M is usually a better default value because many object storage services use 4M as their internal block size, thus using the same block size in JuiceFS usually yields better performance - -`--compress value`
-compression algorithm, choose from `lz4`, `zstd`, `none` (default: "none"). Enabling compression will inevitably affect performance, choose wisely - -`--encrypt-rsa-key value`
-A path to RSA private key (PEM) - -`--encrypt-algo value`
-encrypt algorithm (aes256gcm-rsa, chacha20-rsa) (default: "aes256gcm-rsa") - -`--hash-prefix`
-add a hash prefix to name of objects (default: false) - -`--shards value`
-store the blocks into N buckets by hash of key (default: 0), when N is greater than 0, `bucket` should to be in the form of `%d`, e.g. `--bucket "juicefs-%d"` - -###### Management - -`--capacity value`
-storage space limit in GiB, set to 0 disable limit (default: 0). Capacity will include trash files, if trash is enabled - -`--inodes value`
-the limit for number of inodes (0 means unlimited) (default: 0) - -`--trash-days value`
-number of days after which removed files will be permanently deleted (default: 1) -##### Examples - -```bash # Create a simple test volume (data will be stored in a local directory) -$ juicefs format sqlite3://myjfs.db myjfs +juicefs format sqlite3://myjfs.db myjfs # Create a volume with Redis and S3 -$ juicefs format redis://localhost myjfs --storage s3 --bucket https://mybucket.s3.us-east-2.amazonaws.com +juicefs format redis://localhost myjfs --storage=s3 --bucket=https://mybucket.s3.us-east-2.amazonaws.com # Create a volume with password protected MySQL -$ juicefs format mysql://jfs:mypassword@(127.0.0.1:3306)/juicefs myjfs +juicefs format mysql://jfs:mypassword@(127.0.0.1:3306)/juicefs myjfs # A safer alternative -$ META_PASSWORD=mypassword juicefs format mysql://jfs:@(127.0.0.1:3306)/juicefs myjfs - -# Create a volume with "quota" enabled -$ juicefs format sqlite3://myjfs.db myjfs --inode 1000000 --capacity 102400 - -# Create a volume with "trash" disabled -$ juicefs format sqlite3://myjfs.db myjfs --trash-days 0 -``` +META_PASSWORD=mypassword juicefs format mysql://jfs:@(127.0.0.1:3306)/juicefs myjfs -#### `juicefs config` {#config} +# Create a volume with quota enabled +juicefs format sqlite3://myjfs.db myjfs --inode=1000000 --capacity=102400 -Change config of a volume. Note that after updating some settings, the client may not take effect immediately, and it needs to wait for a certain period of time. The specific waiting time can be controlled by the [`--heartbeat`](#mount) option. - -##### Synopsis - -``` -juicefs config [command options] META-URL +# Create a volume with trash disabled +juicefs format sqlite3://myjfs.db myjfs --trash-days=0 ``` -##### Options - -###### General - -`--yes, -y`
-automatically answer 'yes' to all prompts and run non-interactively (default: false) - -`--force`
-skip sanity check and force update the configurations (default: false) - -###### Data Storage - -`--storage value`
-Object storage type (e.g. `s3`, `gcs`, `oss`, `cos`) (default: `"file"`, please refer to [documentation](../guide/how_to_set_up_object_storage.md#supported-object-storage) for all supported object storage types) - -`--bucket value`
-a bucket URL to store data - -`--access-key value`
-access key for object storage - -`--secret-key value`
-secret key for object storage - -`--session-token value`
-session token for object storage - -`--storage-class value`
-the default storage class +#### Options -`--upload-limit value`
-bandwidth limit for upload in Mbps (default: 0) +|Items|Description| +|-|-| +|`META-URL`|Database URL for metadata storage, see [JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md) for details.| +|`NAME`|Name of the file system| +|`--force`|overwrite existing format (default: false)| +|`--no-update`|don't update existing volume (default: false)| -`--download-limit value`
-bandwidth limit for download in Mbps (default: 0) +#### Data storage options {#format-data-storage-options} -###### Management +|Items|Description| +|-|-| +|`--storage=file`|Object storage type (e.g. `s3`, `gcs`, `oss`, `cos`) (default: `file`, refer to [documentation](../guide/how_to_set_up_object_storage.md#supported-object-storage) for all supported object storage types)| +|`--bucket=/var/jfs`|A bucket URL to store data (default: `$HOME/.juicefs/local` or `/var/jfs`)| +|`--access-key=value`|Access Key for object storage (can also be set via the environment variable `ACCESS_KEY`), see [How to Set Up Object Storage](../guide/how_to_set_up_object_storage.md#aksk) for more.| +|`--secret-key value`|Secret Key for object storage (can also be set via the environment variable `SECRET_KEY`), see [How to Set Up Object Storage](../guide/how_to_set_up_object_storage.md#aksk) for more.| +|`--session-token=value`|session token for object storage, see [How to Set Up Object Storage](../guide/how_to_set_up_object_storage.md#session-token) for more.| +|`--storage-class value`|the default storage class| -`--capacity value`
-limit for space in GiB +#### Data format options {#format-data-format-options} -`--inodes value`
-limit for number of inodes +|Items|Description| +|-|-| +|`--block-size=4096`|size of block in KiB (default: 4096). 4M is usually a better default value because many object storage services use 4M as their internal block size, thus using the same block size in JuiceFS usually yields better performance.| +|`--compress=none`|compression algorithm, choose from `lz4`, `zstd`, `none` (default). Enabling compression will inevitably affect performance, choose wisely.| +|`--encrypt-rsa-key=value`|A path to RSA private key (PEM)| +|`--encrypt-algo=aes256gcm-rsa`|encrypt algorithm (aes256gcm-rsa, chacha20-rsa) (default: "aes256gcm-rsa")| +|`--hash-prefix`|add a hash prefix to name of objects (default: false)| +|`--shards=0`|store the blocks into N buckets by hash of key (default: 0), when N is greater than 0, `bucket` should to be in the form of `%d`, e.g. `--bucket "juicefs-%d"`| -`--trash-days value`
-number of days after which removed files will be permanently deleted +#### Management options {#format-management-options} -`--encrypt-secret`
-encrypt the secret key if it was previously stored in plain format (default: false) +|Items|Description| +|-|-| +|`--capacity=0`|storage space limit in GiB, default to 0 which means no limit. Capacity will include trash files, if [trash](../security/trash.md) is enabled.| +|`--inodes=0`|Limit the number of inodes, default to 0 which means no limit.| +|`--trash-days=1`|By default, delete files are put into [trash](../security/trash.md), this option controls the number of days before trash files are expired, default to 1, set to 0 to disable trash.| -`--min-client-version value`
-minimum client version allowed to connect +### `juicefs config` {#config} -`--max-client-version value`
-maximum client version allowed to connect +Change config of a volume. Note that after updating some settings, the client may not take effect immediately, and it needs to wait for a certain period of time. The specific waiting time can be controlled by the [`--heartbeat`](#mount-metadata-options) option. -`--dir-stats`
-enable dir stats, which is necessary for fast summary and dir quota (default: false) +#### Synopsis -#### Examples +```shell +juicefs config [command options] META-URL -```bash # Show the current configurations -$ juicefs config redis://localhost +juicefs config redis://localhost # Change volume "quota" -$ juicefs config redis://localhost --inode 10000000 --capacity 1048576 +juicefs config redis://localhost --inode 10000000 --capacity 1048576 # Change maximum days before files in trash are deleted -$ juicefs config redis://localhost --trash-days 7 +juicefs config redis://localhost --trash-days 7 # Limit client version that is allowed to connect -$ juicefs config redis://localhost --min-client-version 1.0.0 --max-client-version 1.1.0 +juicefs config redis://localhost --min-client-version 1.0.0 --max-client-version 1.1.0 ``` -#### `juicefs quota`{#quota} +#### Options + +|Items|Description| +|-|-| +|`--yes, -y`|automatically answer 'yes' to all prompts and run non-interactively (default: false)| +|`--force`|skip sanity check and force update the configurations (default: false)| + +#### Data storage options {#config-data-storage-options} + +|Items|Description| +|-|-| +|`--storage=file`|Object storage type (e.g. `s3`, `gcs`, `oss`, `cos`) (default: `"file"`, refer to [documentation](../guide/how_to_set_up_object_storage.md#supported-object-storage) for all supported object storage types).| +|`--bucket=/var/jfs`|A bucket URL to store data (default: `$HOME/.juicefs/local` or `/var/jfs`)| +|`--access-key=value`|Access Key for object storage (can also be set via the environment variable `ACCESS_KEY`), see [How to Set Up Object Storage](../guide/how_to_set_up_object_storage.md#aksk) for more.| +|`--secret-key value`|Secret Key for object storage (can also be set via the environment variable `SECRET_KEY`), see [How to Set Up Object Storage](../guide/how_to_set_up_object_storage.md#aksk) for more.| +|`--session-token=value`|session token for object storage, see [How to Set Up Object Storage](../guide/how_to_set_up_object_storage.md#session-token) for more.| +|`--storage-class value`|the default storage class| +|`--upload-limit=0`|bandwidth limit for upload in Mbps (default: 0)| +|`--download-limit=0`|bandwidth limit for download in Mbps (default: 0)| + +#### Management options {#config-management-options} + +|Items|Description| +|-|-| +|`--capacity value`|limit for space in GiB| +|`--inodes value`|limit for number of inodes| +|`--trash-days value`|number of days after which removed files will be permanently deleted| +|`--encrypt-secret`|encrypt the secret key if it was previously stored in plain format (default: false)| +|`--min-client-version value`|minimum client version allowed to connect| +|`--max-client-version value`|maximum client version allowed to connect| +|`--dir-stats`|enable dir stats, which is necessary for fast summary and dir quota (default: false)| + +### `juicefs quota`{#quota} Manage directory quotas -##### Synopsis +#### Synopsis ```shell juicefs quota command [command options] META-URL -``` - -- `META-URL`: Database URL for metadata storage, see "[JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md)" for details. - -##### Subcommands - -`set`
-Set quota to a directory - -`get`
-Get quota of a directory - -`delete, del`
-Delete quota of a directory -`list, ls`
-List all directory quotas - -`check`
-Check quota consistency of a directory - -##### Options - -`--path value`
-full path of the directory within the volume +# Set quota to a directory +juicefs quota set redis://localhost --path /dir1 --capacity 1 --inodes 100 -`--capacity value`
-hard quota of the directory limiting its usage of space in GiB (default: 0) +# Get quota of a directory +juicefs quota get redis://localhost --path /dir1 -`--inodes value`
-hard quota of the directory limiting its number of inodes (default: 0) +# List all directory quotas +juicefs quota list redis://localhost -`--repair`
-repair inconsistent quota (default: false) +# Delete quota of a directory +juicefs quota delete redis://localhost --path /dir1 -`--strict`
-calculate total usage of directory in strict mode (NOTE: may be slow for huge directory) (default: false) +# Check quota consistency of a directory +juicefs quota check redis://localhost +``` -##### Examples +#### Options -```shell -juicefs quota set redis://localhost --path /dir1 --capacity 1 --inodes 100 -juicefs quota get redis://localhost --path /dir1 -juicefs quota list redis://localhost -juicefs quota delete redis://localhost --path /dir1 -``` +|Items|Description| +|-|-| +|`META-URL`|Database URL for metadata storage, see "[JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md)" for details.| +|`--path value`|full path of the directory within the volume| +|`--capacity value`|hard quota of the directory limiting its usage of space in GiB (default: 0)| +|`--inodes value`|hard quota of the directory limiting its number of inodes (default: 0)| +|`--repair`|repair inconsistent quota (default: false)| +|`--strict`|calculate total usage of directory in strict mode (NOTE: may be slow for huge directory) (default: false)| -#### `juicefs destroy`{#destroy} +### `juicefs destroy` {#destroy} Destroy an existing volume, will delete relevant data in metadata engine and object storage. See [How to destroy a file system](../administration/destroy.md). -##### Synopsis +#### Synopsis -``` +```shell juicefs destroy [command options] META-URL UUID -``` - -##### Options -`--yes, -y`
-automatically answer 'yes' to all prompts and run non-interactively (default: false) - -`--force`
-skip sanity check and force destroy the volume (default: false) - -##### Examples - -```bash juicefs destroy redis://localhost e94d66a8-2339-4abd-b8d8-6812df737892 ``` -#### `juicefs gc` {#gc} - -Deal with leaked objects, and garbage fragments produced by file overwrites. See [Status Check & Maintenance](../administration/status_check_and_maintenance.md#gc). - -##### Synopsis - -``` -juicefs gc [command options] META-URL -``` +#### Options -##### Options +|Items|Description| +|-|-| +|`--yes, -y`|automatically answer 'yes' to all prompts and run non-interactively (default: false)| +|`--force`|skip sanity check and force destroy the volume (default: false)| -`--delete`
-delete leaked objects (default: false) +### `juicefs gc` {#gc} -`--compact`
-compact all chunks with more than 1 slices (default: false). +Deal with leaked objects, and garbage fragments produced by file overwrites. See [Status Check & Maintenance](../administration/status_check_and_maintenance.md#gc). -`--threads value`
-number of threads to delete leaked objects (default: 10) +#### Synopsis -##### Examples +```shell +juicefs gc [command options] META-URL -```bash # Check only, no writable change -$ juicefs gc redis://localhost +juicefs gc redis://localhost # Trigger compaction of all slices -$ juicefs gc redis://localhost --compact +juicefs gc redis://localhost --compact # Delete leaked objects -$ juicefs gc redis://localhost --delete +juicefs gc redis://localhost --delete ``` -#### `juicefs fsck` {#fsck} - -Check consistency of file system. - -##### Synopsis - -``` -juicefs fsck [command options] META-URL -``` - -##### Options +#### Options -`--path value`
-absolute path within JuiceFS to check +|Items|Description| +|-|-| +|`--delete`|delete leaked objects (default: false)| +|`--compact`|compact all chunks with more than 1 slices (default: false).| +|`--threads=10`|number of threads to delete leaked objects (default: 10)| -`--repair`
-repair specified path if it's broken (default: false) +### `juicefs fsck` {#fsck} -`--recursive, -r`
-recursively check or repair (default: false) +Check consistency of file system. -`--sync-dir-stat`
-sync stat of all directories, even if they are existed and not broken (NOTE: it may take a long time for huge trees) (default: false) +#### Synopsis -##### Examples +```shell +juicefs fsck [command options] META-URL -```bash juicefs fsck redis://localhost ``` -#### `juicefs restore` {#restore} +#### Options + +|Items|Description| +|-|-| +|`--path value`|absolute path within JuiceFS to check| +|`--repair`|repair specified path if it's broken (default: false)| +|`--recursive, -r`|recursively check or repair (default: false)| +|`--sync-dir-stat`|sync stat of all directories, even if they are existed and not broken (NOTE: it may take a long time for huge trees) (default: false)| + +### `juicefs restore` {#restore} Rebuild the tree structure for trash files, and put them back to original directories. -##### Synopsis +#### Synopsis ```shell juicefs restore [command options] META HOUR ... -``` - -##### Options - -`--put-back value`
-move the recovered files into original directory (default: false) - -`--threads value`
-number of threads (default: 10) -##### Examples - -```shell juicefs restore redis://localhost/1 2023-05-10-01 ``` -#### `juicefs dump` {#dump} +#### Options -Dump metadata into a JSON file. Refer to ["Metadata backup"](../administration/metadata_dump_load.md#backup) for more information. +|Items|Description| +|-|-| +|`--put-back value`|move the recovered files into original directory (default: false)| +|`--threads value`|number of threads (default: 10)| -##### Synopsis +### `juicefs dump` {#dump} -``` -juicefs dump [command options] META-URL [FILE] -``` +Dump metadata into a JSON file. Refer to ["Metadata backup"](../administration/metadata_dump_load.md#backup) for more information. -- META-URL: Database URL for metadata storage, see ["Metadata Engines Supported by JuiceFS"](../guide/how_to_set_up_metadata_engine.md) for details. -- FILE: Export file path, if not specified, it will be exported to standard output. If the filename ends with `.gz`, it will be automatically compressed. +#### Synopsis ```shell juicefs dump [command options] META-URL [FILE] @@ -493,26 +373,20 @@ juicefs dump redis://localhost meta-dump.json juicefs dump redis://localhost sub-meta-dump.json --subdir /dir/in/jfs ``` -##### Options - -`--subdir value`
-Only export metadata for the specified subdirectory. +#### Options -`--keep-secret-key`
-Export object storage authentication information, the default is `false`. Since it is exported in plain text, pay attention to data security when using it. If the export file does not contain object storage authentication information, you need to use [`juicefs config`](#config) to reconfigure object storage authentication information after the subsequent import is completed. +|Items|Description| +|-|-| +|`META-URL`|Database URL for metadata storage, see [JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md) for details.| +|`FILE`|Export file path, if not specified, it will be exported to standard output. If the filename ends with `.gz`, it will be automatically compressed.| +|`--subdir=path`|Only export metadata for the specified subdirectory.| +|`--keep-secret-key`|Export object storage authentication information, the default is `false`. Since it is exported in plain text, pay attention to data security when using it. If the export file does not contain object storage authentication information, you need to use [`juicefs config`](#config) to reconfigure object storage authentication information after the subsequent import is completed.| -#### `juicefs load` {#load} +### `juicefs load` {#load} Load metadata from a previously dumped JSON file. Read ["Metadata recovery and migration"](../administration/metadata_dump_load.md#recovery-and-migration) to learn more. -##### Synopsis - -``` -juicefs load [command options] META-URL [FILE] -``` - -- META-URL: Database URL for metadata storage, see ["Metadata Engines Supported by JuiceFS"](../guide/how_to_set_up_metadata_engine.md) for details. -- FILE: Import file path, if not specified, it will be imported from standard input. If the filename ends with `.gz`, it will be automatically decompressed. +#### Synopsis ```shell juicefs load [command options] META-URL [FILE] @@ -521,1078 +395,569 @@ juicefs load [command options] META-URL [FILE] juicefs load redis://127.0.0.1:6379/1 meta-dump.json ``` -##### Options - -`--encrypt-rsa-key value`
-The path to the RSA private key file used for encryption. +#### Options -`--encrypt-alg value`
-Encryption algorithm, the default is `aes256gcm-rsa`. +|Items|Description| +|-|-| +|`META-URL`|Database URL for metadata storage, see [JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md) for details.| +|`FILE`|Import file path, if not specified, it will be imported from standard input. If the filename ends with `.gz`, it will be automatically decompressed.| +|`--encrypt-rsa-key=path`|The path to the RSA private key file used for encryption.| +|`--encrypt-alg=aes256gcm-rsa`|Encryption algorithm, the default is `aes256gcm-rsa`.| -### INSPECTOR +## Inspector {#inspector} -#### `juicefs status` {#status} +### `juicefs status` {#status} Show status of JuiceFS. -##### Synopsis +#### Synopsis -``` +```shell juicefs status [command options] META-URL -``` - -##### Options - -`--session value, -s value`
-show detailed information (sustained inodes, locks) of the specified session (SID) (default: 0) -`--more, -m`
-show more statistic information, may take a long time (default: false) - -##### Examples - -```bash juicefs status redis://localhost ``` -#### `juicefs stats` {#stats} - -Show runtime statistics. - -##### Synopsis - -``` -juicefs stats [command options] MOUNTPOINT -``` +#### Options -##### Options +|Items|Description| +|-|-| +|`--session=0, -s 0`|show detailed information (sustained inodes, locks) of the specified session (SID) (default: 0)| +|`--more, -m`|show more statistic information, may take a long time (default: false)| -`--schema value`
-schema string that controls the output sections (u: `usage`, f: `fuse`, m: `meta`, c: `blockcache`, o: `object`, g: `go`) (default: "ufmco") +### `juicefs stats` {#stats} -`--interval value`
-interval in seconds between each update (default: 1) +Show runtime statistics, read [Real-time performance monitoring](../administration/fault_diagnosis_and_analysis.md#performance-monitor) for more. -`--verbosity value`
-verbosity level, 0 or 1 is enough for most cases (default: 0) +#### Synopsis -##### Examples +```shell +juicefs stats [command options] MOUNTPOINT -```bash -$ juicefs stats /mnt/jfs +juicefs stats /mnt/jfs # More metrics -$ juicefs stats /mnt/jfs -l 1 +juicefs stats /mnt/jfs -l 1 ``` -#### `juicefs profile` {#profile} - -Analyze [access log](../administration/fault_diagnosis_and_analysis.md#access-log). - -##### Synopsis - -``` -juicefs profile [command options] MOUNTPOINT/LOGFILE -``` - -##### Options +#### Options -`--uid value, -u value`
-only track specified UIDs (separated by comma ,) +|Items|Description| +|-|-| +|`--schema=ufmco`|schema string that controls the output sections (`u`: usage, `f`: FUSE, `m`: metadata, `c`: block cache, `o`: object storage, `g`: Go) (default: `ufmco`)| +|`--interval=1`|interval in seconds between each update (default: 1)| +|`--verbosity=0`|verbosity level, 0 or 1 is enough for most cases (default: 0)| -`--gid value, -g value`
-only track specified GIDs(separated by comma ,) +### `juicefs profile` {#profile} -`--pid value, -p value`
-only track specified PIDs(separated by comma ,) +Show profiling of operations completed in JuiceFS, based on [access log](../administration/fault_diagnosis_and_analysis.md#access-log). read [Real-time performance monitoring](../administration/fault_diagnosis_and_analysis.md#performance-monitor) for more. -`--interval value`
-flush interval in seconds; set it to 0 when replaying a log file to get an immediate result (default: 2) +#### Synopsis -##### Examples +```shell +juicefs profile [command options] MOUNTPOINT/LOGFILE -```bash # Monitor real time operations -$ juicefs profile /mnt/jfs +juicefs profile /mnt/jfs # Replay an access log -$ cat /mnt/jfs/.accesslog > /tmp/jfs.alog +cat /mnt/jfs/.accesslog > /tmp/jfs.alog # Press Ctrl-C to stop the "cat" command after some time -$ juicefs profile /tmp/jfs.alog +juicefs profile /tmp/jfs.alog # Analyze an access log and print the total statistics immediately -$ juicefs profile /tmp/jfs.alog --interval 0 +juicefs profile /tmp/jfs.alog --interval 0 ``` -#### `juicefs info` {#info} - -Show internal information for given paths or inodes. - -##### Synopsis - -``` -juicefs info [command options] PATH or INODE -``` - -##### Options +#### Options -`--inode, -i`
-use inode instead of path (current dir should be inside JuiceFS) (default: false) +|Items|Description| +|-|-| +|`--uid=value, -u value`|only track specified UIDs (separated by comma)| +|`--gid=value, -g value`|only track specified GIDs (separated by comma)| +|`--pid=value, -p value`|only track specified PIDs (separated by comma)| +|`--interval=2`|flush interval in seconds; set it to 0 when replaying a log file to get an immediate result (default: 2)| -`--recursive, -r`
-get summary of directories recursively (NOTE: it may take a long time for huge trees) (default: false) +### `juicefs info` {#info} -`--strict`
-get accurate summary of directories (NOTE: it may take a long time for huge trees) (default: false) +Show internal information for given paths or inodes. -`--raw`
-show internal raw information (default: false) +#### Synopsis -##### Examples +```shell +juicefs info [command options] PATH or INODE -```bash # Check a path -$ juicefs info /mnt/jfs/foo +juicefs info /mnt/jfs/foo # Check an inode -$ cd /mnt/jfs -$ juicefs info -i 100 -``` - -#### `juicefs debug` {#debug} - -It collects and displays information from multiple dimensions such as the operating environment and system logs to help better locate errors - -##### Synopsis - -``` -juicefs debug [command options] MOUNTPOINT +cd /mnt/jfs +juicefs info -i 100 ``` -##### Options - -`--out-dir value`
-The output directory of the results, automatically created if the directory does not exist (default: ./debug/) +#### Options -`--stats-sec value`
-The number of seconds to sample .stats file (default: 5) +|Items|Description| +|-|-| +|`--inode, -i`|use inode instead of path (current dir should be inside JuiceFS) (default: false)| +|`--recursive, -r`|get summary of directories recursively (NOTE: it may take a long time for huge trees) (default: false)| +|`--strict`|get accurate summary of directories (NOTE: it may take a long time for huge trees) (default: false)| +|`--raw`|show internal raw information (default: false)| -`--limit value`
-The number of log entries collected, from newest to oldest, if not specified, all entries will be collected +### `juicefs debug` {#debug} -`--trace-sec value`
-The number of seconds to sample trace metrics (default: 5) +It collects and displays information from multiple dimensions such as the operating environment and system logs to help better locate errors -`--profile-sec value`
-The number of seconds to sample profile metrics (default: 30) +#### Synopsis -##### Examples +```shell +juicefs debug [command options] MOUNTPOINT -```bash # Collect and display information about the mount point /mnt/jfs -$ juicefs debug /mnt/jfs +juicefs debug /mnt/jfs # Specify the output directory as /var/log -$ juicefs debug --out-dir=/var/log /mnt/jfs +juicefs debug --out-dir=/var/log /mnt/jfs # Get the last up to 1000 log entries -$ juicefs debug --out-dir=/var/log --limit=1000 /mnt/jfs -``` - -#### `juicefs summary` {#summary} - -It is used to show tree summary of target directory. - -##### Synopsis - -```shell -juicefs summary [command options] PATH +juicefs debug --out-dir=/var/log --limit=1000 /mnt/jfs ``` #### Options -`--depth value, -d value`
-depth of tree to show (zero means only show root) (default: 2) +|Items|Description| +|-|-| +|`--out-dir=./debug/`|The output directory of the results, automatically created if the directory does not exist (default: `./debug/`)| +|`--stats-sec=5`|The number of seconds to sample .stats file (default: 5)| +|`--limit=value`|The number of log entries collected, from newest to oldest, if not specified, all entries will be collected| +|`--trace-sec=5`|The number of seconds to sample trace metrics (default: 5)| +|`--profile-sec=30`|The number of seconds to sample profile metrics (default: 30)| -`--entries value, -e value`
-show top N entries (sort by size) (default: 10) +### `juicefs summary` {#summary} -`--strict`
-show accurate summary, including directories and files (may be slow) (default: false) - -`--csv`
-print summary in csv format (default: false) +It is used to show tree summary of target directory. -##### Examples +#### Synopsis ```shell +juicefs summary [command options] PATH + # Show with path -$ juicefs summary /mnt/jfs/foo +juicefs summary /mnt/jfs/foo # Show max depth of 5 -$ juicefs summary --depth 5 /mnt/jfs/foo +juicefs summary --depth 5 /mnt/jfs/foo # Show top 20 entries -$ juicefs summary --entries 20 /mnt/jfs/foo +juicefs summary --entries 20 /mnt/jfs/foo # Show accurate result -$ juicefs summary --strict /mnt/jfs/foo -``` - -### SERVICE - -#### `juicefs mount` {#mount} - -Mount a volume. The volume must be formatted in advance. - -You can use any user to execute the mount command, but please ensure that the user has write permission to the cache directory (`--cache-dir`), please read ["Cache directory"](../guide/cache_management.md#cache-dir) documentation for more information. - -##### Synopsis - +juicefs summary --strict /mnt/jfs/foo ``` -juicefs mount [command options] META-URL MOUNTPOINT -``` - -- `META-URL`: Database URL for metadata storage, see "[JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md)" for details. -- `MOUNTPOINT`: file system mount point, e.g. `/mnt/jfs`, `Z:`. - -##### Options - -###### General - -`-d, --background`
-run in background (default: false) - -`--no-syslog`
-disable syslog (default: false) - -`--log value`
-path of log file when running in background (default: `$HOME/.juicefs/juicefs.log` or `/var/log/juicefs.log`) - -`--force`
-skip sanity check and force update the configurations (default: false) - -`--update-fstab`
-add / update entry in `/etc/fstab`, will create a symlink from `/sbin/mount.juicefs` to JuiceFS executable if not existing (default: false) - -###### FUSE - -`--enable-xattr`
-enable extended attributes (xattr) (default: false) - -`--enable-ioctl`
-enable ioctl (support GETFLAGS/SETFLAGS only) (default: false) - -`--root-squash value`
-mapping local root user (UID = 0) to another one specified as UID:GID - -`--prefix-internal`
-add '.jfs' prefix to all internal files (default: false) - -`-o value`
-other FUSE options, see [FUSE Mount Options](../reference/fuse_mount_options.md) - -###### Meta - -`--subdir value`
-mount a sub-directory as root (default: "") - -`--backup-meta value`
-interval (in seconds) to automatically backup metadata in the object storage (0 means disable backup) (default: "3600") - -`--heartbeat value`
-interval (in seconds) to send heartbeat; it's recommended that all clients use the same heartbeat value (default: "12") - -`--read-only`
-allow lookup/read operations only (default: false) - -`--no-bgjob`
-Disable background jobs, default to false, which means clients by default carry out background jobs, including: - -* Clean up expired files in Trash (look for `cleanupDeletedFiles`, `cleanupTrash` in [`pkg/meta/base.go`](https://github.com/juicedata/juicefs/blob/main/pkg/meta/base.go)) -* Delete slices that's not referenced (look for `cleanupSlices` in [`pkg/meta/base.go`](https://github.com/juicedata/juicefs/blob/main/pkg/meta/base.go)) -* Clean up stale client sessions (look for `CleanStaleSessions` in [`pkg/meta/base.go`](https://github.com/juicedata/juicefs/blob/main/pkg/meta/base.go)) - -Note that compaction isn't affected by this option, it happens automatically with file reads and writes, client will check if compaction is in need, and run in background (take Redis for example, look for `compactChunk` in [`pkg/meta/base.go`](https://github.com/juicedata/juicefs/blob/main/pkg/meta/redis.go)). - -`--atime-mode value`
-Control atime (last time the file was accessed) behavior, support the following modes: - -* `noatime` (default), set when the file is created or when `SetAttr` is explicitly called. Accessing and modifying the file will not affect atime, tracking atime comes at a performance cost, so this is the default behavior -* `relatime` update inode access times relative to mtime (last time when the file data was modified) or ctime (last time when file metadata was changed). Only update atime if atime was earlier than the current mtime or ctime, or the file's atime is more than 1 day old -* `strictatime`, always update atime on access - -`--skip-dir-nlink value`
-number of retries after which the update of directory nlink will be skipped (used for tkv only, 0 means never) (default: 20) - -###### Meta Cache - -`--attr-cache value`
-attributes cache timeout in seconds (default: 1), read [Kernel Metadata Cache](../guide/cache_management.md#kernel-metadata-cache) - -`--entry-cache value`
-file entry cache timeout in seconds (default: 1), read [Kernel Metadata Cache](../guide/cache_management.md#kernel-metadata-cache) - -`--dir-entry-cache value`
-dir entry cache timeout in seconds (default: 1), read [Kernel Metadata Cache](../guide/cache_management.md#kernel-metadata-cache) - -`--open-cache value`
-open file cache timeout in seconds (0 means disable this feature) (default: 0) - -`--open-cache-limit value`
-max number of open files to cache (soft limit, 0 means unlimited) (default: 10000) - -###### Data Storage - -`--storage value`
-Object storage type (e.g. `s3`, `gcs`, `oss`, `cos`) (default: `"file"`, please refer to [documentation](../guide/how_to_set_up_object_storage.md#supported-object-storage) for all supported object storage types) - -`--bucket value`
-customized endpoint to access object store - -`--storage-class value`
-the storage class for data written by current client -`--get-timeout value`
-the max number of seconds to download an object (default: 60) - -`--put-timeout value`
-the max number of seconds to upload an object (default: 60) - -`--io-retries value`
-number of retries after network failure (default: 10) - -`--max-uploads value`
-number of connections to upload (default: 20) - -`--max-deletes value`
-number of threads to delete objects (default: 10) - -`--upload-limit value`
-bandwidth limit for upload in Mbps (default: 0) - -`--download-limit value`
-bandwidth limit for download in Mbps (default: 0) - -###### Data Cache - -`--buffer-size value`
-total read/write buffering in MiB (default: 300) - -`--prefetch value`
-prefetch N blocks in parallel (default: 1) - -`--writeback`
-upload objects in background (default: false), see [Client write data cache](../guide/cache_management.md#writeback) - -`--upload-delay value`
-if writeback mode is enabled, delayed duration for uploading objects ("s", "m", "h") (default: 0s) - -`--cache-dir value`
-directory paths of local cache, use `:` (Linux, macOS) or `;` (Windows) to separate multiple paths (default: `"$HOME/.juicefs/cache"` or `"/var/jfsCache"`), see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--cache-mode value`
-file permissions for cached blocks (default: "0600") - -`--cache-size value`
-size of cached object for read in MiB (default: 102400), see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--free-space-ratio value`
-min free space ratio (default: 0.1), if [Client write data cache](../guide/cache_management.md#writeback) is enabled, this option also controls write cache size, see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--cache-partial-only`
-cache random/small read only (default: false), see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--verify-cache-checksum value`
-Checksum level for cache data. After enabled, checksum will be calculated on divided parts of the cache blocks and stored on disks, which are used for verification during reads. The following strategies are supported:
+#### Options -`--cache-eviction value`
-cache eviction policy (none or 2-random) (default: "2-random") +|Items|Description| +|-|-| +|`--depth value, -d value`|depth of tree to show (zero means only show root) (default: 2)| +|`--entries value, -e value`|show top N entries (sort by size) (default: 10)| +|`--strict`|show accurate summary, including directories and files (may be slow) (default: false)| +|`--csv`|print summary in csv format (default: false)| -`--cache-scan-interval value`
-interval (in seconds) to scan cache-dir to rebuild in-memory index (default: "3600") +## Service {#service} -###### Metrics +### `juicefs mount` {#mount} -`--metrics value`
-address to export metrics (default: "127.0.0.1:9567") +Mount a volume. The volume must be formatted in advance. -`--consul value`
-Consul address to register (default: "127.0.0.1:8500") +JuiceFS can be mounted by root or normal user, but due to their privilege differences, cache directory and log path will vary, read below descriptions for more. -`--no-usage-report`
-do not send usage report (default: false) +#### Synopsis -#### Examples +```shell +juicefs mount [command options] META-URL MOUNTPOINT -```bash # Mount in foreground -$ juicefs mount redis://localhost /mnt/jfs +juicefs mount redis://localhost /mnt/jfs # Mount in background with password protected Redis -$ juicefs mount redis://:mypassword@localhost /mnt/jfs -d +juicefs mount redis://:mypassword@localhost /mnt/jfs -d # A safer alternative -$ META_PASSWORD=mypassword juicefs mount redis://localhost /mnt/jfs -d +META_PASSWORD=mypassword juicefs mount redis://localhost /mnt/jfs -d # Mount with a sub-directory as root -$ juicefs mount redis://localhost /mnt/jfs --subdir /dir/in/jfs +juicefs mount redis://localhost /mnt/jfs --subdir /dir/in/jfs # Enable "writeback" mode, which improves performance at the risk of losing objects -$ juicefs mount redis://localhost /mnt/jfs -d --writeback +juicefs mount redis://localhost /mnt/jfs -d --writeback # Enable "read-only" mode -$ juicefs mount redis://localhost /mnt/jfs -d --read-only +juicefs mount redis://localhost /mnt/jfs -d --read-only # Disable metadata backup -$ juicefs mount redis://localhost /mnt/jfs --backup-meta 0 +juicefs mount redis://localhost /mnt/jfs --backup-meta 0 ``` -#### `juicefs umount`{#umount} +#### Options + +|Items|Description| +|-|-| +|`META-URL`|Database URL for metadata storage, see [JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md) for details.| +|`MOUNTPOINT`|file system mount point, e.g. `/mnt/jfs`, `Z:`.| +|`-d, --background`|run in background (default: false)| +|`--no-syslog`|disable syslog (default: false)| +|`--log=path`|path of log file when running in background (default: `$HOME/.juicefs/juicefs.log` or `/var/log/juicefs.log`)| +|`--update-fstab`|add / update entry in `/etc/fstab`, will create a symlink from `/sbin/mount.juicefs` to JuiceFS executable if not existing (default: false)| + +#### FUSE related options {#mount-fuse-options} + +|Items|Description| +|-|-| +|`--enable-xattr`|enable extended attributes (xattr) (default: false)| +|`-o value`|other FUSE options, see [FUSE Mount Options](../reference/fuse_mount_options.md)| +|`--enable-ioctl`|enable ioctl (support GETFLAGS/SETFLAGS only) (default: false)| +|`--root-squash value`|mapping local root user (UID = 0) to another one specified as UID:GID| +|`--prefix-internal`|add '.jfs' prefix to all internal files (default: false)| + +#### Metadata related options {#mount-metadata-options} + +|Items|Description| +|-|-| +|`--subdir=value`|mount a sub-directory as root (default: "")| +|`--backup-meta=3600`|interval (in seconds) to automatically backup metadata in the object storage (0 means disable backup) (default: "3600")| +|`--heartbeat=12`|interval (in seconds) to send heartbeat; it's recommended that all clients use the same heartbeat value (default: "12")| +|`--read-only`|allow lookup/read operations only (default: false)| +|`--no-bgjob`|Disable background jobs, default to false, which means clients by default carry out background jobs, including:
Note that compaction isn't affected by this option, it happens automatically with file reads and writes, client will check if compaction is in need, and run in background (take Redis for example, look for `compactChunk` in [`pkg/meta/base.go`](https://github.com/juicedata/juicefs/blob/main/pkg/meta/redis.go)).| +|`--atime-mode=noatime`|Control atime (last time the file was accessed) behavior, support the following modes:
| +|`--skip-dir-nlink value`|number of retries after which the update of directory nlink will be skipped (used for tkv only, 0 means never) (default: 20)| + +#### Metadata cache related options {#mount-metadata-cache-options} + +For metadata cache description and usage, refer to [Kernel metadata cache](../guide/cache_management.md#kernel-metadata-cache) and [Client memory metadata cache](../guide/cache_management.md#client-memory-metadata-cache). + +|Items|Description| +|-|-| +|`--attr-cache=1`|attributes cache timeout in seconds (default: 1), read [Kernel metadata cache](../guide/cache_management.md#kernel-metadata-cache)| +|`--entry-cache=1`|file entry cache timeout in seconds (default: 1), read [Kernel metadata cache](../guide/cache_management.md#kernel-metadata-cache)| +|`--dir-entry-cache=1`|dir entry cache timeout in seconds (default: 1), read [Kernel metadata cache](../guide/cache_management.md#kernel-metadata-cache)| +|`--open-cache=0`|open file cache timeout in seconds (0 means disable this feature) (default: 0)| +|`--open-cache-limit value`|max number of open files to cache (soft limit, 0 means unlimited) (default: 10000)| + +#### Data storage related options {#mount-data-storage-options} + +|Items|Description| +|-|-| +|`--storage=file`|Object storage type (e.g. `s3`, `gcs`, `oss`, `cos`) (default: `"file"`, refer to [documentation](../guide/how_to_set_up_object_storage.md#supported-object-storage) for all supported object storage types).| +|`--bucket=value`|customized endpoint to access object storage| +|`--get-timeout=60`|the max number of seconds to download an object (default: 60)| +|`--put-timeout=60`|the max number of seconds to upload an object (default: 60)| +|`--io-retries=10`|number of retries after network failure (default: 10)| +|`--max-uploads=20`|number of connections to upload (default: 20)| +|`--max-deletes=10`|number of threads to delete objects (default: 10)| +|`--upload-limit=0`|bandwidth limit for upload in Mbps (default: 0)| +|`--download-limit=0`|bandwidth limit for download in Mbps (default: 0)| + +#### Data cache related options {#mount-data-cache-options} + +|Items|Description| +|-|-| +|`--buffer-size=300`|total read/write buffering in MiB (default: 300), see [Read/Write buffer](../guide/cache_management.md#buffer-size)| +|`--prefetch=1`|prefetch N blocks in parallel (default: 1), see [Client read data cache](../guide/cache_management.md#client-read-cache)| +|`--writeback`|upload objects in background (default: false), see [Client write data cache](../guide/cache_management.md#writeback)| +|`--upload-delay=0`|When `--writeback` is enabled, you can use this option to add a delay to object storage upload, default to 0, meaning that upload will begin immediately after write. Different units are supported, including `s` (second), `m` (minute), `h` (hour). If files are deleted during this delay, upload will be skipped entirely, when using JuiceFS for temporary storage, use this option to reduce resource usage. Refer to [Client write data cache](../guide/cache_management.md#writeback).| +|`--cache-dir=value`|directory paths of local cache, use `:` (Linux, macOS) or `;` (Windows) to separate multiple paths (default: `$HOME/.juicefs/cache` or `/var/jfsCache`), see [Client read data cache](../guide/cache_management.md#client-read-cache)| +|`--cache-mode value`|file permissions for cached blocks (default: "0600")| +|`--cache-size=102400`|size of cached object for read in MiB (default: 102400), see [Client read data cache](../guide/cache_management.md#client-read-cache)| +|`--free-space-ratio=0.1`|min free space ratio (default: 0.1), if [Client write data cache](../guide/cache_management.md#writeback) is enabled, this option also controls write cache size, see [Client read data cache](../guide/cache_management.md#client-read-cache)| +|`--cache-partial-only`|cache random/small read only (default: false), see [Client read data cache](../guide/cache_management.md#client-read-cache)| +|`--verify-cache-checksum value`|Checksum level for cache data. After enabled, checksum will be calculated on divided parts of the cache blocks and stored on disks, which are used for verification during reads. The following strategies are supported:
| +|`--cache-eviction value`|cache eviction policy (none or 2-random) (default: "2-random")| +|`--cache-scan-interval value`|interval (in seconds) to scan cache-dir to rebuild in-memory index (default: "3600")| + +#### Metrics related options {#mount-metrics-options} + +||Items|Description| +|-|-| +|`--metrics=127.0.0.1:9567`|address to export metrics (default: `127.0.0.1:9567`)| +|`--consul=127.0.0.1:8500`|Consul address to register (default: `127.0.0.1:8500`)| +|`--no-usage-report`|do not send usage report (default: false)| + +### `juicefs umount` {#umount} Unmount a volume. -##### Synopsis +#### Synopsis -``` +```shell juicefs umount [command options] MOUNTPOINT -``` - -##### Options - -`-f, --force`
-force unmount a busy mount point (default: false) - -`--flush`
-wait for all staging chunks to be flushed (default: false) - -##### Examples -```bash juicefs umount /mnt/jfs ``` -#### `juicefs gateway`{#gateway} - -Start an S3-compatible gateway. - -##### Synopsis - -``` -juicefs gateway [command options] META-URL ADDRESS -``` - -- **META-URL**: Database URL for metadata storage, see ["JuiceFS supported metadata engines"](../guide/how_to_set_up_metadata_engine.md) for details. -- **ADDRESS**: S3 gateway address and listening port, for example: `localhost:9000` - -##### Options - -###### General - -`--access-log value`
-path for JuiceFS access log - -`--no-banner`
-disable MinIO startup information (default: false) - -`--multi-buckets`
-use top level of directories as buckets (default: false) - -`--keep-etag`
-save the ETag for uploaded objects (default: false) - -`--umask value`
-umask for new file and directory in octal (default: "022") - -###### Meta - -`--subdir value`
-mount a sub-directory as root (default: "") -`--backup-meta value`
-interval (in seconds) to automatically backup metadata in the object storage (0 means disable backup) (default: "3600") - -`--heartbeat value`
-interval (in seconds) to send heartbeat; it's recommended that all clients use the same heartbeat value (default: "12") - -`--read-only`
-allow lookup/read operations only (default: false) - -`--no-bgjob`
-disable background jobs (clean-up, backup, etc.) (default: false) - -`--atime-mode value`
-Control atime (last time the file was accessed) behavior, support the following modes: - -* `noatime` (default), set when the file is created or when `SetAttr` is explicitly called. Accessing and modifying the file will not affect atime, tracking atime comes at a performance cost, so this is the default behavior -* `relatime` update inode access times relative to mtime (last time when the file data was modified) or ctime (last time when file metadata was changed). Only update atime if atime was earlier than the current mtime or ctime, or the file's atime is more than 1 day old -* `strictatime`, always update atime on access - -`--skip-dir-nlink value`
-number of retries after which the update of directory nlink will be skipped (used for tkv only, 0 means never) (default: 20) - -###### Meta Cache - -`--attr-cache value`
-attributes cache timeout in seconds (default: 1), read [Kernel Metadata Cache](../guide/cache_management.md#kernel-metadata-cache) - -`--entry-cache value`
-file entry cache timeout in seconds (default: 1), read [Kernel Metadata Cache](../guide/cache_management.md#kernel-metadata-cache) - -`--dir-entry-cache value`
-dir entry cache timeout in seconds (default: 1), read [Kernel Metadata Cache](../guide/cache_management.md#kernel-metadata-cache) - -`--open-cache value`
-open file cache timeout in seconds (0 means disable this feature) (default: 0) - -`--open-cache-limit value`
-max number of open files to cache (soft limit, 0 means unlimited) (default: 10000) - -###### Data Storage - -`--storage value`
-Object storage type (e.g. `s3`, `gcs`, `oss`, `cos`) (default: `"file"`, please refer to [documentation](../guide/how_to_set_up_object_storage.md#supported-object-storage) for all supported object storage types) - -`--bucket value`
-customized endpoint to access object store - -`--storage-class value`
-the storage class for data written by current client - -`--get-timeout value`
-the max number of seconds to download an object (default: 60) - -`--put-timeout value`
-the max number of seconds to upload an object (default: 60) - -`--io-retries value`
-number of retries after network failure (default: 10) - -`--max-uploads value`
-number of connections to upload (default: 20) - -`--max-deletes value`
-number of threads to delete objects (default: 10) - -`--upload-limit value`
-bandwidth limit for upload in Mbps (default: 0) - -`--download-limit value`
-bandwidth limit for download in Mbps (default: 0) - -###### Data Cache - -`--buffer-size value`
-total read/write buffering in MiB (default: 300) - -`--prefetch value`
-prefetch N blocks in parallel (default: 1) - -`--writeback`
-upload objects in background (default: false), see [Client write data cache](../guide/cache_management.md#writeback) - -`--upload-delay value`
-if writeback mode is enabled, delayed duration for uploading objects ("s", "m", "h") (default: 0s) - -`--cache-dir value`
-directory paths of local cache, use `:` (Linux, macOS) or `;` (Windows) to separate multiple paths (default: `"$HOME/.juicefs/cache"` or `"/var/jfsCache"`), see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--cache-mode value`
-file permissions for cached blocks (default: "0600") - -`--cache-size value`
-size of cached object for read in MiB (default: 102400), see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--free-space-ratio value`
-min free space ratio (default: 0.1), if [Client write data cache](../guide/cache_management.md#writeback) is enabled, this option also controls write cache size, see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--cache-partial-only`
-cache random/small read only (default: false), see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--verify-cache-checksum value`
-Checksum level for cache data. After enabled, checksum will be calculated on divided parts of the cache blocks and stored on disks, which are used for verification during reads. The following strategies are supported:
- -`--cache-eviction value`
-cache eviction policy (none or 2-random) (default: "2-random") - -`--cache-scan-interval value`
-interval (in seconds) to scan cache-dir to rebuild in-memory index (default: "3600") +#### Options -###### Metrics +|Items|Description| +|-|-| +|`-f, --force`|force unmount a busy mount point (default: false)| +|`--flush`|wait for all staging chunks to be flushed (default: false)| -`--metrics value`
-address to export metrics (default: "127.0.0.1:9567") +### `juicefs gateway` {#gateway} -`--consul value`
-Consul address to register (default: "127.0.0.1:8500") +Start an S3-compatible gateway, read [Deploy JuiceFS S3 Gateway](../deployment/s3_gateway.md) for more. -`--no-usage-report`
-do not send usage report (default: false) +#### Synopsis -##### Examples +```shell +juicefs gateway [command options] META-URL ADDRESS -```bash export MINIO_ROOT_USER=admin export MINIO_ROOT_PASSWORD=12345678 juicefs gateway redis://localhost localhost:9000 ``` -#### `juicefs webdav` {#webdav} - -Start a WebDAV server. - -##### Synopsis - -``` -juicefs webdav [command options] META-URL ADDRESS -``` - -- **META-URL**: Database URL for metadata storage, see "[JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md)" for details. -- **ADDRESS**: WebDAV address and listening port, for example: `localhost:9007` - -##### Options - -###### General - -`--cert-file value`
-certificate file for HTTPS - -`--key-file value`
-key file for HTTPS - -`--gzip`
-compress served files via gzip (default: false) - -`--disallowList`
-disallow list a directory (default: false) - -`--access-log value`
-path for JuiceFS access log - -###### Meta - -`--subdir value`
-mount a sub-directory as root (default: "") -`--backup-meta value`
-interval (in seconds) to automatically backup metadata in the object storage (0 means disable backup) (default: "3600") - -`--heartbeat value`
-interval (in seconds) to send heartbeat; it's recommended that all clients use the same heartbeat value (default: "12") - -`--read-only`
-allow lookup/read operations only (default: false) - -`--no-bgjob`
-disable background jobs (clean-up, backup, etc.) (default: false) - -`--atime-mode value`
-Control atime (last time the file was accessed) behavior, support the following modes: - -* `noatime` (default), set when the file is created or when `SetAttr` is explicitly called. Accessing and modifying the file will not affect atime, tracking atime comes at a performance cost, so this is the default behavior -* `relatime` update inode access times relative to mtime (last time when the file data was modified) or ctime (last time when file metadata was changed). Only update atime if atime was earlier than the current mtime or ctime, or the file's atime is more than 1 day old -* `strictatime`, always update atime on access - -`--skip-dir-nlink value`
-number of retries after which the update of directory nlink will be skipped (used for tkv only, 0 means never) (default: 20) - -###### Meta Cache - -`--attr-cache value`
-attributes cache timeout in seconds (default: 1), read [Kernel Metadata Cache](../guide/cache_management.md#kernel-metadata-cache) - -`--entry-cache value`
-file entry cache timeout in seconds (default: 1), read [Kernel Metadata Cache](../guide/cache_management.md#kernel-metadata-cache) - -`--dir-entry-cache value`
-dir entry cache timeout in seconds (default: 1), read [Kernel Metadata Cache](../guide/cache_management.md#kernel-metadata-cache) - -`--open-cache value`
-open file cache timeout in seconds (0 means disable this feature) (default: 0) - -`--open-cache-limit value`
-max number of open files to cache (soft limit, 0 means unlimited) (default: 10000) - -###### Data Storage - -`--storage value`
-Object storage type (e.g. `s3`, `gcs`, `oss`, `cos`) (default: `"file"`, please refer to [documentation](../guide/how_to_set_up_object_storage.md#supported-object-storage) for all supported object storage types) - -`--bucket value`
-customized endpoint to access object store - -`--storage-class value`
-the storage class for data written by current client - -`--get-timeout value`
-the max number of seconds to download an object (default: 60) - -`--put-timeout value`
-the max number of seconds to upload an object (default: 60) - -`--io-retries value`
-number of retries after network failure (default: 10) - -`--max-uploads value`
-number of connections to upload (default: 20) - -`--max-deletes value`
-number of threads to delete objects (default: 10) - -`--upload-limit value`
-bandwidth limit for upload in Mbps (default: 0) - -`--download-limit value`
-bandwidth limit for download in Mbps (default: 0) - -###### Data Cache - -`--buffer-size value`
-total read/write buffering in MiB (default: 300) - -`--prefetch value`
-prefetch N blocks in parallel (default: 1) - -`--writeback`
-upload objects in background (default: false), see [Client write data cache](../guide/cache_management.md#writeback) - -`--upload-delay value`
-if writeback mode is enabled, delayed duration for uploading objects ("s", "m", "h") (default: 0s) - -`--cache-dir value`
-directory paths of local cache, use `:` (Linux, macOS) or `;` (Windows) to separate multiple paths (default: `"$HOME/.juicefs/cache"` or `"/var/jfsCache"`), see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--cache-mode value`
-file permissions for cached blocks (default: "0600") - -`--cache-size value`
-size of cached object for read in MiB (default: 102400), see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--free-space-ratio value`
-min free space ratio (default: 0.1), if [Client write data cache](../guide/cache_management.md#writeback) is enabled, this option also controls write cache size, see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--cache-partial-only`
-cache random/small read only (default: false), see [Client read data cache](../guide/cache_management.md#client-read-cache) - -`--verify-cache-checksum value`
-Checksum level for cache data. After enabled, checksum will be calculated on divided parts of the cache blocks and stored on disks, which are used for verification during reads. The following strategies are supported:
- -`--cache-eviction value`
-cache eviction policy (none or 2-random) (default: "2-random") +#### Options -`--cache-scan-interval value`
-interval (in seconds) to scan cache-dir to rebuild in-memory index (default: "3600") +Apart from options listed below, this command shares options with `juicefs mount`, be sure to refer to [`mount`](#mount) as well. -###### Metrics +|Items|Description| +|-|-| +|`META-URL`|Database URL for metadata storage, see [JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md) for details.| +|`ADDRESS`|S3 gateway address and listening port, for example: `localhost:9000`| +|`--access-log=path`|path for JuiceFS access log.| +|`--no-banner`|disable MinIO startup information (default: false)| +|`--multi-buckets`|use top level of directories as buckets (default: false)| +|`--keep-etag`|save the ETag for uploaded objects (default: false)| +|`--umask=022`|umask for new file and directory in octal (default: 022)| -`--metrics value`
-address to export metrics (default: "127.0.0.1:9567") +### `juicefs webdav` {#webdav} -`--consul value`
-Consul address to register (default: "127.0.0.1:8500") +Start a WebDAV server, refer to [Deploy WebDAV Server](../deployment/webdav.md) for more. -`--no-usage-report`
-do not send usage report (default: false) +#### Synopsis -##### Examples +```shell +juicefs webdav [command options] META-URL ADDRESS -```bash juicefs webdav redis://localhost localhost:9007 ``` -### tools - -#### `juicefs bench` {#bench} - -Run benchmark, including read/write/stat for big and small files. - -##### Synopsis - -``` -juicefs bench [command options] PATH -``` - -For a detailed introduction to the `bench` subcommand, please refer to the [documentation](../benchmark/performance_evaluation_guide.md#juicefs-bench). +#### Options -##### Options +Apart from options listed below, this command shares options with `juicefs mount`, be sure to refer to [`mount`](#mount) as well. -`--block-size value`
-block size in MiB (default: 1) +|Items|Description| +|-|-| +|`META-URL`|Database URL for metadata storage, see [JuiceFS supported metadata engines](../guide/how_to_set_up_metadata_engine.md) for details.| +|`ADDRESS`|WebDAV address and listening port, for example: `localhost:9007`.| +|`--cert-file`|certificate file for HTTPS| +|`--key-file`|key file for HTTPS| +|`--gzip`|compress served files via gzip (default: false)| +|`--disallowList`|disallow list a directory (default: false)| +|`--access-log=path`|path for JuiceFS access log.| -`--big-file-size value`
-size of big file in MiB (default: 1024) +## Tool {#tool} -`--small-file-size value`
-size of small file in MiB (default: 0.1) +### `juicefs bench` {#bench} -`--small-file-count value`
-number of small files (default: 100) +Run benchmark, including read/write/stat for big and small files. +For a detailed introduction to the `bench` subcommand, refer to the [documentation](../benchmark/performance_evaluation_guide.md#juicefs-bench). -`--threads value, -p value`
-number of concurrent threads (default: 1) +#### Synopsis -#### Examples +```shell +juicefs bench [command options] PATH -```bash # Run benchmarks with 4 threads -$ juicefs bench /mnt/jfs -p 4 +juicefs bench /mnt/jfs -p 4 # Run benchmarks of only small files -$ juicefs bench /mnt/jfs --big-file-size 0 +juicefs bench /mnt/jfs --big-file-size 0 ``` -#### `juicefs objbench` {#objbench} - -Run basic benchmarks on the target object storage to test if it works as expected. - -##### Synopsis - -```shell -juicefs objbench [command options] BUCKET -``` - -For a detailed introduction to the `objbench` subcommand, please refer to the [documentation](../benchmark/performance_evaluation_guide.md#juicefs-objbench). - -##### Options - -`--storage value`
-Object storage type (e.g. `s3`, `gcs`, `oss`, `cos`) (default: `"file"`, please refer to [documentation](../guide/how_to_set_up_object_storage.md#supported-object-storage) for all supported object storage types) - -`--access-key value`
-Access Key for object storage (can also be set via the environment variable `ACCESS_KEY`) - -`--secret-key value`
-Secret Key for object storage (can also be set via the environment variable `SECRET_KEY`) - -`--block-size value`
-size of each IO block in KiB (default: 4096) - -`--big-object-size value`
-size of each big object in MiB (default: 1024) +#### Options -`--small-object-size value`
-size of each small object in KiB (default: 128) +|Items|Description| +|-|-| +|`--block-size=1`|block size in MiB (default: 1)| +|`--big-file-size=1024`|size of big file in MiB (default: 1024)| +|`--small-file-size=0.1`|size of small file in MiB (default: 0.1)| +|`--small-file-count=100`|number of small files (default: 100)| +|`--threads=1, -p 1`|number of concurrent threads (default: 1)| -`--small-objects value`
-number of small objects (default: 100) +### `juicefs objbench` {#objbench} -`--skip-functional-tests`
-skip functional tests (default: false) +Run basic benchmarks on the target object storage to test if it works as expected. Read [documentation](../benchmark/performance_evaluation_guide.md#juicefs-objbench) for more. -`--threads value, -p value`
-number of concurrent threads (default: 4) +#### Synopsis -##### Examples +```shell +juicefs objbench [command options] BUCKET -```bash # Run benchmarks on S3 -$ ACCESS_KEY=myAccessKey SECRET_KEY=mySecretKey juicefs objbench --storage s3 https://mybucket.s3.us-east-2.amazonaws.com -p 6 +ACCESS_KEY=myAccessKey SECRET_KEY=mySecretKey juicefs objbench --storage=s3 https://mybucket.s3.us-east-2.amazonaws.com -p 6 ``` -#### `juicefs warmup` {#warmup} +#### Options + +|Items|Description| +|-|-| +|`--storage=file`|Object storage type (e.g. `s3`, `gcs`, `oss`, `cos`) (default: `file`, refer to [documentation](../guide/how_to_set_up_object_storage.md#supported-object-storage) for all supported object storage types)| +|`--access-key=value`|Access Key for object storage (can also be set via the environment variable `ACCESS_KEY`), see [How to Set Up Object Storage](../guide/how_to_set_up_object_storage.md#aksk) for more.| +|`--secret-key value`|Secret Key for object storage (can also be set via the environment variable `SECRET_KEY`), see [How to Set Up Object Storage](../guide/how_to_set_up_object_storage.md#aksk) for more.| +|`--block-size=4096`|size of each IO block in KiB (default: 4096)| +|`--big-object-size=1024`|size of each big object in MiB (default: 1024)| +|`--small-object-size=128`|size of each small object in KiB (default: 128)| +|`--small-objects=100`|number of small objects (default: 100)| +|`--skip-functional-tests`|skip functional tests (default: false)| +|`--threads=4, -p 4`|number of concurrent threads (default: 4)| + +### `juicefs warmup` {#warmup} Download data to local cache in advance, to achieve better performance on application's first read. You can specify a mount point path to recursively warm-up all files under this path. You can also specify a file through the `--file` option to only warm-up the files contained in it. If the files needing warming up resides in many different directories, you should specify their names in a text file, and pass to the `warmup` command using the `--file` option, allowing `juicefs warmup` to download concurrently, which is significantly faster than calling `juicefs warmup` multiple times, each with a single file. -##### Synopsis +#### Synopsis -``` +```shell juicefs warmup [command options] [PATH ...] -``` - -##### Options - -`--file value, -f value`
-file containing a list of paths (each line is a file path) - -`--threads value, -p value`
-number of concurrent workers, default to 50. Reduce this number in low bandwidth environment to avoid download timeouts -`--background, -b`
-run in background (default: false) +# Warm up all files in datadir +juicefs warmup /mnt/jfs/datadir -##### Examples +# Warm up selected files +echo '/jfs/f1 +/jfs/f2 +/jfs/f3' > /tmp/filelist.txt +juicefs warmup -f /tmp/filelist.txt +``` -```bash -# Warm all files in datadir -$ juicefs warmup /mnt/jfs/datadir +#### Options -# Warm only three files in datadir -$ cat /tmp/filelist -/mnt/jfs/datadir/f1 -/mnt/jfs/datadir/f2 -/mnt/jfs/datadir/f3 -$ juicefs warmup -f /tmp/filelist -``` +|Items|Description| +|-|-| +|`--file=path, -f path`|file containing a list of paths (each line is a file path)| +|`--threads=50, -p 50`|number of concurrent workers, default to 50. Reduce this number in low bandwidth environment to avoid download timeouts| +|`--background, -b`|run in background (default: false)| -#### `juicefs rmr`{#rmr} +### `juicefs rmr` {#rmr} -Remove all the files and subdirectories, similar to rm -rf, except this command deals with metadata directly (bypassing POSIX API), thus is much faster. +Remove all the files and subdirectories, similar to `rm -rf`, except this command deals with metadata directly (bypassing kernel), thus is much faster. -If trash is enabled, deleted files are moved into trash. read more at [Trash](../security/trash.md). +If trash is enabled, deleted files are moved into trash. Read more at [Trash](../security/trash.md). -##### Synopsis +#### Synopsis -``` +```shell juicefs rmr PATH ... -``` -##### Examples - -```bash juicefs rmr /mnt/jfs/foo ``` -#### `juicefs sync`{#sync} +### `juicefs sync` {#sync} -Sync between two storage. +Sync between two storage, read [Data migration](../guide/sync.md) for more. -##### Synopsis +#### Synopsis -``` +```shell juicefs sync [command options] SRC DST -``` - -- **SRC**: source path -- **DST**: destination path - -The format of both source and destination paths is `[NAME://][ACCESS_KEY:SECRET_KEY[:TOKEN]@]BUCKET[.ENDPOINT][/PREFIX]`, in which: - -- `NAME`: JuiceFS supported data storage types (e.g. `s3`, `oss`) (please refer to [this document](../guide/how_to_set_up_object_storage.md#supported-object-storage)). -- `ACCESS_KEY` and `SECRET_KEY`: The credential required to access the data storage (please refer to [this document](../guide/how_to_set_up_object_storage.md#access-key-and-secret-key)). -- `TOKEN` token used to access the object storage, as some object storage supports the use of temporary token to obtain permission for a limited time -- `BUCKET[.ENDPOINT]`: The access address of the data storage service. The format may be different for different storage types, and please refer to [the document](../guide/how_to_set_up_object_storage.md#supported-object-storage). -- `[/PREFIX]`: Optional, a prefix for the source and destination paths that can be used to limit synchronization of data only in certain paths. - -For a detailed introduction to the `sync` subcommand, please refer to the [documentation](../guide/sync.md). - -##### Options - -###### Selection - -`--start KEY, -s KEY`
-the first KEY to sync - -`--end KEY, -e KEY`
-the last KEY to sync - -`--exclude PATTERN`
-exclude Key matching PATTERN - -`--include PATTERN`
-don't exclude Key matching PATTERN, need to be used with `--exclude` option - -`--limit value`
-limit the number of objects that will be processed (default: -1) - -`--update, -u`
-update existing file if the source is newer (default: false) - -`--force-update, -f`
-always update existing file (default: false) - -`--existing, --ignore-non-existing`
-skip creating new files on destination (default: false) - -`--ignore-existing`
-skip updating files that already exist on destination (default: false) - -###### Action - -`--dirs`
-Sync directories or holders (default: false) - -`--perms`
-preserve permissions (default: false) -`--links, -l`
-copy symlinks as symlinks (default: false) - -`--delete-src, --deleteSrc`
-delete objects from source after synced (default: false) +# Sync object from OSS to S3 +juicefs sync oss://mybucket.oss-cn-shanghai.aliyuncs.com s3://mybucket.s3.us-east-2.amazonaws.com -`--delete-dst, --deleteDst`
-delete extraneous objects from destination (default: false) +# Sync objects from S3 to JuiceFS +juicefs sync s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://META-URL/ -`--check-all`
-verify integrity of all files in source and destination (default: false) +# SRC: a1/b1,a2/b2,aaa/b1 DST: empty sync result: aaa/b1 +juicefs sync --exclude='a?/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://META-URL/ -`--check-new`
-verify integrity of newly copied files (default: false) +# SRC: a1/b1,a2/b2,aaa/b1 DST: empty sync result: a1/b1,aaa/b1 +juicefs sync --include='a1/b1' --exclude='a[1-9]/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://META-URL/ -`--dry`
-don't copy file (default: false) +# SRC: a1/b1,a2/b2,aaa/b1,b1,b2 DST: empty sync result: a1/b1,b2 +juicefs sync --include='a1/b1' --exclude='a*' --include='b2' --exclude='b?' s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://META-URL/ +``` -###### Storage +As shown in the examples, the format of both source (`SRC`) and destination (`DST`) paths is: -`--threads value, -p value`
-number of concurrent threads (default: 10) +``` +[NAME://][ACCESS_KEY:SECRET_KEY[:TOKEN]@]BUCKET[.ENDPOINT][/PREFIX] +``` -`--list-threads value`
-number of threads to list objects (default: 1) +In which: -`--list-depth value`
-list the top N level of directories in parallel (default: 1) +- `NAME`: JuiceFS supported data storage types like `s3`, `oss`, refer to [this document](../guide/how_to_set_up_object_storage.md#supported-object-storage) for a full list. +- `ACCESS_KEY` and `SECRET_KEY`: The credential required to access the data storage, refer to [this document](../guide/how_to_set_up_object_storage.md#aksk). +- `TOKEN` token used to access the object storage, as some object storage supports the use of temporary token to obtain permission for a limited time +- `BUCKET[.ENDPOINT]`: The access address of the data storage service. The format may be different for different storage types, and refer to [the document](../guide/how_to_set_up_object_storage.md#supported-object-storage). +- `[/PREFIX]`: Optional, a prefix for the source and destination paths that can be used to limit synchronization of data only in certain paths. -`--no-https`
-do not use HTTPS (default: false) +#### Selection related options {#sync-selection-related-options} -`--storage-class value`
-the storage class for destination +|Items|Description| +|-|-| +|`--start=KEY, -s KEY, --end=KEY, -e KEY`|Provide object storage key range for syncing.| +|`--exclude=PATTERN`|Exclude keys matching PATTERN.| +|`--include=PATTERN`|Include keys matching PATTERN, need to be used with `--exclude`.| +|`--limit=-1`|Limit the number of objects that will be processed, default to -1 which means unlimited.| +|`--update, -u`|Update existing files if the source files' `mtime` is newer, default to false.| +|`--force-update, -f`|Always update existing file, default to false.| +|`--existing, --ignore-non-existing`|Skip creating new files on destination, default to false.| +|`--ignore-existing`|Skip updating files that already exist on destination, default to false.| -`--bwlimit value`
-limit bandwidth in Mbps (0 means unlimited) (default: 0) +#### Action related options {#sync-action-related-options} -###### Cluster +|Items|Description| +|-|-| +|`--dirs`|Sync empty directories as well.| +|`--perms`|Preserve permissions, default to false.| +|`--links, -l`|Copy symlinks as symlinks default to false.| +|`--delete-src, --deleteSrc`|Delete objects that already exist in destination. Different from rsync, files won't be deleted at the first run, instead they will be deleted at the next run, after files are successfully copied to the destination.| +|`--delete-dst, --deleteDst`|Delete extraneous objects from destination.| +|`--check-all`|Verify the integrity of all files in source and destination, default to false. Comparison is done on byte streams, which comes at a performance cost.| +|`--check-new`|Verify the integrity of newly copied files, default to false. Comparison is done on byte streams, which comes at a performance cost.| +|`--dry`|Don't actually copy any file.| -`--manager value`
-manager address +#### Storage related options {#sync-storage-related-options} -`--worker value`
-hosts (separated by comma) to launch worker +|Items|Description| +|-|-| +|`--threads=10, -p 10`|Number of concurrent threads, default to 10.| +|`--list-threads=1`|Number of `list` threads, default to 1. Read [concurrent `list`](../guide/sync.md#concurrent-list) to learn its usage.| +|`--list-depth=1`|Depth of concurrent `list` operation, default to 1. Read [concurrent `list`](../guide/sync.md#concurrent-list) to learn its usage.| +|`--no-https`|Do not use HTTPS, default to false.| +|`--storage-class value`|the storage class for destination| +|`--bwlimit=0`|Limit bandwidth in Mbps default to 0 which means unlimited.| -##### Examples +#### Cluster related options {#sync-cluster-related-options} -```bash -# Sync object from OSS to S3 -$ juicefs sync oss://mybucket.oss-cn-shanghai.aliyuncs.com s3://mybucket.s3.us-east-2.amazonaws.com +|Items|Description| +|-|-| +|`--manager=ADDR`|Manager node address used in distributed syncing, this is an internal option that's used in the executed command on the worker nodes.| +|`--worker=ADDR,ADDR`|Worker node addresses used in distributed syncing, comma separated.| -# Sync objects from S3 to JuiceFS -$ juicefs mount -d redis://localhost /mnt/jfs -$ juicefs sync s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/ +### `juicefs clone` {#clone} -# SRC: a1/b1,a2/b2,aaa/b1 DST: empty sync result: aaa/b1 -$ juicefs sync --exclude='a?/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/ +This command can clone a file or directory without copying the underlying data, similar to the `cp` command, but very fast. -# SRC: a1/b1,a2/b2,aaa/b1 DST: empty sync result: a1/b1,aaa/b1 -$ juicefs sync --include='a1/b1' --exclude='a[1-9]/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/ - -# SRC: a1/b1,a2/b2,aaa/b1,b1,b2 DST: empty sync result: a1/b1,b2 -$ juicefs sync --include='a1/b1' --exclude='a*' --include='b2' --exclude='b?' s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/ -``` - -#### `juicefs clone`{#clone} - -clone a file or directory without copying the underlying data - -##### Synopsis +#### Synopsis ```shell juicefs clone [command options] SRC DST -``` -##### Options - -`--preserve, -p`
-preserve the UID, GID, and mode of the file (default: false) - -##### Examples - -```shell # Clone a file -$ juicefs clone /mnt/jfs/file1 /mnt/jfs/file2 +juicefs clone /mnt/jfs/file1 /mnt/jfs/file2 # Clone a directory -$ juicefs clone /mnt/jfs/dir1 /mnt/jfs/dir2 +juicefs clone /mnt/jfs/dir1 /mnt/jfs/dir2 # Clone with preserving the UID, GID, and mode of the file -$ juicefs clone -p /mnt/jfs/file1 /mnt/jfs/file2 +juicefs clone -p /mnt/jfs/file1 /mnt/jfs/file2 ``` + +#### Options + +|Items|Description| +|-|-| +|`--preserve, -p`|preserve the UID, GID, and mode of the file (default: false)| diff --git a/docs/zh_cn/administration/fault_diagnosis_and_analysis.md b/docs/zh_cn/administration/fault_diagnosis_and_analysis.md index a415504b587b..c74bc73c9a2c 100644 --- a/docs/zh_cn/administration/fault_diagnosis_and_analysis.md +++ b/docs/zh_cn/administration/fault_diagnosis_and_analysis.md @@ -166,7 +166,7 @@ $ tree ./debug │ ├── stats.5s.txt │ ├── stats.txt │ └── system-info.log -└── tmp-test1-20230609104324.zip +└── tmp-test1-20230609104324.zip ``` ## 实时性能监控 {#performance-monitor} diff --git a/docs/zh_cn/deployment/production_deployment_recommendations.md b/docs/zh_cn/deployment/production_deployment_recommendations.md index 8a6e395c9e75..4308953118c4 100644 --- a/docs/zh_cn/deployment/production_deployment_recommendations.md +++ b/docs/zh_cn/deployment/production_deployment_recommendations.md @@ -88,4 +88,4 @@ logrotate -d /etc/logrotate.d/juicefs ## 命令行自动补全 -JuiceFS 为 Bash 和 Zsh 提供了命令行自动补全脚本，方便在命令行中使用 `juicefs` 命令，具体请参考[文档](../reference/command_reference.md#自动补全)。 +JuiceFS 为 Bash 和 Zsh 提供了命令行自动补全脚本，方便在命令行中使用 `juicefs` 命令，具体请参考[文档](../reference/command_reference.md#auto-completion)。 diff --git a/docs/zh_cn/guide/cache_management.md b/docs/zh_cn/guide/cache_management.md index 0eb0f25bf207..c6578b55d44f 100644 --- a/docs/zh_cn/guide/cache_management.md +++ b/docs/zh_cn/guide/cache_management.md @@ -61,7 +61,7 @@ JuiceFS 客户端在 `open` 操作即打开一个文件时，其文件属性会 作为对比，JuiceFS 商业版提供更丰富的客户端内存的元数据缓存功能，并且支持主动失效，阅读[商业版文档](https://juicefs.com/docs/zh/cloud/guide/cache/#client-memory-metadata-cache)以了解。 -## 数据缓存 +## 数据缓存 {#data-cache} JuiceFS 对数据也提供多种缓存机制来提高性能，包括内核中的页缓存和客户端所在机器的本地缓存，以及客户端自身的内存读写缓冲区。读请求会依次尝试内核分页缓存、JuiceFS 进程的预读缓冲区、本地磁盘缓存，当缓存中没找到对应数据时才会从对象存储读取，并且会异步写入各级缓存保证下一次访问的性能。 diff --git a/docs/zh_cn/guide/how_to_set_up_object_storage.md b/docs/zh_cn/guide/how_to_set_up_object_storage.md index dbdc2cc16ff6..ee7dbf62090f 100644 --- a/docs/zh_cn/guide/how_to_set_up_object_storage.md +++ b/docs/zh_cn/guide/how_to_set_up_object_storage.md @@ -55,7 +55,7 @@ juicefs format --storage s3 \ 执行上述命令后，JuiceFS 客户端会创建 4 个 bucket，分别为 `myjfs-0`、`myjfs-1`、`myjfs-2` 和 `myjfs-3`。 -## Access Key 和 Secret Key +## Access Key 和 Secret Key {#aksk} 一般而言，对象存储通过 Access Key ID 和 Access Key Secret 验证用户身份，对应到 JuiceFS 文件系统就是 `--access-key` 和 `--secret-key` 这两个选项（或者简称为 AK、SK）。 @@ -72,7 +72,7 @@ juicefs format --storage s3 \ 公有云通常允许用户创建 IAM（Identity and Access Management）角色，例如：[AWS IAM 角色](https://docs.aws.amazon.com/zh_cn/IAM/latest/UserGuide/id_roles.html) 或 [阿里云 RAM 角色](https://help.aliyun.com/document_detail/93689.html)，可将角色分配给 VM 实例。如果云服务器实例已经拥有读写对象存储的权限，则无需再指定 `--access-key` 和 `--secret-key`。 -## 使用临时访问凭证 +## 使用临时访问凭证 {#session-token} 永久访问凭证一般有两个部分：Access Key 和 Secret Key，而临时访问凭证一般包括 3 个部分：Access Key、Secret Key 与 token，并且临时访问凭证具有过期时间，一般在几分钟到几个小时之间。 diff --git a/docs/zh_cn/reference/command_reference.md b/docs/zh_cn/reference/command_reference.md index bee1421e5fdc..928606ef3fce 100644 --- a/docs/zh_cn/reference/command_reference.md +++ b/docs/zh_cn/reference/command_reference.md @@ -2,17 +2,15 @@ title: 命令参考 sidebar_position: 1 slug: /command_reference -description: 本文提供 JuiceFS 包含的所有命令及选项的说明、用法和示例。 +description: JuiceFS 客户端的所有命令及选项的说明、用法和示例。 --- import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; -## 概览 +在终端输入 `juicefs` 并执行，就能看到所有可用的命令。在每个子命令后面添加 `-h/--help` 并运行，就能获得该命令的详细帮助信息，例如 `juicefs format -h`。 -在终端输入 `juicefs` 并执行，你就会看到所有可用的命令。另外，你可以在每个命令后面添加 `-h/--help` 标记获得该命令的详细帮助信息。 - -```shell +``` NAME: juicefs - A POSIX file system built on Redis and object storage. @@ -20,7 +18,7 @@ USAGE: juicefs [global options] command [command options] [arguments...] VERSION: - 1.1.0-beta1+2023-06-08.5ef17ba0 + 1.1.0 COMMANDS: ADMIN: @@ -68,11 +66,7 @@ COPYRIGHT: Apache License 2.0 ``` -:::note 注意 -如果命令选项是布尔（boolean）类型，例如 `--debug` ，无需设置任何值，只要在命令中添加 `--debug` 即代表启用该功能，反之则代表不启用。 -::: - -## 自动补全 +## 自动补全 {#auto-completion} 通过加载 [`hack/autocomplete`](https://github.com/juicedata/juicefs/tree/main/hack/autocomplete) 目录下的对应脚本可以启用命令的自动补全，例如： @@ -115,373 +109,263 @@ echo "source path/to/zsh_autocomplete" >> ~/.zshrc 另外，如果你是在 Linux 系统上使用 bash，也可以直接将脚本拷贝到 `/etc/bash_completion.d` 目录并将其重命名为 `juicefs`： ```shell -sudo cp hack/autocomplete/bash_autocomplete /etc/bash_completion.d/juicefs +cp hack/autocomplete/bash_autocomplete /etc/bash_completion.d/juicefs source /etc/bash_completion.d/juicefs ``` -## 命令列表 +## 管理 {#admin} -### 管理 +### `juicefs format` {#format} -#### `juicefs format` {#format} +创建并格式化文件系统，如果 `META-URL` 中已经存在一个文件系统，不会再次进行格式化。如果文件系统创建后需要调整配置，请使用 [`juicefs config`](#config)。 -创建文件系统，如果 `META-URL` 中已经存在一个文件系统，则不会再次进行格式化。如果文件系统创建后需要调整配置，请使用 [`juicefs config`](#config)。 +#### 概览 -##### 使用 - -``` +```shell juicefs format [command options] META-URL NAME -``` - -- **META-URL**：用于元数据存储的数据库 URL，详情查看「[JuiceFS 支持的元数据引擎](../guide/how_to_set_up_metadata_engine.md)」。 -- **NAME**：文件系统名称 - -##### 选项 - -###### 常规 - -`--force`
-强制覆盖当前的格式化配置 (默认：false) - -`--no-update`
-不要修改已有的格式化配置 (默认：false) - -###### 数据存储 - -`--storage value`
-对象存储类型 (例如 `s3`、`gcs`、`oss`、`cos`) (默认：`"file"`，请参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)查看所有支持的对象存储类型) - -`--bucket value`
-存储数据的桶路径 (默认：`"$HOME/.juicefs/local"` 或 `"/var/jfs"`) - -`--access-key value`
-对象存储的 Access Key (也可通过环境变量 `ACCESS_KEY` 设置) - -`--secret-key value`
-对象存储的 Secret Key (也可通过环境变量 `SECRET_KEY` 设置) - -`--session-token value`
-对象存储的 session token - -`--storage-class value`
-默认存储类型 - -###### 数据格式 - -`--block-size value`
-块大小；单位为 KiB (默认：4096)。4M 是一个较好的默认值，不少对象存储（比如 S3）都将 4M 设为内部的块大小，因此将 JuiceFS block size 设为相同大小，往往也能获得更好的性能 - -`--compress value`
-压缩算法 (`lz4`, `zstd`, `none`) (默认："none")，开启压缩将不可避免地对性能产生一定影响，请权衡。 - -`--encrypt-rsa-key value`
-RSA 私钥的路径 (PEM) - -`--encrypt-algo value`
-加密算法 (aes256gcm-rsa, chacha20-rsa) (默认："aes256gcm-rsa") - -`--hash-prefix`
-给每个对象添加 hash 前缀 (默认：false) - -`--shards value`
-将数据块根据名字哈希存入 N 个桶中 (默认：0)，当 N 大于 0 时，`bucket` 需要写成 `%d` 的形式，例如 `--bucket "juicefs-%d"` - -###### 管理 - -`--capacity value`
-容量配额；单位为 GiB (默认：不限制)。如果启用了回收站，那么配额大小也将包含回收站文件 - -`--inodes value`
-文件数配额 (默认：不限制) -`--trash-days value`
-文件被自动清理前在回收站内保留的天数 (默认：1) - -##### 示例 - -```shell # 创建一个简单的测试卷（数据将存储在本地目录中） -$ juicefs format sqlite3://myjfs.db myjfs +juicefs format sqlite3://myjfs.db myjfs # 使用 Redis 和 S3 创建卷 -$ juicefs format redis://localhost myjfs --storage s3 --bucket https://mybucket.s3.us-east-2.amazonaws.com +juicefs format redis://localhost myjfs --storage=s3 --bucket=https://mybucket.s3.us-east-2.amazonaws.com # 使用带有密码的 MySQL 创建卷 -$ juicefs format mysql://jfs:mypassword@(127.0.0.1:3306)/juicefs myjfs +juicefs format mysql://jfs:mypassword@(127.0.0.1:3306)/juicefs myjfs # 更安全的方法 -$ META_PASSWORD=mypassword juicefs format mysql://jfs:@(127.0.0.1:3306)/juicefs myjfs +META_PASSWORD=mypassword juicefs format mysql://jfs:@(127.0.0.1:3306)/juicefs myjfs # 创建一个开启配额设置的卷 -$ juicefs format sqlite3://myjfs.db myjfs --inode 1000000 --capacity 102400 +juicefs format sqlite3://myjfs.db myjfs --inode=1000000 --capacity=102400 # 创建一个关闭了回收站的卷 -$ juicefs format sqlite3://myjfs.db myjfs --trash-days 0 +juicefs format sqlite3://myjfs.db myjfs --trash-days=0 ``` -#### `juicefs config` {#config} - -修改指定文件系统的配置项。注意更新某些设置以后，客户端未必能立刻生效，需要等待一定时间，具体的等待时间可以通过 [`--heartbeat`](#mount) 选项控制。 - -##### 使用 - -``` -juicefs config [command options] META-URL -``` - -##### 选项 - -###### 常规 - -`--yes, -y`
-对所有提示自动回答 "yes" 并以非交互方式运行 (默认值：false) - -`--force`
-跳过合理性检查并强制更新指定配置项 (默认：false) - -###### 数据存储 +#### 参数 -`--storage value`
-对象存储类型 (例如 `s3`、`gcs`、`oss`、`cos`) (默认：`"file"`，请参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)查看所有支持的对象存储类型) +|项 | 说明| +|-|-| +|`META-URL`|用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md)。| +|`NAME`|文件系统名称。| +|`--force`|强制覆盖当前的格式化配置，默认为 false。| +|`--no-update`|不要修改已有的格式化配置，默认为 false。| -`--bucket value`
-存储数据的桶路径 (默认：`"$HOME/.juicefs/local"` 或 `"/var/jfs"`) +#### 数据存储参数 {#format-data-storage-options} -`--access-key value`
-对象存储的 Access Key (也可通过环境变量 `ACCESS_KEY` 设置) +|项 | 说明| +|-|-| +|`--storage=file`|对象存储类型，例如 `s3`、`gcs`、`oss`、`cos`。默认为 `file`，参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)查看所有支持的对象存储类型。| +|`--bucket=path`|存储数据的桶路径（默认：`$HOME/.juicefs/local` 或 `/var/jfs`）。| +|`--access-key=value`|对象存储的 Access Key，也可通过环境变量 `ACCESS_KEY` 设置。查看[如何设置对象存储](../guide/how_to_set_up_object_storage.md#aksk)以了解更多。| +|`--secret-key=value`|对象存储的 Secret Key，也可通过环境变量 `SECRET_KEY` 设置。查看[如何设置对象存储](../guide/how_to_set_up_object_storage.md#aksk)以了解更多。| +|`--session-token=value`|对象存储的临时访问凭证（Session Token），查看[如何设置对象存储](../guide/how_to_set_up_object_storage.md#session-token)以了解更多。| +|`--storage-class=value`|默认存储类型。| -`--secret-key value`
-对象存储的 Secret Key (也可通过环境变量 `SECRET_KEY` 设置) +#### 数据格式参数 {#format-data-format-options} -`--session-token value`
-对象存储的 session token +|项 | 说明| +|-|-| +|`--block-size=4096`|块大小，单位为 KiB，默认 4096。4M 是一个较好的默认值，不少对象存储（比如 S3）都将 4M 设为内部的块大小，因此将 JuiceFS block size 设为相同大小，往往也能获得更好的性能。| +|`--compress=none`|压缩算法，支持 `lz4`、`zstd`、`none`（默认），启用压缩将不可避免地对性能产生一定影响。| +|`--encrypt-rsa-key=value`|RSA 私钥的路径，查看[数据加密](../security/encrypt.md)以了解更多。| +|`--encrypt-algo=aes256gcm-rsa`|加密算法 (aes256gcm-rsa, chacha20-rsa) (默认："aes256gcm-rsa")| +|`--hash-prefix`|给每个对象添加 hash 前缀，默认为 false。| +|`--shards=0`|将数据块根据名字哈希存入 N 个桶中，默认为 0。当 N 大于 0 时，`bucket` 需要包含 `%d` 占位符，例如 `--bucket=juicefs-%d`。| -`--storage-class value`
-默认存储类型 +#### 管理参数 {#format-management-options} -`--upload-limit value`
-上传带宽限制，单位为 Mbps (默认：0) +|项 | 说明| +|-|-| +|`--capacity=0`|容量配额，单位为 GiB，默认为 0 代表不限制。如果启用了[回收站](../security/trash.md)，那么配额大小也将包含回收站文件。| +|`--inodes=0`|文件数配额，默认为 0 代表不限制。| +|`--trash-days=1`|文件被删除后，默认会进入[回收站](../security/trash.md)，该选项控制已删除文件在回收站内保留的天数，默认为 1，设为 0 以禁用回收站。| -`--download-limit value`
-下载带宽限制，单位为 Mbps (默认：0) +### `juicefs config` {#config} -###### 管理 - -`--capacity value`
-容量配额；单位为 GiB (默认：不限制)。如果启用了回收站，那么配额大小也将包含回收站文件 - -`--inodes value`
-文件数配额 (默认：不限制) - -`--trash-days value`
-文件被自动清理前在回收站内保留的天数 (默认：1) - -`--encrypt-secret`
-如果密钥之前以原格式存储，则加密密钥 (默认值：false) - -`--min-client-version value`
-允许连接的最小客户端版本 - -`--max-client-version value`
-允许连接的最大客户端版本 - -`--dir-stats`
-开启目录统计，这是快速汇总和目录配额所必需的 (默认值：false) +修改指定文件系统的配置项。注意更新某些设置以后，客户端未必能立刻生效，需要等待一定时间，具体的等待时间可以通过 [`--heartbeat`](#mount) 选项控制。 -##### 示例 +#### 概览 ```shell +juicefs config [command options] META-URL + # 显示当前配置 -$ juicefs config redis://localhost +juicefs config redis://localhost # 改变目录的配额 -$ juicefs config redis://localhost --inode 10000000 --capacity 1048576 +juicefs config redis://localhost --inode 10000000 --capacity 1048576 # 更改回收站中文件可被保留的最长天数 -$ juicefs config redis://localhost --trash-days 7 +juicefs config redis://localhost --trash-days 7 # 限制允许连接的客户端版本 -$ juicefs config redis://localhost --min-client-version 1.0.0 --max-client-version 1.1.0 +juicefs config redis://localhost --min-client-version 1.0.0 --max-client-version 1.1.0 ``` -#### `juicefs quota` {#quota} +#### 参数 -管理目录配额 +|项 | 说明| +|-|-| +|`--yes, -y`|对所有提示自动回答 "yes" 并以非交互方式运行 (默认值：false)| +|`--force`|跳过合理性检查并强制更新指定配置项 (默认：false)| -##### 使用 - -```shell - juicefs quota command [command options] META-URL -``` +#### 数据存储参数 {#config-data-storage-options} -##### 子命令 +|项 | 说明| +|-|-| +|`--storage=file`|对象存储类型，例如 `s3`、`gcs`、`oss`、`cos`。默认为 `file`，参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)查看所有支持的对象存储类型。| +|`--bucket=path`|存储数据的桶路径（默认：`$HOME/.juicefs/local` 或 `/var/jfs`）。| +|`--access-key=value`|对象存储的 Access Key，也可通过环境变量 `ACCESS_KEY` 设置。查看[如何设置对象存储](../guide/how_to_set_up_object_storage.md#aksk)以了解更多。| +|`--secret-key=value`|对象存储的 Secret Key，也可通过环境变量 `SECRET_KEY` 设置。查看[如何设置对象存储](../guide/how_to_set_up_object_storage.md#aksk)以了解更多。| +|`--session-token=value`|对象存储的临时访问凭证（Session Token），查看[如何设置对象存储](../guide/how_to_set_up_object_storage.md#session-token)以了解更多。| +|`--storage-class=value`|默认存储类型。| +|`--upload-limit=0`|上传带宽限制，单位为 Mbps (默认：0)| +|`--download-limit=0`|下载带宽限制，单位为 Mbps (默认：0)| -`set`
-为目录设置配额 +#### 管理参数 {#config-management-options} -`get`
-获取目录配额信息 +|项 | 说明| +|-|-| +|`--capacity value`|容量配额，单位为 GiB| +|`--inodes value`|文件数配额| +|`--trash-days value`|文件被自动清理前在回收站内保留的天数| +|`--encrypt-secret`|如果密钥之前以原格式存储，则加密密钥 (默认值：false)| +|`--min-client-version value`|允许连接的最小客户端版本| +|`--max-client-version value`|允许连接的最大客户端版本| +|`--dir-stats`|开启目录统计，这是快速汇总和目录配额所必需的 (默认值：false)| -`delete, del`
-删除目录配额 +### `juicefs quota` {#quota} -`list, ls`
-列出所有目录配额 +管理目录配额 -`check`
-检查目录配额的一致性 +#### 概览 -##### 选项 +```shell +juicefs quota command [command options] META-URL -`--path value`
-卷中目录的全路径 +# 为目录设置配额 +juicefs quota set redis://localhost --path /dir1 --capacity 1 --inodes 100 -`--capacity value`
-目录空间硬限制，单位 GiB (默认：0) +# 获取目录配额信息 +juicefs quota get redis://localhost --path /dir1 -`--inodes value`
-用于硬限制目录 inode 数 (默认：0) +# 列出所有目录配额 +juicefs quota list redis://localhost -`--repair`
-修复不一致配额 (默认：false) +# 删除目录配额 +juicefs quota delete redis://localhost --path /dir1 -`--strict`
-在严格模式下计算目录的总使用量 (注意：对于大目录可能很慢) (默认：false) +# 检查目录配额的一致性 +juicefs quota check redis://localhost +``` -##### 示例 +#### 参数 -```shell -juicefs quota set redis://localhost --path /dir1 --capacity 1 --inodes 100 -juicefs quota get redis://localhost --path /dir1 -juicefs quota list redis://localhost -juicefs quota delete redis://localhost --path /dir1 -``` +|项 | 说明| +|-|-| +|`META-URL`|用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md)。| +|`--path value`|卷中目录的全路径| +|`--capacity value`|目录空间硬限制，单位 GiB (默认：0)| +|`--inodes value`|用于硬限制目录 inode 数 (默认：0)| +|`--repair`|修复不一致配额 (默认：false)| +|`--strict`|在严格模式下计算目录的总使用量 (注意：对于大目录可能很慢) (默认：false)| -#### `juicefs destroy` {#destroy} +### `juicefs destroy` {#destroy} 销毁一个已经存在的文件系统，将会清空元数据引擎与对象存储中的相关数据。详见[「如何销毁文件系统」](../administration/destroy.md)。 -##### 使用 +#### 概览 -``` +```shell juicefs destroy [command options] META-URL UUID -``` - -##### 选项 - -`--yes, -y`
-对所有提示自动回答 "yes" 并以非交互方式运行 (默认值：false) - -`--force`
-跳过合理性检查并强制销毁文件系统 (默认：false) -##### 示例 - -```shell juicefs destroy redis://localhost e94d66a8-2339-4abd-b8d8-6812df737892 ``` -#### `juicefs gc` {#gc} +#### 参数 -用来处理「对象泄漏」，以及因为覆盖写而产生的碎片数据的命令。详见[「状态检查 & 维护」](../administration/status_check_and_maintenance.md#gc)。 +|项 | 说明| +|-|-| +|`--yes, -y`|对所有提示自动回答 "yes" 并以非交互方式运行 (默认值：false)| +|`--force`|跳过合理性检查并强制销毁文件系统 (默认：false)| -##### 使用 +### `juicefs gc` {#gc} -``` -juicefs gc [command options] META-URL -``` - -##### 选项 - -`--delete`
-删除泄漏的对象 (默认：false) - -`--compact`
-整理所有文件的碎片 (默认：false). - -`--threads value`
-用于删除泄漏对象的线程数 (默认：10) +用来处理「对象泄漏」，以及因为覆盖写而产生的碎片数据的命令。详见[「状态检查 & 维护」](../administration/status_check_and_maintenance.md#gc)。 -##### 示例 +#### 概览 ```shell +juicefs gc [command options] META-URL + # 只检查，没有更改的能力 -$ juicefs gc redis://localhost +juicefs gc redis://localhost # 触发所有 slices 的压缩 -$ juicefs gc redis://localhost --compact +juicefs gc redis://localhost --compact # 删除泄露的对象 -$ juicefs gc redis://localhost --delete +juicefs gc redis://localhost --delete ``` -#### `juicefs fsck` {#fsck} - -检查文件系统一致性。 - -##### 使用 +#### 参数 -``` -juicefs fsck [command options] META-URL -``` +|项 | 说明| +|-|-| +|`--delete`|删除泄漏的对象 (默认：false)| +|`--compact`|整理所有文件的碎片 (默认：false).| +|`--threads=10`|用于删除泄漏对象的线程数 (默认：10)| -##### 选项 +### `juicefs fsck` {#fsck} -`--path value`
-待检查的 JuiceFS 中的绝对路径 - -`--repair`
-发现损坏后尽可能修复 (默认：false) - -`--recursive, -r`
-递归检查或修复 (默认值：false) - -`--sync-dir-stat`
-同步所有目录的状态，即使他们没有损坏 (注意：巨大的文件树可能会花费很长时间) (默认：false) +检查文件系统一致性。 -##### 示例 +#### 概览 ```shell +juicefs fsck [command options] META-URL + juicefs fsck redis://localhost ``` -#### `juicefs restore` {#restore} +#### 参数 + +|项 | 说明| +|-|-| +|`--path value`|待检查的 JuiceFS 中的绝对路径| +|`--repair`|发现损坏后尽可能修复 (默认：false)| +|`--recursive, -r`|递归检查或修复 (默认值：false)| +|`--sync-dir-stat`|同步所有目录的状态，即使他们没有损坏 (注意：巨大的文件树可能会花费很长时间) (默认：false)| + +### `juicefs restore` {#restore} 重新构建回收站文件的树结构，并将它们放回原始目录。 -##### 使用 +#### 概览 ```shell juicefs restore [command options] META HOUR ... -``` - -##### 选项 - -`--put-back value`
-将恢复的文件移动到原始目录 (默认值：false) -`--threads value`
-线程数 (默认：10) - -##### 示例 - -```shell juicefs restore redis://localhost/1 2023-05-10-01 ``` -#### `juicefs dump` {#dump} +#### 参数 -导出元数据。阅读[「元数据备份」](../administration/metadata_dump_load.md#backup)以了解更多。 +|项 | 说明| +|-|-| +|`--put-back value`|将恢复的文件移动到原始目录 (默认值：false)| +|`--threads value`|线程数 (默认：10)| -##### 使用 +### `juicefs dump` {#dump} -``` -juicefs dump [command options] META-URL [FILE] -``` +导出元数据。阅读[「元数据备份」](../administration/metadata_dump_load.md#backup)以了解更多。 -- META-URL: 用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md) -- FILE: 导出文件路径，如果不指定，则会导出到标准输出。如果文件名以 `.gz` 结尾，将会自动压缩 +#### 概览 ```shell +juicefs dump [command options] META-URL [FILE] + # 导出元数据至 meta-dump.json juicefs dump redis://localhost meta-dump.json @@ -489,1105 +373,589 @@ juicefs dump redis://localhost meta-dump.json juicefs dump redis://localhost sub-meta-dump.json --subdir /dir/in/jfs ``` -##### 选项 +#### 参数 -`--subdir value`
-只导出指定子目录的元数据。 +|项 | 说明| +|-|-| +|`META-URL`|用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md)。| +|`FILE`|导出文件路径，如果不指定，则会导出到标准输出。如果文件名以 `.gz` 结尾，将会自动压缩。| +|`--subdir=path`|只导出指定子目录的元数据。| +|`--keep-secret-key`|导出对象存储认证信息，默认为 `false`。由于是明文导出，使用时注意数据安全。如果导出文件不包含对象存储认证信息，后续的导入完成后，需要用 [`juicefs config`](#config) 重新配置对象存储认证信息。| -`--keep-secret-key`
-导出对象存储认证信息，默认为 `false`。由于是明文导出，使用时注意数据安全。如果导出文件不包含对象存储认证信息，后续的导入完成后，需要用 [`juicefs config`](#config) 重新配置对象存储认证信息。 - -#### `juicefs load` {#load} +### `juicefs load` {#load} 将元数据导入一个空的文件系统。阅读[「元数据恢复与迁移」](../administration/metadata_dump_load.md#recovery-and-migration)以了解更多。 -##### 使用 +#### 概览 -``` +```shell juicefs load [command options] META-URL [FILE] -``` -- META-URL: 用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md)。 -- FILE: 导入文件路径，如果不指定，则会从标准输入导入。如果文件名以 `.gz` 结尾，将会自动解压。 - -```shell # 将元数据备份文件 meta-dump.json 导入数据库 juicefs load redis://127.0.0.1:6379/1 meta-dump.json ``` -##### 选项 +#### 参数 -`--encrypt-rsa-key value`
-加密所使用的 RSA 私钥文件路径。 +|项 | 说明| +|-|-| +|`META-URL`|用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md)。| +|`FILE`|导入文件路径，如果不指定，则会从标准输入导入。如果文件名以 `.gz` 结尾，将会自动解压。| +|`--encrypt-rsa-key=path`|加密所使用的 RSA 私钥文件路径。| +|`--encrypt-algo=aes256gcm-rsa`|加密算法，默认为 `aes256gcm-rsa`。| -`--encrypt-algo value`
-加密算法，默认为 `aes256gcm-rsa`。 +## 检视 {#inspector} -### 检视 - -#### `juicefs status`{#status} +### `juicefs status` {#status} 显示 JuiceFS 的状态。 -##### 使用 +#### 概览 -``` +```shell juicefs status [command options] META-URL -``` - -##### 选项 - -`--session value, -s value`
-展示指定会话 (SID) 的具体信息 (默认：0) -`--more, -m`
-显示更多的统计信息，可能需要很长时间 (默认值：false) - -##### 示例 - -```shell juicefs status redis://localhost ``` -#### `juicefs stats` {#stats} +#### 参数 -展示实时的性能统计信息。 +|项 | 说明| +|-|-| +|`--session=0, -s 0`|展示指定会话 (SID) 的具体信息 (默认：0)| +|`--more, -m`|显示更多的统计信息，可能需要很长时间 (默认值：false)| -##### 使用 +### `juicefs stats` {#stats} -``` -juicefs stats [command options] MOUNTPOINT -``` - -##### 选项 +展示实时的性能统计信息，阅读[「实时性能监控」](../administration/fault_diagnosis_and_analysis.md#performance-monitor)以了解更多。 -`--schema value`
-控制输出内容的标题字符串 (u: `usage`, f: `fuse`, m: `meta`, c: `blockcache`, o: `object`, g: `go`) (默认："ufmco") - -`--interval value`
-更新间隔；单位为秒 (默认：1) - -`--verbosity value`
-详细级别；通常 0 或 1 已足够 (默认：0) - -##### 示例 +#### 概览 ```shell -$ juicefs stats /mnt/jfs - -# 更多的指标 -$ juicefs stats /mnt/jfs -l 1 -``` - -#### `juicefs profile` {#profile} - -分析[访问日志](../administration/fault_diagnosis_and_analysis.md#access-log)。 +juicefs stats [command options] MOUNTPOINT -##### 使用 +juicefs stats /mnt/jfs +# 更多的指标 +juicefs stats /mnt/jfs -l 1 ``` -juicefs profile [command options] MOUNTPOINT/LOGFILE -``` - -##### 选项 -`--uid value, -u value`
-仅跟踪指定 UIDs (用逗号分隔) +#### 参数 -`--gid value, -g value`
-仅跟踪指定 GIDs (用逗号分隔) +|项 | 说明| +|-|-| +|`--schema=ufmco`|控制输出内容的标题字符串，默认为 `ufmco`，含义如下：
`u`：usage
`f`：FUSE
`m`：metadata
`c`：block cache
`o`：object storage
`g`：Go| +|`--interval=1`|更新间隔；单位为秒 (默认：1)| +|`--verbosity=0`|详细级别；通常 0 或 1 已足够 (默认：0)| -`--pid value, -p value`
-仅跟踪指定 PIDs (用逗号分隔) +### `juicefs profile` {#profile} -`--interval value`
-显示间隔；在回放模式中将其设置为 0 可以立即得到整体的统计结果；单位为秒 (默认：2) +展示基于[文件系统访问日志](../administration/fault_diagnosis_and_analysis.md#access-log)的实时监控数据，阅读[「实时性能监控」](../administration/fault_diagnosis_and_analysis.md#performance-monitor)以了解更多。 -##### 示例 +#### 概览 ```shell +juicefs profile [command options] MOUNTPOINT/LOGFILE + # 监控实时操作 -$ juicefs profile /mnt/jfs +juicefs profile /mnt/jfs # 重放访问日志 -$ cat /mnt/jfs/.accesslog > /tmp/jfs.alog +cat /mnt/jfs/.accesslog > /tmp/jfs.alog # 一段时间后按 Ctrl-C 停止 “cat” 命令 -$ juicefs profile /tmp/jfs.alog +juicefs profile /tmp/jfs.alog # 分析访问日志并立即打印总统计数据 -$ juicefs profile /tmp/jfs.alog --interval 0 +juicefs profile /tmp/jfs.alog --interval 0 ``` -#### `juicefs info` {#info} +#### 参数 -显示指定路径或 inode 的内部信息。 +|项 | 说明| +|-|-| +|`--uid=value, -u value`|仅跟踪指定 UIDs (用逗号分隔)| +|`--gid=value, -g value`|仅跟踪指定 GIDs (用逗号分隔)| +|`--pid=value, -p value`|仅跟踪指定 PIDs (用逗号分隔)| +|`--interval=2`|显示间隔；在回放模式中将其设置为 0 可以立即得到整体的统计结果；单位为秒 (默认：2)| -##### 使用 +### `juicefs info` {#info} -``` -juicefs info [command options] PATH or INODE -``` - -##### 选项 - -`--inode, -i`
-使用 inode 号而不是路径 (当前目录必须在 JuiceFS 挂载点内) (默认：false) - -`--recursive, -r`
-递归获取所有子目录的概要信息（注意：当指定一个目录结构很复杂的路径时可能会耗时很长） (默认：false) - -`--strict`
-获取准确的目录概要 (注意：巨大的文件树可能会花费很长的时间) (默认：false) - -`--raw`
-显示内部原始信息 (默认：false) +显示指定路径或 inode 的内部信息。 -##### 示例 +#### 概览 ```shell +juicefs info [command options] PATH or INODE + # 检查路径 -$ juicefs info /mnt/jfs/foo +juicefs info /mnt/jfs/foo # 检查 inode -$ cd /mnt/jfs -$ juicefs info -i 100 -``` - -#### `juicefs debug` {#debug} - -从运行环境、系统日志等多个维度收集和展示信息，帮助更好地定位错误 - -##### 使用 - +cd /mnt/jfs +juicefs info -i 100 ``` -juicefs debug [command options] MOUNTPOINT -``` - -##### 选项 - -`--out-dir value`
-结果输出目录，若目录不存在则自动创建 (默认：./debug/) -`--limit value`
-收集的日志条目数，从新到旧，若不指定则收集全部条目 +#### 参数 -`--stats-sec value`
-.stats 文件采样秒数 (默认：5) +|项 | 说明| +|-|-| +|`--inode, -i`|使用 inode 号而不是路径 (当前目录必须在 JuiceFS 挂载点内) (默认：false)| +|`--recursive, -r`|递归获取所有子目录的概要信息，当指定一个目录结构很复杂的路径时可能会耗时很长） (默认：false)| +|`--strict`|获取准确的目录概要 (注意：巨大的文件树可能会花费很长的时间) (默认：false)| +|`--raw`|显示内部原始信息 (默认：false)| -`--trace-sec value`
-trace 指标采样秒数 (默认：5) +### `juicefs debug` {#debug} -`--profile-sec value`
-profile 指标采样秒数 (默认：30) +从运行环境、系统日志等多个维度收集和展示信息，帮助更好地定位错误 -##### 示例 +#### 概览 ```shell +juicefs debug [command options] MOUNTPOINT + # 收集并展示挂载点 /mnt/jfs 的各类信息 -$ juicefs debug /mnt/jfs +juicefs debug /mnt/jfs # 指定输出目录为 /var/log -$ juicefs debug --out-dir=/var/log /mnt/jfs +juicefs debug --out-dir=/var/log /mnt/jfs # 收集最后 1000 条日志条目 -$ juicefs debug --out-dir=/var/log --limit=1000 /mnt/jfs +juicefs debug --out-dir=/var/log --limit=1000 /mnt/jfs ``` -#### `juicefs summary` {#summary} +#### 参数 -用于显示目标目录的树摘要 +|项 | 说明| +|-|-| +|`--out-dir=./debug/`|结果输出目录，若目录不存在则自动创建，默认为 `./debug/`。| +|`--limit=value`|收集的日志条目数，从新到旧，若不指定则收集全部条目| +|`--stats-sec=5`|.stats 文件采样秒数 (默认：5)| +|`--trace-sec=5`|trace 指标采样秒数 (默认：5)| +|`--profile-sec=30`|profile 指标采样秒数 (默认：30)| -##### 使用 +### `juicefs summary` {#summary} -```shell -juicefs summary [command options] PATH -``` - -##### 选项 - -`--depth value, -d value`
-显示树的深度 (0 表示只显示根) (默认：2) - -`--entries value, -e value`
-显示前 N 个 entry (按大小排序)(默认：10) - -`--strict`
-显示准确的摘要，包括目录和文件 (可能很慢) (默认值：false) - -`--csv`
-以 CSV 格式打印摘要 (默认：false) +显示目标目录树摘要。 -##### 示例 +#### 概览 ```shell -$ juicefs summary /mnt/jfs/foo +juicefs summary [command options] PATH + +juicefs summary /mnt/jfs/foo # 显示最大深度为 5 -$ juicefs summary --depth 5 /mnt/jfs/foo +juicefs summary --depth 5 /mnt/jfs/foo # 显示前 20 个 entry -$ juicefs summary --entries 20 /mnt/jfs/foo +juicefs summary --entries 20 /mnt/jfs/foo # 显示准确的结果 -$ juicefs summary --strict /mnt/jfs/foo +juicefs summary --strict /mnt/jfs/foo ``` -### 服务 - -#### `juicefs mount` {#mount} - -挂载一个已经创建的文件系统。 - -你可以用任意用户执行挂载命令，不过请确保该用户对缓存目录（`--cache-dir`）有写权限，请阅读[「缓存位置」](../guide/cache_management.md#cache-dir)文档了解更多信息。 - -##### 使用 - -``` -juicefs mount [command options] META-URL MOUNTPOINT -``` - -- `META-URL`：用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md)。 -- `MOUNTPOINT`：文件系统挂载点，例如：`/mnt/jfs`、`Z:`。 - -##### 选项 - -###### 常规 - -`-d, --background`
-后台运行 (默认：false) - -`--no-syslog`
-禁用系统日志 (默认：false) - -`--log value`
-后台运行时日志文件的位置 (默认：`$HOME/.juicefs/juicefs.log` 或 `/var/log/juicefs.log`) - -`--force`
-强制挂载即使挂载点已经被相同的文件系统挂载 (默认值:false) - -`--update-fstab`
-新增／更新 `/etc/fstab` 中的条目，如果不存在将会创建一个从 `/sbin/mount.juicefs` 到 JuiceFS 可执行文件的软链接 (默认：false) - -###### FUSE - -`--enable-xattr`
-启用扩展属性 (xattr) 功能 (默认：false) - -`--enable-ioctl`
-启用 ioctl (仅支持 GETFLAGS/SETFLAGS) (默认：false) - -`--root-squash value`
-将本地 root 用户 (UID=0) 映射到一个指定用户，如 UID:GID - -`--prefix-internal`
-添加 `.jfs` 前缀到所有内部文件 (默认：false) - -`-o value`
-其他 FUSE 选项，详见 [FUSE 挂载选项](../reference/fuse_mount_options.md) - -###### 元数据 - -`--subdir value`
-将某个子目录挂载为根 (默认："") - -`--backup-meta value`
-自动备份元数据到对象存储的间隔时间；单位秒 (0 表示不备份) (默认：3600) - -`--heartbeat value`
-发送心跳的间隔（单位秒），建议所有客户端使用相同的心跳值 (默认：12) - -`--read-only`
-只读模式 (默认：false) - -`--no-bgjob`
-禁用后台任务，默认为 false，也就是说客户端会默认运行后台任务。后台任务包含： - -* 清理回收站中过期的文件（在 [`pkg/meta/base.go`](https://github.com/juicedata/juicefs/blob/main/pkg/meta/base.go) 中搜索 `cleanupDeletedFiles` 和 `cleanupTrash`） -* 清理引用计数为 0 的 Slice（在 [`pkg/meta/base.go`](https://github.com/juicedata/juicefs/blob/main/pkg/meta/base.go) 中搜索 `cleanupSlices`） -* 清理过期的客户端会话（在 [`pkg/meta/base.go`](https://github.com/juicedata/juicefs/blob/main/pkg/meta/base.go) 中搜索 `CleanStaleSessions`） - -特别地，碎片合并（Compaction）不受该选项的影响，而是随着文件读写操作，自动判断是否需要合并，然后异步执行（以 Redis 为例，在 [`pkg/meta/base.go`](https://github.com/juicedata/juicefs/blob/main/pkg/meta/redis.go) 中搜索 `compactChunk`） - -`--atime-mode value`
-控制如何更新 atime（文件最后被访问的时间）。支持以下模式： - -* `noatime`（默认），仅在文件创建和主动调用 `SetAttr` 时设置，平时访问与修改文件不影响 atime 值。考虑到更新 atime 需要运行额外的事务，对性能有影响，因此默认关闭 -* `relatime`，仅在 mtime（文件内容修改时间）或 ctime（文件元数据修改时间）比 atime 新，或者 atime 超过 24 小时没有更新时进行更新 -* `strictatime`，持续更新 atime - -`--skip-dir-nlink value`
-跳过更新目录 nlink 前的重试次数 (仅用于 TKV, 0 代表不重试) (默认：20) - -###### 元数据缓存 - -`--attr-cache value`
-属性缓存过期时间；单位为秒 (默认：1)。详见[「内核元数据缓存」](../guide/cache_management.md#kernel-metadata-cache) - -`--entry-cache value`
-文件项缓存过期时间；单位为秒 (默认：1)。详见[「内核元数据缓存」](../guide/cache_management.md#kernel-metadata-cache) - -`--dir-entry-cache value`
-目录项缓存过期时间；单位为秒 (默认：1)。详见[「内核元数据缓存」](../guide/cache_management.md#kernel-metadata-cache) - -`--open-cache value`
-打开的文件的缓存过期时间（0 代表关闭这个特性）；单位为秒 (默认：0)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--open-cache-limit value`
-允许缓存的最大文件个数 (软限制，0 代表不限制) (默认：10000) - -###### 数据存储 - -`--storage value`
-对象存储类型 (例如 `s3`、`gcs`、`oss`、`cos`) (默认：`"file"`，请参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)查看所有支持的对象存储类型) - -`--bucket value`
-为当前挂载点指定访问访对象存储的 endpoint - -`--storage-class value`
-当前客户端写入数据的存储类型 - -`--get-timeout value`
-下载一个对象的超时时间；单位为秒 (默认：60) - -`--put-timeout value`
-上传一个对象的超时时间；单位为秒 (默认：60) - -`--io-retries value`
-网络异常时的重试次数 (默认：10) - -`--max-uploads value`
-上传对象的连接数 (默认：20) +#### 参数 -`--max-deletes value`
-删除对象的连接数 (默认：10) +|项 | 说明| +|-|-| +|`--depth value, -d value`|显示树的深度 (0 表示只显示根) (默认：2)| +|`--entries value, -e value`|显示前 N 个 entry (按大小排序)(默认：10)| +|`--strict`|显示准确的摘要，包括目录和文件 (可能很慢) (默认值：false)| +|`--csv`|以 CSV 格式打印摘要 (默认：false)| -`--upload-limit value`
-上传带宽限制，单位为 Mbps (默认：0) +## 服务 {#service} -`--download-limit value`
-下载带宽限制，单位为 Mbps (默认：0) +### `juicefs mount` {#mount} -###### 数据缓存 - -`--buffer-size value`
-读写缓存的总大小；单位为 MiB (默认：300) - -`--prefetch value`
-并发预读 N 个块 (默认：1) - -`--writeback`
-后台异步上传对象 (默认：false)。阅读[「客户端写缓存」](../guide/cache_management.md#writeback)了解更多 - -`--upload-delay value`
-数据上传到对象存储的延迟时间，支持秒分时精度，对应格式分别为 ("s", "m", "h")，默认为 0 秒。如果在等待的时间内数据被应用删除，则无需再上传到对象存储，既提升了性能也节省了成本，如果数据只是临时落盘，之后会迅速删除，考虑用该选项进行优化。 - -`--cache-dir value`
-本地缓存目录路径；使用 `:`（Linux、macOS）或 `;`（Windows）隔离多个路径 (默认：`"$HOME/.juicefs/cache"` 或 `"/var/jfsCache"`)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--cache-mode value`
-缓存块的文件权限 (默认："0600") - -`--cache-size value`
-缓存对象的总大小；单位为 MiB (默认：102400)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--free-space-ratio value`
-最小剩余空间比例 (默认：0.1)。如果启用了[「客户端写缓存」](../guide/cache_management.md#writeback)，则该参数还控制着写缓存占用空间。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--cache-partial-only`
-仅缓存随机小块读 (默认：false)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--verify-cache-checksum value`
-缓存数据一致性检查级别，启用 Checksum 校验后，生成缓存文件时会对数据切分做 Checksum 并记录于文件末尾，供读缓存时进行校验。支持以下级别：
- -`--cache-eviction value`
-缓存逐出策略 (none 或 2-random) (默认值："2-random") - -`--cache-scan-interval value`
-扫描缓存目录重建内存索引的间隔 (以秒为单位) (默认："3600") - -###### 指标 - -`--metrics value`
-监控数据导出地址 (默认："127.0.0.1:9567") - -`--consul value`
-Consul 注册中心地址 (默认："127.0.0.1:8500") +挂载一个已经创建的文件系统。 -`--no-usage-report`
-不发送使用量信息 (默认：false) +JuiceFS 支持用 root 以及普通用户挂载，但由于权限不同，挂载时所使用的的缓存目录和日志文件等路径会有所区别，详见下方参数说明。 -##### 示例 +#### 概要 ```shell +juicefs mount [command options] META-URL MOUNTPOINT + # 前台挂载 -$ juicefs mount redis://localhost /mnt/jfs +juicefs mount redis://localhost /mnt/jfs # 使用带密码的 redis 后台挂载 -$ juicefs mount redis://:mypassword@localhost /mnt/jfs -d +juicefs mount redis://:mypassword@localhost /mnt/jfs -d # 更安全的方式 -$ META_PASSWORD=mypassword juicefs mount redis://localhost /mnt/jfs -d +META_PASSWORD=mypassword juicefs mount redis://localhost /mnt/jfs -d # 将一个子目录挂载为根目录 -$ juicefs mount redis://localhost /mnt/jfs --subdir /dir/in/jfs +juicefs mount redis://localhost /mnt/jfs --subdir /dir/in/jfs -# 启用 “writeback” 模式，这可以提高性能，但有丢失对象的风险 -$ juicefs mount redis://localhost /mnt/jfs -d --writeback +# 开启写缓存（writeback）模式，可以提升写入性能但同时有数据丢失风险 +juicefs mount redis://localhost /mnt/jfs -d --writeback # 开启只读模式 -$ juicefs mount redis://localhost /mnt/jfs -d --read-only +juicefs mount redis://localhost /mnt/jfs -d --read-only # 关闭元数据自动备份 -$ juicefs mount redis://localhost /mnt/jfs --backup-meta 0 -``` +juicefs mount redis://localhost /mnt/jfs --backup-meta 0 +``` + +#### 参数 + +|项 | 说明| +|-|-| +|`META-URL`|用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md)。| +|`MOUNTPOINT`|文件系统挂载点，例如：`/mnt/jfs`、`Z:`。| +|`-d, --background`|后台运行，默认为 false。| +|`--no-syslog`|禁用系统日志，默认为 false。| +|`--log=path`|后台运行时日志文件的位置 (默认：`$HOME/.juicefs/juicefs.log` 或 `/var/log/juicefs.log`)| +|`--force`|强制挂载即使挂载点已经被相同的文件系统挂载 (默认值:false)| +|`--update-fstab`|新增／更新 `/etc/fstab` 中的条目，如果不存在将会创建一个从 `/sbin/mount.juicefs` 到 JuiceFS 可执行文件的软链接，默认为 false。| + +#### FUSE 相关参数 {#mount-fuse-options} + +|项 | 说明| +|-|-| +|`--enable-xattr`|启用扩展属性 (xattr) 功能，默认为 false。| +|`--enable-ioctl`|启用 ioctl (仅支持 GETFLAGS/SETFLAGS) (默认：false)| +|`--root-squash value`|将本地 root 用户 (UID=0) 映射到一个指定用户，如 UID:GID| +|`--prefix-internal`|添加 `.jfs` 前缀到所有内部文件 (默认：false)| +|`-o value`|其他 FUSE 选项，详见 [FUSE 挂载选项](../reference/fuse_mount_options.md)| + +#### 元数据相关参数 {#mount-metadata-options} + +|项 | 说明| +|-|-| +|`--subdir=value`|挂载指定的子目录，默认挂载整个文件系统。| +|`--backup-meta=3600`|自动备份元数据到对象存储的间隔时间；单位秒，默认 3600，设为 0 表示不备份。| +|`--heartbeat=12`|发送心跳的间隔（单位秒），建议所有客户端使用相同的心跳值 (默认：12)| +|`--read-only`|启用只读模式挂载。| +|`--no-bgjob`|禁用后台任务，默认为 false，也就是说客户端会默认运行后台任务。后台任务包含：
特别地，碎片合并（Compaction）不受该选项的影响，而是随着文件读写操作，自动判断是否需要合并，然后异步执行（以 Redis 为例，在 [`pkg/meta/base.go`](https://github.com/juicedata/juicefs/blob/main/pkg/meta/redis.go) 中搜索 `compactChunk`）| +|`--atime-mode=noatime`|控制如何更新 atime（文件最后被访问的时间）。支持以下模式：
| +|`--skip-dir-nlink value`|跳过更新目录 nlink 前的重试次数 (仅用于 TKV, 0 代表不重试) (默认：20)| + +#### 元数据缓存参数 {#mount-metadata-cache-options} + +元数据缓存的介绍和使用，详见[「内核元数据缓存」](../guide/cache_management.md#kernel-metadata-cache)及[「客户端内存元数据缓存」](../guide/cache_management.md#client-memory-metadata-cache)。 + +|项 | 说明| +|-|-| +|`--attr-cache=1`|属性缓存过期时间；单位为秒，默认为 1。| +|`--entry-cache=1`|文件项缓存过期时间；单位为秒，默认为 1。| +|`--dir-entry-cache=1`|目录项缓存过期时间；单位为秒，默认为 1。| +|`--open-cache=0`|打开的文件的缓存过期时间，单位为秒，默认为 0，代表关闭该特性。| +|`--open-cache-limit value`|允许缓存的最大文件个数 (软限制，0 代表不限制) (默认：10000)| + +#### 数据存储参数 {#mount-data-storage-options} + +|项 | 说明| +|-|-| +|`--storage=file`|对象存储类型 (例如 `s3`、`gcs`、`oss`、`cos`) (默认：`"file"`，参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)查看所有支持的对象存储类型)| +|`--bucket=value`|为当前挂载点指定访问对象存储的 Endpoint。| +|`--storage-class value`|当前客户端写入数据的存储类型| +|`--get-timeout=60`|下载一个对象的超时时间；单位为秒 (默认：60)| +|`--put-timeout=60`|上传一个对象的超时时间；单位为秒 (默认：60)| +|`--io-retries=10`|网络异常时的重试次数 (默认：10)| +|`--max-uploads=20`|上传对象的连接数 (默认：20)| +|`--max-deletes=10`|删除对象的连接数 (默认：10)| +|`--upload-limit=0`|上传带宽限制，单位为 Mbps (默认：0)| +|`--download-limit=0`|下载带宽限制，单位为 Mbps (默认：0)| + +#### 数据缓存相关参数 {#mount-data-cache-options} + +|项 | 说明| +|-|-| +|`--buffer-size=300`|读写缓冲区的总大小；单位为 MiB (默认：300)。阅读[「读写缓冲区」](../guide/cache_management.md#buffer-size)了解更多。| +|`--prefetch=1`|并发预读 N 个块 (默认：1)。阅读[「客户端读缓存」](../guide/cache_management.md#client-read-cache)了解更多。| +|`--writeback`|后台异步上传对象，默认为 false。阅读[「客户端写缓存」](../guide/cache_management.md#writeback)了解更多。| +|`--upload-delay=0`|启用 `--writeback` 后，可以使用该选项控制数据延迟上传到对象存储，默认为 0 秒，相当于写入后立刻上传。该选项也支持 `s`（秒）、`m`（分）、`h`（时）这些单位。如果在等待的时间内数据被应用删除，则无需再上传到对象存储。如果数据只是临时落盘，可以考虑用该选项节约资源。阅读[「客户端写缓存」](../guide/cache_management.md#writeback)了解更多。| +|`--cache-dir=value`|本地缓存目录路径；使用 `:`（Linux、macOS）或 `;`（Windows）隔离多个路径 (默认：`$HOME/.juicefs/cache` 或 `/var/jfsCache`)。阅读[「客户端读缓存」](../guide/cache_management.md#client-read-cache)了解更多。| +|`--cache-mode value`|缓存块的文件权限 (默认："0600")| +|`--cache-size=102400`|缓存对象的总大小；单位为 MiB (默认：102400)。阅读[「客户端读缓存」](../guide/cache_management.md#client-read-cache)了解更多。| +|`--free-space-ratio=0.1`|最小剩余空间比例，默认为 0.1。如果启用了[「客户端写缓存」](../guide/cache_management.md#writeback)，则该参数还控制着写缓存占用空间。阅读[「客户端读缓存」](../guide/cache_management.md#client-read-cache)了解更多。| +|`--cache-partial-only`|仅缓存随机小块读，默认为 false。阅读[「客户端读缓存」](../guide/cache_management.md#client-read-cache)了解更多。| +|`--verify-cache-checksum=full`|缓存数据一致性检查级别，启用 Checksum 校验后，生成缓存文件时会对数据切分做 Checksum 并记录于文件末尾，供读缓存时进行校验。支持以下级别：
| +|`--cache-eviction value`|缓存逐出策略 (none 或 2-random) (默认值："2-random")| +|`--cache-scan-interval value`|扫描缓存目录重建内存索引的间隔 (以秒为单位) (默认："3600")| + +#### 监控相关参数 {#mount-metrics-options} + +|项 | 说明| +|-|-| +|`--metrics=127.0.0.1:9567`|监控数据导出地址，默认为 `127.0.0.1:9567`。| +|`--consul=127.0.0.1:8500`|Consul 注册中心地址，默认为 `127.0.0.1:8500`。| +|`--no-usage-report`|不发送使用量信息 (默认：false)| + +### `juicefs umount` {#umount} + +卸载 JuiceFS 文件系统。 + +#### 概要 -#### `juicefs umount`{#umount} - -卸载一个文件文件系统。 - -##### 使用 - -``` +```shell juicefs umount [command options] MOUNTPOINT -``` - -##### 选项 - -`-f, --force`
-强制卸载一个忙碌的文件系统 (默认：false) -`--flush`
-等待所有暂存块被刷新 (默认值：false) - -##### 示例 - -```shell juicefs umount /mnt/jfs ``` -#### `juicefs gateway`{#gateway} - -启动一个 S3 兼容的网关。 - -##### 使用 - -``` -juicefs gateway [command options] META-URL ADDRESS -``` - -- **META-URL**：用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md)。 -- **ADDRESS**：S3 网关地址和监听的端口，例如：`localhost:9000` - -##### 选项 - -###### 常规 - -`--access-log value`
-访问日志的路径 - -`--no-banner`
-禁用 MinIO 的启动信息 (默认：false) - -`--multi-buckets`
-使用第一级目录作为存储桶 (默认：false) - -`--keep-etag`
-保留对象上传时的 ETag (默认：false) - -`--umask value` -新文件和新目录的 umask 的八进制格式 (默认值:“022”) - -###### 元数据 - -`--subdir value`
-将某个子目录挂载为根 (默认："") - -`--backup-meta value`
-自动备份元数据到对象存储的间隔时间；单位秒 (0 表示不备份) (默认：3600) - -`--heartbeat value`
-发送心跳的间隔 (秒);建议所有客户端使用相同的心跳值 (默认：12) - -`--read-only`
-只读模式 (默认：false) - -`--no-bgjob`
-禁用后台作业（清理、备份等）（默认值：false） - -`--atime-mode value`
-控制如何更新 atime（文件最后被访问的时间）。支持以下模式： - -* `noatime`（默认），仅在文件创建和主动调用 `SetAttr` 时设置，平时访问与修改文件不影响 atime 值。考虑到更新 atime 需要运行额外的事务，对性能有影响，因此默认关闭 -* `relatime`，仅在 mtime（文件内容修改时间）或 ctime（文件元数据修改时间）比 atime 新，或者 atime 超过 24 小时没有更新时进行更新 -* `strictatime`，持续更新 atime - -`--skip-dir-nlink value`
-跳过更新目录 nlink 前的重试次数 (仅用于 TKV, 0 代表不重试) (默认：20) - -###### 元数据缓存 - -`--attr-cache value`
-属性缓存过期时间；单位为秒 (默认：1)。详见[「内核元数据缓存」](../guide/cache_management.md#kernel-metadata-cache) - -`--entry-cache value`
-文件项缓存过期时间；单位为秒 (默认：1)。详见[「内核元数据缓存」](../guide/cache_management.md#kernel-metadata-cache) - -`--dir-entry-cache value`
-目录项缓存过期时间；单位为秒 (默认：1)。详见[「内核元数据缓存」](../guide/cache_management.md#kernel-metadata-cache) - -`--open-cache value`
-打开的文件的缓存过期时间（0 代表关闭这个特性）；单位为秒 (默认：0)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--open-cache-limit value`
-允许缓存的最大文件个数 (软限制，0 代表不限制) (默认：10000) +#### 参数 -###### 数据存储 +|项 | 说明| +|-|-| +|`-f, --force`|强制卸载一个忙碌的文件系统 (默认：false)| +|`--flush`|等待所有暂存块被刷新 (默认值：false)| -`--storage value`
-对象存储类型 (例如 `s3`、`gcs`、`oss`、`cos`) (默认：`"file"`，请参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)查看所有支持的对象存储类型) +### `juicefs gateway` {#gateway} -`--bucket value`
-为当前挂载点指定访问访对象存储的 endpoint +启动一个 S3 兼容的网关，详见[「配置 JuiceFS S3 网关」](../deployment/s3_gateway.md)。 -`--storage-class value`
-当前客户端写入数据的存储类型 - -`--get-timeout value`
-下载一个对象的超时时间；单位为秒 (默认：60) - -`--put-timeout value`
-上传一个对象的超时时间；单位为秒 (默认：60) - -`--io-retries value`
-网络异常时的重试次数 (默认：10) - -`--max-uploads value`
-上传对象的连接数 (默认：20) - -`--max-deletes value`
-删除对象的连接数 (默认：10) - -`--upload-limit value`
-上传带宽限制，单位为 Mbps (默认：0) - -`--download-limit value`
-下载带宽限制，单位为 Mbps (默认：0) - -###### 数据缓存 - -`--buffer-size value`
-读写缓存的总大小；单位为 MiB (默认：300) - -`--prefetch value`
-并发预读 N 个块 (默认：1) - -`--writeback`
-后台异步上传对象 (默认：false)。阅读[「客户端写缓存」](../guide/cache_management.md#writeback)了解更多 - -`--upload-delay value`
-数据上传到对象存储的延迟时间，支持秒分时精度，对应格式分别为 ("s", "m", "h")，默认为 0 秒。如果在等待的时间内数据被应用删除，则无需再上传到对象存储，既提升了性能也节省了成本，如果数据只是临时落盘，之后会迅速删除，考虑用该选项进行优化。 - -`--cache-dir value`
-本地缓存目录路径；使用 `:`（Linux、macOS）或 `;`（Windows）隔离多个路径 (默认：`"$HOME/.juicefs/cache"` 或 `"/var/jfsCache"`)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--cache-mode value`
-缓存块的文件权限 (默认："0600") - -`--cache-size value`
-缓存对象的总大小；单位为 MiB (默认：102400)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--free-space-ratio value`
-最小剩余空间比例 (默认：0.1)。如果启用了[「客户端写缓存」](../guide/cache_management.md#writeback)，则该参数还控制着写缓存占用空间。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--cache-partial-only`
-仅缓存随机小块读 (默认：false)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--verify-cache-checksum value`
-缓存数据一致性检查级别，启用 Checksum 校验后，生成缓存文件时会对数据切分做 Checksum 并记录于文件末尾，供读缓存时进行校验。支持以下级别：
- -`--cache-eviction value`
-缓存逐出策略 (none 或 2-random) (默认值："2-random") - -`--cache-scan-interval value`
-扫描缓存目录重建内存索引的间隔 (以秒为单位) (默认："3600") - -###### 指标 - -`--metrics value`
-监控数据导出地址 (默认："127.0.0.1:9567") - -`--consul value`
-Consul 注册中心地址 (默认："127.0.0.1:8500") - -`--no-usage-report`
-不发送使用量信息 (默认：false) - -##### 示例 +#### 概览 ```shell +juicefs gateway [command options] META-URL ADDRESS + export MINIO_ROOT_USER=admin export MINIO_ROOT_PASSWORD=12345678 juicefs gateway redis://localhost localhost:9000 ``` -#### `juicefs webdav` {#webdav} - -启动一个 WebDAV 服务。 - -##### 使用 - -``` -juicefs webdav [command options] META-URL ADDRESS -``` - -- **META-URL**：用于元数据存储的数据库 URL，详情查看「[JuiceFS 支持的元数据引擎](../guide/how_to_set_up_metadata_engine.md)」。 -- **ADDRESS**：WebDAV 服务监听的地址与端口，例如：`localhost:9007` - -##### 选项 - -###### 常规 - -`--cert-file value`
-HTTPS 证书文件 - -`--key-file value`
-HTTPS 密钥文件 - -`--gzip`
-通过 gzip 压缩提供的文件（默认值：false） - -`--disallowList`
-禁止列出目录（默认值：false） - -`--access-log value`
-访问日志的路径 - -###### 元数据 - -`--subdir value`
-将某个子目录挂载为根 (默认："") - -`--backup-meta value`
-自动备份元数据到对象存储的间隔时间；单位秒 (0 表示不备份) (默认：3600) - -`--heartbeat value`
-发送心跳的间隔 (秒);建议所有客户端使用相同的心跳值 (默认：12) - -`--read-only`
-只读模式 (默认：false) - -`--no-bgjob`
-禁用后台作业（清理、备份等）（默认值：false） - -`--atime-mode value`
-控制如何更新 atime（文件最后被访问的时间）。支持以下模式： - -* `noatime`（默认），仅在文件创建和主动调用 `SetAttr` 时设置，平时访问与修改文件不影响 atime 值。考虑到更新 atime 需要运行额外的事务，对性能有影响，因此默认关闭 -* `relatime`，仅在 mtime（文件内容修改时间）或 ctime（文件元数据修改时间）比 atime 新，或者 atime 超过 24 小时没有更新时进行更新 -* `strictatime`，持续更新 atime - -`--skip-dir-nlink value`
-跳过更新目录 nlink 前的重试次数 (仅用于 TKV, 0 代表不重试) (默认：20) - -###### 元数据缓存 - -`--attr-cache value`
-属性缓存过期时间；单位为秒 (默认：1)。详见[「内核元数据缓存」](../guide/cache_management.md#kernel-metadata-cache) - -`--entry-cache value`
-文件项缓存过期时间；单位为秒 (默认：1)。详见[「内核元数据缓存」](../guide/cache_management.md#kernel-metadata-cache) - -`--dir-entry-cache value`
-目录项缓存过期时间；单位为秒 (默认：1)。详见[「内核元数据缓存」](../guide/cache_management.md#kernel-metadata-cache) - -`--open-cache value`
-打开的文件的缓存过期时间（0 代表关闭这个特性）；单位为秒 (默认：0)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--open-cache-limit value`
-允许缓存的最大文件个数 (软限制，0 代表不限制) (默认：10000) - -###### 数据存储 - -`--storage value`
-对象存储类型 (例如 `s3`、`gcs`、`oss`、`cos`) (默认：`"file"`，请参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)查看所有支持的对象存储类型) - -`--bucket value`
-为当前挂载点指定访问访对象存储的 endpoint +#### 参数 -`--storage-class value`
-当前客户端写入数据的存储类型 +除下方列出的参数，该命令还与 `juicefs mount` 共享参数，因此需要结合 [`mount`](#mount) 一起参考。 -`--get-timeout value`
-下载一个对象的超时时间；单位为秒 (默认：60) +|项 | 说明| +|-|-| +|`META-URL`|用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md)。| +|`ADDRESS`|S3 网关地址和监听的端口，例如：`localhost:9000`| +|`--access-log=path`|访问日志的路径| +|`--no-banner`|禁用 MinIO 的启动信息 (默认：false)| +|`--multi-buckets`|使用第一级目录作为存储桶 (默认：false)| +|`--keep-etag`|保留对象上传时的 ETag (默认：false)| +|`--umask=022`|新文件和新目录的 umask 的八进制格式 (默认值：022)| -`--put-timeout value`
-上传一个对象的超时时间；单位为秒 (默认：60) +### `juicefs webdav` {#webdav} -`--io-retries value`
-网络异常时的重试次数 (默认：10) +启动一个 WebDAV 服务，阅读[「配置 WebDAV 服务」](../deployment/webdav.md)以了解更多。 -`--max-uploads value`
-上传对象的连接数 (默认：20) - -`--max-deletes value`
-删除对象的连接数 (默认：10) - -`--upload-limit value`
-上传带宽限制，单位为 Mbps (默认：0) - -`--download-limit value`
-下载带宽限制，单位为 Mbps (默认：0) - -###### 数据缓存 - -`--buffer-size value`
-读写缓存的总大小；单位为 MiB (默认：300) - -`--prefetch value`
-并发预读 N 个块 (默认：1) - -`--writeback`
-后台异步上传对象 (默认：false)。阅读[「客户端写缓存」](../guide/cache_management.md#writeback)了解更多 - -`--upload-delay value`
-数据上传到对象存储的延迟时间，支持秒分时精度，对应格式分别为 ("s", "m", "h")，默认为 0 秒。如果在等待的时间内数据被应用删除，则无需再上传到对象存储，既提升了性能也节省了成本，如果数据只是临时落盘，之后会迅速删除，考虑用该选项进行优化。 - -`--cache-dir value`
-本地缓存目录路径；使用 `:`（Linux、macOS）或 `;`（Windows）隔离多个路径 (默认：`"$HOME/.juicefs/cache"` 或 `"/var/jfsCache"`)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--cache-mode value`
-缓存块的文件权限 (默认："0600") - -`--cache-size value`
-缓存对象的总大小；单位为 MiB (默认：102400)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--free-space-ratio value`
-最小剩余空间比例 (默认：0.1)。如果启用了[「客户端写缓存」](../guide/cache_management.md#writeback)，则该参数还控制着写缓存占用空间。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--cache-partial-only`
-仅缓存随机小块读 (默认：false)。阅读[「缓存」](../guide/cache_management.md)了解更多 - -`--verify-cache-checksum value`
-缓存数据一致性检查级别，启用 Checksum 校验后，生成缓存文件时会对数据切分做 Checksum 并记录于文件末尾，供读缓存时进行校验。支持以下级别：
- -`--cache-eviction value`
-缓存逐出策略 (none 或 2-random) (默认值："2-random") - -`--cache-scan-interval value`
-扫描缓存目录重建内存索引的间隔 (以秒为单位) (默认：3600) - -###### 指标 - -`--metrics value`
-监控数据导出地址 (默认："127.0.0.1:9567") - -`--consul value`
-Consul 注册中心地址 (默认："127.0.0.1:8500") - -`--no-usage-report`
-不发送使用量信息 (默认：false) - -##### 示例 +#### 概览 ```shell -juicefs webdav redis://localhost localhost:9007 -``` - -### 工具 - -#### `juicefs bench` {#bench} - -对指定的路径做基准测试，包括对大文件和小文件的读/写/获取属性操作。 - -##### 使用 +juicefs webdav [command options] META-URL ADDRESS +juicefs webdav redis://localhost localhost:9007 ``` -juicefs bench [command options] PATH -``` - -有关 `bench` 子命令的详细介绍，请参考[文档](../benchmark/performance_evaluation_guide.md#juicefs-bench)。 -##### 选项 +#### 参数 -`--block-size value`
-块大小；单位为 MiB (默认：1) +除下方列出的参数，该命令还与 `juicefs mount` 共享参数，因此需要结合 [`mount`](#mount) 参数一起参考。 -`--big-file-size value`
-大文件大小；单位为 MiB (默认：1024) +|项 | 说明| +|-|-| +|`META-URL`|用于元数据存储的数据库 URL，详情查看[「JuiceFS 支持的元数据引擎」](../guide/how_to_set_up_metadata_engine.md)。| +|`ADDRESS`|WebDAV 服务监听的地址与端口，例如：`localhost:9007`| +|`--cert-file`|HTTPS 证书文件| +|`--key-file`|HTTPS 密钥文件| +|`--gzip`|通过 gzip 压缩提供的文件（默认值：false）| +|`--disallowList`|禁止列出目录（默认值：false）| +|`--access-log=path`|访问日志的路径| -`--small-file-size value`
-小文件大小；单位为 MiB (默认：0.1) +## 工具 {#tool} -`--small-file-count value`
-小文件数量 (默认：100) +### `juicefs bench` {#bench} -`--threads value, -p value`
-并发线程数 (默认：1) +对指定的路径做基准测试，包括对大文件和小文件的读/写/获取属性操作。详细介绍参考[文档](../benchmark/performance_evaluation_guide.md#juicefs-bench)。 -##### 示例 +#### 概览 ```shell +juicefs bench [command options] PATH + # 使用4个线程运行基准测试 -$ juicefs bench /mnt/jfs -p 4 +juicefs bench /mnt/jfs -p 4 # 只运行小文件的基准测试 -$ juicefs bench /mnt/jfs --big-file-size 0 +juicefs bench /mnt/jfs --big-file-size 0 ``` -#### `juicefs objbench` {#objbench} - -测试对象存储接口的正确性与基本性能 - -##### 使用 - -```shell -juicefs objbench [command options] BUCKET -``` - -有关 `objbench` 子命令的详细介绍，请参考[文档](../benchmark/performance_evaluation_guide.md#juicefs-objbench)。 - -##### 选项 - -`--storage value`
-对象存储类型 (例如 `s3`、`gcs`、`oss`、`cos`) (默认：`"file"`，请参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)查看所有支持的对象存储类型) - -`--access-key value`
-对象存储的 Access Key (也可通过环境变量 `ACCESS_KEY` 设置) - -`--secret-key value`
-对象存储的 Secret Key (也可通过环境变量 `SECRET_KEY` 设置) +#### 参数 -`--block-size value`
-每个 IO 块的大小（以 KiB 为单位）（默认值：4096） +|项 | 说明| +|-|-| +|`--block-size=1`|块大小；单位为 MiB (默认：1)| +|`--big-file-size=1024`|大文件大小；单位为 MiB (默认：1024)| +|`--small-file-size=0.1`|小文件大小；单位为 MiB (默认：0.1)| +|`--small-file-count=100`|小文件数量 (默认：100)| +|`--threads=1, -p 1`|并发线程数 (默认：1)| -`--big-object-size value`
-大文件的大小（以 MiB 为单位）（默认值：1024） +### `juicefs objbench` {#objbench} -`--small-object-size value`
-每个小文件的大小（以 KiB 为单位）（默认值：128） +测试对象存储接口的正确性与基本性能，详细介绍参考[文档](../benchmark/performance_evaluation_guide.md#juicefs-objbench)。 -`--small-objects value`
-小文件的数量（默认值：100） - -`--skip-functional-tests`
-跳过功能测试（默认值：false） - -`--threads value, -p value`
-上传下载等操作的并发数（默认值：4） - -##### 示例 +#### 概览 ```shell +juicefs objbench [command options] BUCKET + # 测试 S3 对象存储的基准性能 -$ ACCESS_KEY=myAccessKey SECRET_KEY=mySecretKey juicefs objbench --storage s3 https://mybucket.s3.us-east-2.amazonaws.com -p 6 +ACCESS_KEY=myAccessKey SECRET_KEY=mySecretKey juicefs objbench --storage=s3 https://mybucket.s3.us-east-2.amazonaws.com -p 6 ``` -#### `juicefs warmup` {#warmup} +#### 参数 + +|项 | 说明| +|-|-| +|`--storage=file`|对象存储类型 (例如 `s3`、`gcs`、`oss`、`cos`) (默认：`file`，参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)查看所有支持的对象存储类型)| +|`--access-key=value`|对象存储的 Access Key，也可通过环境变量 `ACCESS_KEY` 设置。查看[如何设置对象存储](../guide/how_to_set_up_object_storage.md#aksk)以了解更多。| +|`--secret-key=value`|对象存储的 Secret Key，也可通过环境变量 `SECRET_KEY` 设置。查看[如何设置对象存储](../guide/how_to_set_up_object_storage.md#aksk)以了解更多。| +|`--block-size=4096`|每个 IO 块的大小（以 KiB 为单位）（默认值：4096）| +|`--big-object-size=1024`|大文件的大小（以 MiB 为单位）（默认值：1024）| +|`--small-object-size=128`|每个小文件的大小（以 KiB 为单位）（默认值：128）| +|`--small-objects=100`|小文件的数量（默认值：100）| +|`--skip-functional-tests`|跳过功能测试（默认值：false）| +|`--threads=4, -p 4`|上传下载等操作的并发数（默认值：4）| + +### `juicefs warmup` {#warmup} 将文件提前下载到缓存，提升后续本地访问的速度。可以指定某个挂载点路径，递归对这个路径下的所有文件进行缓存预热；也可以通过 `--file` 选项指定文本文件，在文本文件中指定需要预热的文件名。 如果需要预热的文件分布在许多不同的目录，推荐将这些文件名保存到文本文件中并用 `--file` 参数传给预热命令，这样做能利用 `warmup` 的并发功能，速度会显著优于多次调用 `juicefs warmup`，在每次调用里传入单个文件。 -##### 使用 +#### 概览 -``` +```shell juicefs warmup [command options] [PATH ...] -``` -##### 选项 +# 预热目录中的所有文件 +juicefs warmup /mnt/jfs/datadir -`--file value, -f value`
-指定一个包含一组路径的文件（每一行为一个文件路径） +# 只预热指定文件 +echo '/jfs/f1 +/jfs/f2 +/jfs/f3' > /tmp/filelist.txt +juicefs warmup -f /tmp/filelist.txt +``` -`--threads value, -p value`
-并发的工作线程数，默认 50。如果带宽不足导致下载失败，需要减少并发度，控制下载速度 +#### 参数 -`--background, -b`
-后台运行（默认：false） +|项 | 说明| +|-|-| +|`--file=value, -f value`|指定一个包含一组路径的文件（每一行为一个文件路径）。| +|`--threads=50, -p 50`|并发的工作线程数，默认 50。如果带宽不足导致下载失败，需要减少并发度，控制下载速度。| +|`--background, -b`|后台运行（默认：false）| -##### 示例 +### `juicefs rmr` {#rmr} -```shell -# 预热目录中的所有文件 -$ juicefs warmup /mnt/jfs/datadir - -# 只预热目录中 3 个文件 -$ cat /tmp/filelist -/mnt/jfs/datadir/f1 -/mnt/jfs/datadir/f2 -/mnt/jfs/datadir/f3 -$ juicefs warmup -f /tmp/filelist -``` - -#### `juicefs rmr`{#rmr} - -快速删除目录里的所有文件和子目录，效果等同于 `rm -rf`，但该命令直接操纵元数据，不经过 POSIX，所以速度更快。 +快速删除目录里的所有文件和子目录，效果等同于 `rm -rf`，但该命令直接操纵元数据，不经过内核，所以速度更快。 如果文件系统启用了回收站功能，被删除的文件会进入回收站。详见[「回收站」](../security/trash.md)。 -##### 使用 +#### 概览 -``` +```shell juicefs rmr PATH ... -``` -##### 示例 - -```shell juicefs rmr /mnt/jfs/foo ``` -#### `juicefs sync` {#sync} +### `juicefs sync` {#sync} -在两个存储系统之间同步数据。 +在两个存储之间同步数据，阅读[「数据同步」](../guide/sync.md)以了解更多。 -##### 使用 +#### 概览 -``` +```shell juicefs sync [command options] SRC DST -``` - -- **SRC**：源路径 -- **DST**：目标路径 - -源路径和目标路径的格式均为 `[NAME://][ACCESS_KEY:SECRET_KEY[:TOKEN]@]BUCKET[.ENDPOINT][/PREFIX]`，其中： - -- `NAME`：JuiceFS 支持的数据存储类型（如 `s3`、`oss`），请参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)。 -- `ACCESS_KEY` 和 `SECRET_KEY`：访问数据存储所需的密钥信息，请参考[文档](../guide/how_to_set_up_object_storage.md#access-key-和-secret-key)。 -- `TOKEN` 用来访问对象存储的 token，部分对象存储支持使用临时的 token 以获得有限时间的权限 -- `BUCKET[.ENDPOINT]`：数据存储服务的访问地址，不同存储类型格式可能不同，具体请参考[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)。 -- `[/PREFIX]`：可选，源路径和目标路径的前缀，可用于限定只同步某些路径中的数据。 - -有关 `sync` 子命令的详细介绍，请参考[文档](../guide/sync.md)。 - -##### 选项 - -###### 选择条件 - -`--start KEY, -s KEY`
-同步的第一个对象名 - -`--end KEY, -e KEY`
-同步的最后一个对象名 - -`--exclude PATTERN`
-排除匹配 PATTERN 的 Key - -`--include PATTERN`
-不排除匹配 PATTERN 的 Key，需要与 `--exclude` 选项配合使用。 - -`--limit value`
-限制将要处理的对象的数量 (默认：-1) - -`--update, -u`
-当源文件更新时修改已存在的文件 (默认：false) - -`--force-update, -f`
-强制修改已存在的文件 (默认：false) - -`--existing, --ignore-non-existing`
-跳过在目标上创建新文件 (默认值：false) - -`--ignore-existing`
-跳过更新目标上已经存在的文件 (默认值：false) - -###### 行为 -`--dirs`
-同步目录 (默认：false) - -`--perms`
-保留权限设置 (默认：false) - -`--links, -l`
-将符号链接复制为符号链接 (默认：false) - -`--delete-src, --deleteSrc`
-同步后删除源存储的对象 (默认：false) - -`--delete-dst, --deleteDst`
-删除目标存储下的不相关对象 (默认：false) - -`--check-all`
-验证源路径和目标路径中所有文件的数据完整性 (默认：false) - -`--check-new`
-验证新拷贝文件的数据完整性 (默认：false) - -`--dry`
-不拷贝文件 (默认：false) +# 从 OSS 同步到 S3 +juicefs sync oss://mybucket.oss-cn-shanghai.aliyuncs.com s3://mybucket.s3.us-east-2.amazonaws.com -###### 对象存储 +# 从 S3 直接同步到 JuiceFS +juicefs sync s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://META-URL/ -`--threads value, -p value`
-并发线程数 (默认：10) +# 源端: a1/b1,a2/b2,aaa/b1 目标端: empty 同步结果: aaa/b1 +juicefs sync --exclude='a?/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://META-URL/ -`--list-threads value`
-列出对象的线程数 (默认：1) +# 源端: a1/b1,a2/b2,aaa/b1 目标端: empty 同步结果: a1/b1,aaa/b1 +juicefs sync --include='a1/b1' --exclude='a[1-9]/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://META-URL/ -`--list-depth value`
-顶级目录的前 N 层用于并行 list (默认：1) +# 源端: a1/b1,a2/b2,aaa/b1,b1,b2 目标端: empty 同步结果: a1/b1,b2 +juicefs sync --include='a1/b1' --exclude='a*' --include='b2' --exclude='b?' s3://mybucket.s3.us-east-2.amazonaws.com/ jfs://META-URL/ +``` -`--no-https`
-不要使用 HTTPS (默认：false) +源路径（`SRC`）和目标路径（`DST`）的格式均为： -`--storage-class value`
-目标端的新建文件的存储类型 +``` +[NAME://][ACCESS_KEY:SECRET_KEY[:SESSIONTOKEN]@]BUCKET[.ENDPOINT][/PREFIX] +``` -`--bwlimit value`
-限制最大带宽；单位为 Mbps (0 表示不限制) (默认：0) +其中： -###### 集群模式 +- `NAME`：JuiceFS 支持的数据存储类型，比如 `s3`、`oss`，完整列表见[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)。 +- `ACCESS_KEY` 和 `SECRET_KEY`：访问数据存储所需的密钥信息，参考[文档](../guide/how_to_set_up_object_storage.md#aksk)。 +- `TOKEN` 用来访问对象存储的 token，部分对象存储支持使用临时的 token 以获得有限时间的权限 +- `BUCKET[.ENDPOINT]`：数据存储服务的访问地址，不同存储类型格式可能不同，详见[文档](../guide/how_to_set_up_object_storage.md#supported-object-storage)。 +- `[/PREFIX]`：可选，源路径和目标路径的前缀，可用于限定只同步某些路径中的数据。 -`--manager value`
-管理者地址 +#### 选择条件相关参数 {#sync-selection-related-options} -`--worker value`
-工作节点列表 (使用逗号分隔) +|项 | 说明| +|-|-| +|`--start=KEY, -s KEY, --end=KEY, -e KEY`|提供 KEY 范围，来指定对象存储的 List 范围。| +|`--exclude=PATTERN`|排除匹配 `PATTERN` 的 Key。| +|`--include=PATTERN`|不排除匹配 `PATTERN` 的 Key，需要与 `--exclude` 选项配合使用。| +|`--limit=-1`|限制将要处理的对象的数量，默认为 -1 表示不限制| +|`--update, -u`|当源文件更新时（`mtime` 更新），覆盖已存在的文件，默认为 false。| +|`--force-update, -f`|强制覆盖已存在的文件，默认为 false。| +|`--existing, --ignore-non-existing`|不创建任何新文件，默认为 false。| +|`--ignore-existing`|不更新任何已经存在的文件，默认为 false。| -##### 示例 +#### 行为相关参数 {#sync-action-related-options} -```shell -# 从 OSS 同步到 S3 -$ juicefs sync oss://mybucket.oss-cn-shanghai.aliyuncs.com s3://mybucket.s3.us-east-2.amazonaws.com +|项 | 说明| +|-|-| +|`--dirs`|同步目录（包括空目录）。| +|`--perms`|保留权限设置，默认为 false。| +|`--links, -l`|将符号链接复制为符号链接，默认为 false，此时会查找并同步符号链接所指向的文件。| +|`--delete-src, --deleteSrc`|如果目标存储已经存在，删除源存储的对象。与 rsync 不同，为保数据安全，首次执行时不会删除源存储文件，只有拷贝成功后再次运行时，扫描确认目标存储已经存在相关文件，才会删除源存储文件。| +|`--delete-dst, --deleteDst`|删除目标存储下的不相关对象。| +|`--check-all`|校验源路径和目标路径中所有文件的数据完整性，默认为 false。校验方式是基于字节流对比，因此也将带来相应的开销。| +|`--check-new`|校验新拷贝文件的数据完整性，默认为 false。校验方式是基于字节流对比，因此也将带来相应的开销。| +|`--dry`|仅打印执行计划，不实际拷贝文件。| -# 从 S3 同步到 JuiceFS -$ juicefs mount -d redis://localhost /mnt/jfs -$ juicefs sync s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/ +#### 对象存储相关参数 {#sync-storage-related-options} -# 源端: a1/b1,a2/b2,aaa/b1 目标端: empty 同步结果: aaa/b1 -$ juicefs sync --exclude='a?/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/ +|项 | 说明| +|-|-| +|`--threads=10, -p 10`|并发线程数，默认为 10。| +|`--list-threads=1`|并发 `list` 线程数，默认为 1。阅读[并发 `list`](../guide/sync.md#concurrent-list)以了解如何使用。| +|`--list-depth=1`|并发 `list` 目录深度，默认为 1。阅读[并发 `list`](../guide/sync.md#concurrent-list)以了解如何使用。| +|`--no-https`|不要使用 HTTPS，默认为 false。| +|`--storage-class value`|目标端的新建文件的存储类型| +|`--bwlimit=0`|限制最大带宽，单位 Mbps，默认为 0 表示不限制。| -# 源端: a1/b1,a2/b2,aaa/b1 目标端: empty 同步结果: a1/b1,aaa/b1 -$ juicefs sync --include='a1/b1' --exclude='a[1-9]/b*' s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/ +#### 分布式相关参数 {#sync-cluster-related-options} -# 源端: a1/b1,a2/b2,aaa/b1,b1,b2 目标端: empty 同步结果: a1/b1,b2 -$ juicefs sync --include='a1/b1' --exclude='a*' --include='b2' --exclude='b?' s3://mybucket.s3.us-east-2.amazonaws.com/ /mnt/jfs/ -``` +|`--manager=ADDR`|分布式同步模式中，Manager 的节点地址，此为内部参数，在 Worker 节点上运行的同步进程中会包含该设置。| +|`--worker=ADDR,ADDR`|分布式同步模式中，工作节点列表，使用逗号分隔。| -#### `juicefs clone`{#clone} +### `juicefs clone` {#clone} -该命令可以克隆文件或目录但不复制底层数据，类似于 cp 命令，但是非常快。 +该命令可以克隆文件或目录但不复制底层数据，类似于 `cp` 命令，但是非常快。 -##### 使用 +#### 概览 ```shell juicefs clone [command options] SRC DST -``` - -##### 选项 - -`--preserve, -p`
-保留文件的 UID、GID 和 mode (默认值：false) - -##### 示例 -```shell # 克隆文件 -$ juicefs clone /mnt/jfs/file1 /mnt/jfs/file2 +juicefs clone /mnt/jfs/file1 /mnt/jfs/file2 # 克隆目录 -$ juicefs clone /mnt/jfs/dir1 /mnt/jfs/dir2 +juicefs clone /mnt/jfs/dir1 /mnt/jfs/dir2 -# 克隆时保留文件的 uid、gid 和 mode -$ juicefs clone -p /mnt/jfs/file1 /mnt/jfs/file2 +# 克隆时保留文件的 UID、GID 和 mode +juicefs clone -p /mnt/jfs/file1 /mnt/jfs/file2 ``` + +#### 参数 + +|项 | 说明| +|-|-| +|`--preserve, -p`|保留文件的 UID、GID 和 mode (默认值：false)|