v1.2.1

This is the first patch release for JuiceFS v1.2. It has 14 commits from 6 contributors, thanks to @zhijian-pro @jiefenghuang @SandyXSD @menwenjun @zwwhdls @tangyoupeng !

Changed

• cmd/mount: get FUSE fd from CSI when required (#4991)

Bugfix

• cmd/mount: fix running on windows (#5042)
• cmd/warmup: fix the issue that evict sub command may remove staging blocks incorrectly (#5095)
• cmd/gc: fix the issue that objects may be deleted by mistake in large systems (#5068)
• cmd/gateway: skip initiating system bucket when metadata is read only (#4992)
• cmd/gateway: improve the performance of ListObjects (#4969, #5013)
• cmd/sync: fix the bandwidth limit when doing streaming copy (#5046)
• cmd/sync: fix concurrent error handling in multipart copy operation (#5044)
• cmd/debug: fix the arguments of ps command on macOS (#5022)
• meta: fix the issue that hardlink's quota update in the same directory may be incorrect (#5069)
• meta: fix the issue that quota may be updated twice when unlink/rename opened files (#5043)
• chunk: fix file matching regular expressions for staging blocks (#5083)
• hadoop: fix libjfs singleton (#5003)

v1.1.4

This is the fourth patch release for JuiceFS v1.1. It has 5 commits from 4 contributors, thanks to @zhijian-pro @SandyXSD @jiefenghuang @tangyoupeng !

Bugfix

• cmd/gc: fix the issue that objects may be deleted by mistake in large systems (#5068)
• meta: fix the issue that hardlink's quota update in the same directory may be incorrect (#5069)
• meta: fix the issue that quota may be updated twice when unlink/rename opened files (#5043)
• chunk: fix file matching regular expressions for staging blocks (#5083)
• hadoop: fix libjfs singleton (#5003)

v1.1.3

This is the third patch release for JuiceFS v1.1. It has 21 commits from 6 contributors, thanks to @SandyXSD @tangyoupeng @zhijian-pro @davies @polyrabbit @prOOrc !

Changed

• deps: upgrade golang.org/x/net to 0.23 (#4911)

Bugfix

• cmd/gateway: fix the issue that memory is not released immediately after object downloaded (#4507)
• cmd/gateway: fix the size of JfsObjects.CompleteMultipartUpload (#4893)
• cmd/sync: fix the issue that manager's IP address is never set in cluster mode (#4939)
• cmd/clone: set the attr should be done after checking permissions (#4745)
• cmd/clone: fix source mode check (#4856)
• meta: fix the issue that df may see negative values (#4591)
• meta: fix the protection check for compaction (#4901)
• meta/redis: fix some transactions that are not wrapped by multi/exec (#4801)
• meta/sql: fix the issue that the nanosecond parts of mtime/ctime may be incorrect (#4618)
• meta/postgreSQL: fix the address parse (#4731)
• vfs: fix the issue that file length may not be updated immediately after fallocate & copyfilerange (#4427)
• vfs: fix the issue that file entry may still be returned if it's deleted during readdir (#4453)
• vfs: fix the issue that the source file may not be up-to-date in copy_file_range (#4436)
• vfs: fix panic when reading and writing s.writer concurrently (#4710)
• vfs: fix the race of getting file length (#4876)
• vfs: fix the issue that plock may be unlocked by mistake (#4910)
• hadoop: make libjfs singleton to avoid possible jnr weakhashmap infinity loop (#4407)
• hadoop: use auto-increment fs handle to avoid possible conflict (#4411)
• hadoop: fix guid lookupGroup (#4532)
• hadoop: rmr check perm only when the directory is not empty (#4539)

v1.2.0

This is the third major release of JuiceFS, and is an LTS version as well that will be maintained for 24 months.
Starting from v1.1 it has 464 commits from 44 contributors, thanks to @zhijian-pro @davies @zhoucheng361 @SandyXSD @jiefenghuang @polyrabbit @CaitinChen @yuhr123 @Hexilee @tangyoupeng @YunhuiChen @eryugey @timfeirg @Suave @solracsf @kyungwan-nam @xiaogaozi @1ambd4 @stefanb @XDTD @occupyhabit @menwenjun @li1213987842 @testwill @fengleng @deferdeter @camcui @uvletter @T-TRz879 @yuchen-sun @TechQuery @reAsOn2010 @mtnk2005 @kanemathers @JoJossd @sjp00556 @CodingPoeta @abserari @AChangFeng @feeyman @l0wl3vel @chnliyong @JoyLiuc @prOOrc!

Highlights

• Supports graceful restart for mount process (#4392, #4450, #4487, #4499, #4509, #4511, #4524, #4547, #4555, #4792, #4823, etc.)
• Supports POSIX ACL (#4421, #4437, #4443, #4490, #4503, #4518, #4522, #4523, #4516, #4551, #4640)
• Supports Identity and Access Management (IAM) and events notify for gateway (#4030, #4593, #4622, #4845, #4891)

New

• Added a new command compact to trigger compaction on a specified path (#4337)
• Supports using NFS as object storage (#3823, #4101, #4105, #4111, #4147, #4163, #4718)
• Supports using Dragonfly as object storage (#4057)
• Supports using Bunny as object storage (#4719)
• cmd: add a new option --log-id to append user-specified ID in log (#4040)
• cmd/mount: supports restarting mount process gracefully (#4392, #4450, #4487, #4499, #4509, #4511, #4524, #4547, #4555, #4792, #4823, etc.)
• cmd/format: add a new option --enable-acl to enable POSIX ACL (#4421, #4437, #4443, #4490, #4503, #4518, #4522, #4523, #4516, #4551, #4640)
• cmd/mount: add a new option --skip-dir-mtime to eliminate burst mtime updates of a directory (#4189)
• cmd/mount: add a new option --upload-hours to limit the period for background uploads (#4250)
• cmd/mount: add a new option --cache-expire to set expired time for cached blocks (#4255, #4365, #4434)
• cmd/mount: add a new option --custom-labels to set labels for metrics (#4312, #4612)
• cmd/mount: add a new environment variable JFS_MOUNT_TIMEOUT to adjust timeout for checking mount point ready (#4219)
• cmd/mount: add a new option max-stage-write to enable concurrent write to both stage files and object storage in writeback mode (#4743)
• cmd/gateway: supports IAM and events notify (#4030, #4593, #4622, #4845, #4891)
• cmd/gateway&webdav: add a new option --background to support daemon running (#3993, #4631, #4708, #4915)
• cmd/gateway: add a new option --domain to support virtual-host-style requests (#4462)
• cmd/warmup: add a new option --evict to manually evict cached blocks (#4370)
• cmd/warmup: and a new option --check to check the ratio of already warmed blocks (#4370)
• cmd/dump: add a new option --fast=false to reduce memory usage with a slower dump process (#4253)
• cmd/dump: add a new option --skip-trash to skip the trash directory when backing up metadata (#4479, #4563)
• cmd/dump: add a new option --threads to adjust speed for dump (#4581)
• cmd/sync: add a new option --inpace to put file in-place, eliminating the rename operation (#4104)
• cmd/sync: supports two stars ** and three stars *** to match all files recursively (#4422, #4448, #4460, #4468, #4474)
• cmd/sync: add a new option --match-full-path to match the filter against full path (#4492)
• cmd/sync: add new options max-size, min-size, max-age and min-age to filter files based on size and age (#4912, 7491e33, #4913)
• cmd/sync: add p8s metrics to better monitor the sync progress (#4119)
• meta: supports Open File Description (OFD) lock (#4083)
• chunk: disk cache switch to consistent hash and add bad disk detection (#4543, #4749, #4780, #4784, #4871, #4848)
• object: add query string to disable md5 and crc verification when s3 compatible storage does not support them (#4606)
• object: add a new environment variable JFS_S3_VHOST_STYLE=1 to enable vhost style (#4697)
• hadoop: add push-labels option to set custom labels for metrics (#4312)

Changed

• cmd: support human friendly values for options (#4246, #4558, #4567, #4592, #4641, #4671)
• cmd/mount: only use writeback_cache to enable fuse writeback (#4632)
• cmd/gateway: judge loop symbolic links rather than hard-code deep limit (#4045)
• cmd/gateway: create MinioMetaBucket when it doesn't exist (#4383)
• cmd/gateway: create .minio.sys directory even if multi-buckets are not enabled (#4367)
• cmd/dump: use a temporary file during dumping (#4767)
• cmd/sync: limit the maximum memory size for multiple uploads (#4135)
• cmd/sync: speed up sync for huge file (#4456)
• cmd/sync: use larger io-size when writing to backends (#4546)
• cmd/sync: skip listing the destination in force mode (#4602)
• cmd/sync: use HEAD instead of LIST for a single object (#4623)
• cmd/sync: limit the total size of each batch task from the manager (#4886)
• meta: add mount time in session info (#3987)
• meta: export meta backup metrics (#4480)
• meta: force compaction when there are too many slices in one chunk (#4573)
• meta: remove tail zero slice when compacting a chunk (#4789)
• meta/redis: reduce the memory usage when dumping large directories (#4481)
• meta/tkv & meta/sql: speedup the dump process using multiple threads (#4501)
• meta/badger: upgrade to v4 (#4564)
• chunk: prefetch the whole block only on successful partial read (#4658)
• chunk: delete leaked objects if it's already deleted by other goroutines (#4748)
• vfs: fuse ops metric supports showing method label (#4554, #4560)
• vfs: add errno label to fuse ops metric (#4656)
• object/tos: adjust min part size from 5 MiB to 4 MiB (#4688)
• object/ceph: allow delete when ceph cluster is full (#4131)
• object/ceph: speedup list for ceph (#4218)
• object/ceph: set default path for admin socket and log file (#4608)
• hadoop: support human friendly values for options (#4655, #4813)
• hadoop: support using ; to separate values for juicefs.users and juicefs.group (#4724)
• deps: upgrade golang.org/x/net to 0.23 (#4911)

Bugfix

• cmd/gateway: fix ListMultipartUploads sorting rule and support delimiter (#4297)
• cmd/gateway: fix HEAD directory returns 200 but expect 404 (#4486)
• cmd/gateway: fix the issue that memory is not released immediately after object downloaded (#4507)
• cmd/gateway: fix empty bucket display exception (#4875)
• cmd/gateway: fix the size of JfsObjects.CompleteMultipartUpload (#4893)
• cmd/gateway: fix listing objects with symbolic links (#4919)
• cmd/sync: fix the issue that empty files are always synchronized (#4223)
• cmd/sync: execute chmod after chown to avoid setuid/setgid failure (#4418)
• cmd/sync: fix the issue that setuid/setgid/sticky bits are not set when using jfs protocol (#4419)
• cmd/sync: return the error of listing objects (#4879)
• cmd/sync: fix the issue that manager's IP address is never set in cluster mode (#4939)
• cmd/clone: set the attr should be done after checking permissions (#4745)
• cmd/clone: fix source mode check (#4856)
• cmd/debug: add a timeout when collecting the information (#4824)
• meta: fix the issue that df may see negative values (#4591)
• meta: fix the error message in quota that always shows inode 0 (#4741)
• meta: fix the protection check for compaction (#4901)
• meta: fix the error code of baseMeta.Open (#4920)
• meta/redis: fix some transactions that are not wrapped by multi/exec (#4801)
• meta/sql: use upsert in Write() to avoid possible deadlock (#4529)
• meta/postgreSQL: fix the address parse (#4731)
• vfs: fix the issue that file entry may still be returned if it's deleted during readdir (#4453)
• vfs: fix the issue that file length may not be updated immediately after fallocate & copyfilerange (#4427)
• vfs: fix the race of getting file length (#4876)
• vfs: fix the issue that plock may be unlocked by mistake (#4910)
• hadoop: make libjfs singleton to avoid possible jnr weakhashmap infinity loop (#4407)
• hadoop: fix guid lookupGroup (#4532)
• hadoop: use auto-increment fs handle to avoid possible conflict (#4411)
• hadoop: fix the issue that dfsio does not work because mapreduce.output.fileoutputformat.compress is true (#4646)

v1.2.0-rc1

This is the first release candidate for JuiceFS v1.2. It has 22 commits from 7 contributors, thanks to @zhoucheng361 @SandyXSD @davies @zhijian-pro @YunhuiChen @prOOrc @yuhr123 !

New

• cmd/sync: add new options max-size, min-size, max-age and min-age to filter files based on size and age (#4912, 7491e33, #4913)

Changed

• sync: limit the total size of each batch task from the manager (#4886)
• deps: upgrade golang.org/x/net to 0.23 (#4911)

Bugfix

• cmd/gateway: fix gateway add and list service account (#4891)
• cmd/gateway: fix the size of JfsObjects.CompleteMultipartUpload (#4893)
• cmd/sync: return the error of listing objects (#4879)
• meta: fix the protection check for compaction (#4901)
• vfs: fix the issue that plock may be unlocked by mistake (#4910)

v1.2.0-beta2

This is the second beta release for JuiceFS v1.2. Since beta1 it has 84 commits from 14 contributors, thanks to @zhoucheng361 @zhijian-pro @CaitinChen @SandyXSD @jiefenghuang @yuhr123 @polyrabbit @davies @tangyoupeng @kyungwan-nam @testwill @chnliyong @JoyLiuc @xiaogaozi !

New

• cmd/mount: add a new option max-stage-write to enable concurrent write to both stage files and object storage in writeback mode (#4743)

Changed

• cmd/mount: support graceful restart on macOS (#4792, #4823)
• cmd/dump: use a temporary file during dumping (#4767)
• meta: remove tail zero slice when compacting a chunk (#4789)
• chunk: improve the management of failed cache disks (#4749, #4780, #4784, #4871, #4848)
• chunk: delete leaked objects if it's already deleted by other goroutines (#4748)
• hadoop: support using ; to separate values for juicefs.users and juicefs.group (#4724)

Bugfix

• cmd/mount: fix FUSE option for ACL (#4787)
• cmd/mount: fix duplicated FUSE service (#4773)
• cmd/gateway: add rw locks to lock multiple requests within a single machine (#4845)
• cmd/gateway: fix empty bucket display exception (#4875)
• cmd/clone: set the attr should be done after checking permissions (#4745)
• cmd/clone: fix source mode check (#4856)
• cmd/debug: add a timeout when collecting the information (#4824)
• cmd/debug: fix the collection in dual-mount-processes mode (#4816)
• meta: fix the error message in quota that always shows inode 0 (#4741)
• meta/redis: fix some transactions that are not wrapped by multi/exec (#4801)
• meta/postgreSQL: fix the address parse (#4731)
• vfs: update data of internal files only when it's empty (#4770)
• vfs: dump the unread portion of the access log (#4855)
• vfs: fix the race of getting file length (#4876)
• hadoop: fix the value of upload/download limit (#4813)

v1.2.0-beta1

This is the first beta release for JuiceFS v1.2. Since v1.1 it has 354 commits from 41 contributors, thanks to @zhijian-pro @davies @zhoucheng361 @SandyXSD @jiefenghuang @polyrabbit @CaitinChen @yuhr123 @Hexilee @tangyoupeng @YunhuiChen @eryugey @timfeirg @Suave @solracsf @kyungwan-nam @xiaogaozi @1ambd4 @stefanb @XDTD @occupyhabit @menwenjun @li1213987842 @testwill @fengleng @deferdeter @camcui @uvletter @T-TRz879 @yuchen-sun @TechQuery @reAsOn2010 @mtnk2005 @kanemathers @JoJossd @sjp00556 @CodingPoeta @abserari @AChangFeng @feeyman @l0wl3vel !

Highlights

• Supports graceful restart for mount process (#4392, #4450, #4487, #4499, #4509, #4511, #4524, #4547, #4555)
• Supports POSIX ACL (#4421, #4437, #4443, #4490, #4503, #4518, #4522, #4523, #4516, #4551, #4640)
• Supports Identity and Access Management (IAM) and events notify for gateway (#4030, #4593)

New

• Added a new command compact to trigger compaction on a specified path (#4337)
• Supports using NFS as object storage (#3823, #4101, #4105, #4111, #4147, #4163, #4718)
• Supports using Dragonfly as object storage (#4057)
• Supports using Bunny as object storage (#4719)
• cmd: add a new option --log-id to append user-specified ID in log (#4040)
• cmd/format: add a new option --enable-acl to enable POSIX ACL (#4421, #4437, #4443, #4490, #4503, #4518, #4522, #4523, #4516, #4551, #4640)
• cmd/mount: add a new option --skip-dir-mtime to eliminate burst mtime updates of a directory (#4189)
• cmd/mount: add a new option --upload-hours to limit the period for background uploads (#4250)
• cmd/mount: add a new option --cache-expire to set expired time for cached blocks (#4255, #4365, #4434)
• cmd/mount: add a new option --custom-labels to set labels for metrics (#4312, #4612)
• cmd/mount: add a new environment variable JFS_MOUNT_TIMEOUT to adjust timeout for checking mount point ready (#4219)
• cmd/mount: supports restarting mount process gracefully (#4392, #4450, #4487, #4499, #4509, #4511, #4524, #4547, #4555)
• cmd/gateway: supports IAM and events notify (#4030, #4593, #4622)
• cmd/gateway&webdav: add a new option --background to support daemon running (#3993, #4631, #4708)
• cmd/gateway: add a new option --domain to support virtual-host-style requests (#4462)
• cmd/warmup: add a new option --evict to manually evict cached blocks (#4370)
• cmd/warmup: and a new option --check to check the ratio of already warmed blocks (#4370)
• cmd/dump: add a new option --fast=false to reduce memory usage with a slower dump process (#4253)
• cmd/dump: add a new option --skip-trash to skip the trash directory when backing up metadata (#4479, #4563)
• cmd/dump: add a new option --threads to adjust speed for dump (#4581)
• cmd/sync: add a new option --inpace to put file in-place, eliminating the rename operation (#4104)
• cmd/sync: supports two stars ** and three stars *** to match all files recursively (#4422, #4448, #4460, #4468, #4474)
• cmd/sync: add a new option --match-full-path to match the filter against full path (#4492)
• cmd/sync: add p8s metrics to better monitor the sync progress (#4119)
• meta: supports Open File Description (OFD) lock (#4083)
• chunk: disk cache switch to consistent hash and add bad disk detection (#4543)
• object: add query string to disable md5 and crc verification when s3 compatible storage does not support them (#4606)
• object: add a new environment variable JFS_S3_VHOST_STYLE=1 to enable vhost style (#4697)
• hadoop: add push-labels option to set custom labels for metrics (#4312)

Changed

• cmd: support human friendly values for options (#4246, #4558, #4567, #4592, #4641, #4671)
• cmd/mount: only use writeback_cache to enable fuse writeback (#4632)
• cmd/gateway: judge loop symbolic links rather than hard-code deep limit (#4045)
• cmd/gateway: create MinioMetaBucket when it doesn't exist (#4383)
• cmd/gateway: create .minio.sys directory even if multi-buckets are not enabled (#4367)
• cmd/sync: limit the maximum memory size for multiple uploads (#4135)
• cmd/sync: speed up sync for huge file (#4456)
• cmd/sync: use larger io-size when writing to backends (#4546)
• cmd/sync: skip listing the destination in force mode (#4602)
• cmd/sync: use HEAD instead of LIST for a single object (#4623)
• meta: add mount time in session info (#3987)
• meta: export meta backup metrics (#4480)
• meta: force compaction when there are too many slices in one chunk (#4573)
• meta/tkv & meta/sql: speedup the dump process using multiple threads (#4501)
• meta/redis: reduce the memory usage when dumping large directories (#4481)
• meta/badger: upgrade to v4 (#4564)
• chunk: prefetch the whole block only on successful partial read (#4658)
• vfs: fuse ops metric supports showing method label (#4554, #4560)
• vfs: add errno label to fuse ops metric (#4656)
• object/tos: adjust min part size from 5 MiB to 4 MiB (#4688)
• object/ceph: allow delete when ceph cluster is full (#4131)
• object/ceph: speedup list for ceph (#4218)
• object/ceph: set default path for admin socket and log file (#4608)
• hadoop: support human friendly values for options (#4655)

Bugfix

• cmd/gateway: fix ListMultipartUploads sorting rule and support delimiter (#4297)
• cmd/gateway: fix HEAD directory returns 200 but expect 404 (#4486)
• cmd/gateway: fix the issue that memory is not released immediately after object downloaded (#4507)
• cmd/sync: fix the issue that empty files are always synchronized (#4223)
• cmd/sync: execute chmod after chown to avoid setuid/setgid failure (#4418)
• cmd/sync: fix the issue that setuid/setgid/sticky bits are not set when using jfs protocol (#4419)
• meta: fix the issue that df may see negative values (#4591)
• meta/sql: use upsert in Write() to avoid possible deadlock (#4529)
• vfs: fix the issue that file entry may still be returned if it's deleted during readdir (#4453)
• vfs: fix the issue that file length may not be updated immediately after fallocate & copyfilerange (#4427)
• hadoop: make libjfs singleton to avoid possible jnr weakhashmap infinity loop (#4407)
• hadoop: fix guid lookupGroup (#4532)
• hadoop: use auto-increment fs handle to avoid possible conflict (#4411)
• hadoop: fix the issue that dfsio does not work because mapreduce.output.fileoutputformat.compress is true (#4646)

v1.1.2

This is the second patch release for JuiceFS v1.1. It has 16 commits from 6 contributors, thanks to @zhijian-pro @davies @SandyXSD @tangyoupeng @mtnk2005 @AChangFeng !

Changed

• Build(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#4287)

Bugfix

• cmd/sync: fix the issue that name of the temporary file may be too long (#4215)
• cmd/sync: fix the issue that password of source is not removed (#4390)
• cmd/sync: fix symlink when using jfs:// schema (#4286)
• cmd/bench: fix the issue that the final result of progress bar is not correct (#4388)
• meta: trigger compaction more aggresively when there are too many slices (#4309)
• meta: fix the issue that truncated slices may never be compacted (#4320)
• meta/redis: use TxPipelined() instead of Pipelined() (#4264)
• object: fix the issue that sometimes checksum is not verified (#4213)
• object/obs: don't decode the object key in list result (#4331)
• object/cos: fix multipart upload with long key (#4258)
• object/ceph: fix the issue that list with many threads may panic (#4216)
• hadoop: fix guid update (#4299)
• windows: fix the issue that truncate may return syscall.EBADF (#4238)

v1.0.7

This is the seventh patch release for JuiceFS v1.0. It has 12 commits from 5 contributors, thanks to @davies @SandyXSD @zhijian-pro @tangyoupeng @mtnk2005 !

Changed

• Build(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#4287)

Bugfix

• cmd/sync: fix the issue that name of the temporary file may be too long (#4215)
• cmd/sync: fix the issue that password of source is not removed (#4390)
• meta: trigger compaction more aggresively when there are too many slices (#4309)
• meta: fix the issue that truncated slices may never be compacted (#4320)
• meta/redis: use TxPipelined() instead of Pipelined() (#4264)
• object/obs: don't decode the object key in list result (#4331)
• object/cos: fix multipart upload with long key (#4258)
• object/ceph: fix the issue that list with many threads may panic (#4216)
• hadoop: fix guid update (#4299)

v1.1.1

This is the first patch release for JuiceFS v1.1. It has 17 commits from 6 contributors, thanks to @zhijian-pro @davies @Hexilee @SandyXSD @tangyoupeng @fengleng !

Changed

• cmd/quota: set limitation before calculating the used stats (#4158)
• meta: increase inodeBatch & sliceIdBatch (#4188)
• meta: skip cleaning session when sid is 0 (#4133)
• deps: upgrade golang/x/net to 0.17 (#4136)

Bugfix

• cmd/gateway: create the parent directory of the target on demand when copying an object (#4058)
• cmd/restore: fix the issue that dirStats and quota are not properly set (#4095)
• cmd/sync: fix copying times, owner and mode of symlinks (#4151)
• cmd/sync: fix the issue that checksum are wrongly calculated for symlinks (#4082)
• cmd & object: fix incorrect paths in Windows (#4138)
• meta: allow flock converting current exclusive lock to a shared lock (#4179)
• meta: fix possible password leak when it contains the @ symbol (#4071)
• meta/mysql: don't escape the mysql password (#4069)
• meta/tkv: fix offset overflow in CopyFileRange (#4130)
• meta/tkv: fix the issue that info command may wrongly fail on read-only mountpoint (#4085)
• object: skip listing directories with no permission (#4125)
• object/s3: fix nil when StorageClass is missing from List (#4183)
• hadoop: fix guid refresh by avoid using closed fs (#4180)