Skip to content

Releases: apache/opendal

v0.46.0

06 May 01:26
@tisonkun tisonkun
v0.46.0
This tag was signed with the committer’s verified signature.
tisonkun tison
GPG key ID: CD8564CB6A21E526
Learn about vigilant mode.
84586e5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.46.0

Upgrade to v0.46

Public API

MSRV Changed to 1.75

Since 0.46, OpenDAL requires Rust 1.75.0 or later to use features like RPITIT and AFIT.

Services Feature Flag

Starting with version 0.46, OpenDAL only includes the memory service by default to prevent compiling unnecessary service code. To use other services, please activate their respective feature flags.

Additionally, we have removed all reqwest-related feature flags:

  • Users must now directly use reqwest's feature flags for options like rustls, native-tls, etc.
  • The rustls feature is no longer enabled by default; it must be activated manually.
  • OpenDAL no longer offers the trust-dns option; users should configure the client builder directly.

Range Based Read

Since v0.46, OpenDAL transformed it's Read IO trait to range based instead of stateful poll based IO. This change will make the IO more efficient, easier for concurrency and ready for completion based IO.

opendal::Reader now have APIs like:

let r = op.reader("test.txt").await?;
let buf = r.read(1024..2048).await?;

Buffer Based IO

Since version 0.46, OpenDAL features a native Buffer struct that supports both contiguous and non-contiguous buffers. This update enhances IO efficiency by minimizing unnecessary byte copying and enabling vectored IO.

OpenDAL's Reader will return Buffer and Writer will accept Buffer as input. Users who have implemented their own IO traits should update their code to use the new Buffer struct.

let r = op.reader("test.txt").await?;
// read returns `Buffer`
let buf: Buffer = r.read(1024..2048).await?;

let w = op.writer("test.txt").await?;

// Buffer can be created from continues bytes.
w.write("hello, world").await?;
// Buffer can also be created from non-continues bytes.
w.write(vec![Bytes::from("hello,"), Bytes::from("world!")]).await?;

// Make sure file has been written completely.
w.close().await?;

To enhance usability, we've integrated bridges into bytes::Buf and bytes::BufMut, allowing users to directly interact with the bytes API.

let r = op.reader("test.txt").await?;
let mut bs = vec![];
// read_into accepts bytes::BufMut
let buf: Buffer = r.read_into(&mut bs, 1024..2048).await?;

let w = op.writer("test.txt").await?;

// write_from accepts bytes::Buf
w.write_from("hello, world".as_bytes()).await?;

// Make sure file has been written completely.
w.close().await?;

Bridge API

OpenDAL's Reader and Writer previously implemented APIs such as AsyncRead and AsyncWrite directly. This design was not user-friendly, as it could lead to unexpected costs that users were unaware of in advance.

Since v0.46, OpenDAL provides bridge APIs for Reader and Writer instead.

let r = op.reader("test.txt").await?;

// Convert into futures AsyncRead + AsyncSeek.
let reader = r.into_futures_async_read(1024..2048);
// Convert into futures bytes stream.
let stream = r.into_bytes_stream(1024..2048);

let w = op.writer("test.txt").await?;

// Convert into futures AsyncWrite
let writer = w.into_futures_async_write();
// Convert into futures bytes sink;
let sink = w.into_bytes_sink();

Raw API

Async in IO trait

Since version 0.46, OpenDAL has adopted Rust's native async_in_trait for our core IO traits, including oio::Read, oio::Write, and oio::List.

This update eliminates the need for manually written, poll-based state machines and simplifies the codebase. Consequently, OpenDAL now requires Rust version 1.75.0 or later.

Users who have implemented their own IO traits should update their code to use the new async trait syntax.

What's Changed

Added

  • feat(services/github): add github contents support by @hoslo in #4281
  • feat: Allow selecting webpki for reqwest by @arlyon in #4303
  • feat(services/swift): add support for storage_url configuration in swift service by @zjregee in #4302
  • feat(services/swift): add ceph test setup for swift by @zjregee in #4307
  • docs(website): add local content search based on lunr plugin by @m1911star in #4348
  • feat(services/sled): add SledConfig by @yufan022 in #4351
  • feat : Implementing config for part of services by @AnuRage-git in #4344
  • feat(bindings/java): explicit async runtime by @tisonkun in #4376
  • feat(services/surrealdb): support surrealdb service by @yufan022 in #4375
  • feat(bindings/java): avoid double dispose NativeObject by @tisonkun in #4377
  • feat : Implement config for services/foundationdb by @AnuRage-git in #4355
  • feat: add ofs ctrl-c exit unmount hook by @oowl in #4393
  • feat: Implement RFC-4382 Range Based Read by @Xuanwo in #4381
  • feat(ofs): rename2 lseek copy_file_range getattr API support by @oowl in #4395
  • feat(services/github): make access_token optional by @hoslo in #4404
  • feat(core/oio): Add readable buf by @Xuanwo in #4407
  • feat(ofs): add freebsd OS support by @oowl in #4403
  • feat(core/raw/oio): Add Writable Buf by @Xuanwo in #4410
  • feat(bin/ofs): Add behavior test for ofs by @ho-229 in #4373
  • feat(core/raw/buf): Reduce one allocation by Arc::from_iter by @Xuanwo in #4440
  • feat: ?Send async trait for HttpBackend when the target is wasm32 by @waynexia in #4444
  • feat: add HttpClient::with() constructor by @waynexia in #4447
  • feat: Move Buffer as public API by @Xuanwo in #4450
  • feat: Optimize buffer implementation and add stream support by @Xuanwo in #4458
  • feat(core): Implement FromIterator for Buffer by @Xuanwo in #4459
  • feat(services/ftp): Support multiple write by @xxxuuu in #4425
  • feat(raw/oio): block write change to buffer by @hoslo in #4466
  • feat(core): Implement read and read_into for Reader by @Xuanwo in #4467
  • feat(core): Implement into_stream for Reader by @Xuanwo in #4473
  • feat(core): Tune buffer operations based on benchmark results by @Xuanwo in #4468
  • feat(raw/oio): Use Buffer as cache in RangeWrite by @reswqa in #4476
  • feat(raw/oio): Use Buffer as cache in OneshotWrite by @reswqa in #4477
  • feat: add list/copy/rename for dropbox by @zjregee in #4424
  • feat(core): Implement write and write_from for Writer by @zjregee in #4482
  • feat(core): Add auto ranged read and concurrent read support by @Xuanwo in #4486
  • feat(core): Implement fetch for Reader by @Xuanwo in #4488
  • feat(core): Allow concurrent reading on bytes stream by @Xuanwo in #4499
  • feat: provide send-wrapper to contidionally implement Send for operators by @waynexia in #4443
  • feat(bin/ofs): privileged mount by @ho-229 in #4507
  • feat(services/compfs): init compfs by @George-Miao in #4519
  • feat(raw/oio): Add PooledBuf to allow reuse buffer by @Xuanwo in #4522
  • feat(services/fs): Add PooledBuf in fs to allow reusing memory by @Xuanwo in #4525
  • feat(core): add access methods for Buffer by @George-Miao in #4530
  • feat(core): implement IoBuf for Buffer by @George-Miao in #4532
  • feat(services/compfs): compio runtime and compfs structure by @George-Miao in #4534
  • feat(core): change Result to default error by @George-Miao in #4535
  • feat(services/github): support list recursive by @hoslo in #4423
  • feat: Add crc32c checksums to S3 Service by @JWackerbauer in #4533
  • feat: Add into_bytes_sink for Writer by @Xuanwo in #4541

Changed

  • refactor(core/raw): Migrate oio::Read to async in trait by @Xuanwo in #4336
  • refactor(core/raw): Align oio::BlockingRead API with oio::Read by @Xuanwo in #4349
  • refactor(core/oio): Migrate oio::List to async fn in trait by @Xuanwo in #4352
  • refactor(core/raw): Migrate oio::Write from WriteBuf to Bytes by @Xuanwo in https://github...
Read more

Contributors

jbampton, mobiusklein, and 26 other contributors
Assets 2
Loading

v0.45.1

02 Mar 13:38
@Xuanwo Xuanwo
v0.45.1
This tag was signed with the committer’s verified signature.
Xuanwo Xuanwo
GPG key ID: A61687076CEAD424
Learn about vigilant mode.
5079125
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.45.1

Pacakages

Name Version
core 0.45.1
bin/oay 0.41.1
bin/oli 0.41.1
bin/ofs 0.0.2
bindings/python 0.45.1
bindings/nodejs 0.45.1
bindings/c 0.44.3
bindings/zig 0.0.0
bindings/dotnet 0.1.1
bindings/haskell 0.44.3
bindings/java 0.45.1
bindings/lua 0.1.1
bindings/ruby 0.1.1
bindings/swift 0.0.0
bindings/ocaml 0.0.0
bindings/php 0.1.1
bindings/cpp 0.44.3
bindings/go 0.0.0
integrations/object_store 0.43.0
integrations/dav-server 0.0.1

What's Changed

Added

  • feat(services/vercel_blob): support vercel blob by @hoslo in #4103
  • feat(bindings/python): add ruff as linter by @asukaminato0721 in #4135
  • feat(services/hdfs-native): Add capabilities for hdfs-native service by @jihuayu in #4174
  • feat(services/sqlite): Add list capability supported for sqlite by @jihuayu in #4180
  • feat(services/azblob): support multi write for azblob by @wcy-fdu in #4181
  • feat(release): Implement releasing OpenDAL components seperately by @Xuanwo in #4196
  • feat: object store adapter based on v0.9 by @waynexia in #4233
  • feat(bin/ofs): implement fuse for linux by @ho-229 in #4179
  • feat(services/memcached): change to binary protocal by @hoslo in #4252
  • feat(services/memcached): setup auth test for memcached by @hoslo in #4259
  • feat(services/yandex_disk): setup test for yandex disk by @hoslo in #4264
  • feat: add ci support for ceph_rados by @ZhengLin-Li in #4191
  • feat: Implement Config for part of services by @Xuanwo in #4277
  • feat: add jfrog test setup for webdav by @zjregee in #4265

Changed

  • refactor(bindings/python): simplify async writer aexit by @suyanhanx in #4128
  • refactor(service/d1): Add D1Config by @jihuayu in #4129
  • refactor: Rewrite webdav to improve code quality by @Xuanwo in #4280

Fixed

  • fix: Azdls returns 403 while continuation contains = by @Xuanwo in #4105
  • fix(bindings/python): missed to call close for the file internally by @zzl221000 in #4122
  • fix(bindings/python): sync writer exit close raise error by @suyanhanx in #4127
  • fix(services/chainsafe): fix 423 http status by @hoslo in #4148
  • fix(services/webdav): Add possibility to answer without response if file isn't exist by @AJIOB in #4170
  • fix(services/webdav): Recreate root directory if need by @AJIOB in #4173
  • fix(services/webdav): remove base_dir component by @hoslo in #4231
  • fix(core): Poll TimeoutLayer::sleep once to make sure timer registered by @Xuanwo in #4230
  • fix(services/webdav): Fix endpoint suffix not handled by @Xuanwo in #4257
  • fix(services/webdav): Fix possible error with value loosing from config by @AJIOB in #4262

Docs

CI

  • ci: Use old version of seafile mc instead by @Xuanwo in #4107
  • ci: Refactor workflows layout by @Xuanwo in #4139
  • ci: Migrate hdfs default setup by @Xuanwo in #4140
  • ci: Refactor check.sh into check.py to get ready for multi components release by @Xuanwo in #4159
  • ci: Add test case for hdfs over gcs bucket by @ArmandoZ in #4145
  • ci: Add hdfs test case for s3 by @ArmandoZ in #4184
  • ci: Add hdfs test case for azurite by @ArmandoZ in #4185
  • ci: Add support for releasing all rust packages by @Xuanwo in #4200
  • ci: Fix dependabot not update by @Xuanwo in #4202
  • ci: reduce the open pull request limits to 1 by @jbampton in #4225
  • ci: Remove version suffix from package versions by @Xuanwo in #4254
  • ci: Fix fuzz test for minio s3 name change by @Xuanwo in #4266
  • ci: Mark python 3.13 is not supported by @Xuanwo in #4269
  • ci: Disable yandex disk test for too slow by @Xuanwo in #4274
  • ci: Split python CI into release and checks by @Xuanwo in #4275
  • ci(release): Make sure LICENSE and NOTICE files are included by @Xuanwo in #4283
  • ci(release): Refactor and merge the check.py into verify.py by @Xuanwo in #4284

Chore

Read more

Contributors

jayvdb, jbampton, and 15 other contributors
Assets 2
Loading
0 Join discussion

v0.45.0

05 Feb 15:55
@morristai morristai
v0.45.0
This tag was signed with the committer’s verified signature.
morristai Morris Tai
GPG key ID: DDBFEDDFD8121197
Learn about vigilant mode.
7190c07
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.45.0

Upgrade to v0.45

Core

Public API

BlockingLayer is not enabled by default

To further enhance the optionality of tokio, we have introduced a new feature called layers-blocking. The default usage of the blocking layer has been disabled. To utilize the BlockingLayer, please enable the layers-blocking feature.

TimeoutLayer deprecated with_speed

The with_speed API has been deprecated. Please use with_io_timeout instead.

Raw API

No raw API changes.

What's Changed

Added

  • feat(ofs): introduce ofs execute bin by @oowl in #4033
  • feat: add a missing news by @WenyXu in #4056
  • feat(services/koofr): set test for koofr by @suyanhanx in #4050
  • feat(layers/dtrace): Support User Statically-Defined Tracing(aka USDT) on Linux by @Zheaoli in #4053
  • feat(website): add missing news and organize the onboarding guide by @morristai in #4072
  • feat(services/chainsafe): setup test for chainsafe by @hoslo in #4089
  • docs: Add apache iceberg-rust as user by @liurenjie1024 in #4100

Changed

  • refactor!: avoid hard dep to tokio rt by @tisonkun in #4061
  • refactor: Implement IntoFuture for operator futures to remove an alloc by @Xuanwo in #4098

Fixed

  • fix(examples/cpp): display the results to standard output. by @SYaoJun in #4040
  • fix(service/icloud):Missing 'X-APPLE-WEBAUTH-USER cookie' and URL initialized failed by @bokket in #4029
  • fix: Implement timeout layer correctly by using timeout by @Xuanwo in #4059
  • fix(koofr): create_dir when exist by @hoslo in #4062
  • fix(seafile): test_list_dir_with_metakey by @hoslo in #4063
  • fix: list path recursive should not return path itself by @youngsofun in #4067
  • fix: Upgrade suppaftp to address build failure by @Xuanwo in #4091

Docs

  • docs: Remove not needed actions in release guide by @Xuanwo in #4037
  • docs: fix spelling errors in README.md by @SYaoJun in #4039
  • docs: New PMC member Liuqing Yue by @Xuanwo in #4047
  • docs: New Committer Yang Shuai by @Xuanwo in #4054
  • docs(services/sftp): add more explanation for endpoint config by @silver-ymz in #4055
  • docs: Add upgrade docs for core by @Xuanwo in #4095
  • docs: fix reference in rustdocs of FuturePresignWrite by @qrilka in #4097
  • docs: Add docs of options for xxx_with APIs by @Xuanwo in #4099

CI

  • ci(services/s3): Use minio/minio image instead by @Xuanwo in #4070
  • ci: Fix CI after moving out of workspacs by @Xuanwo in #4081

Chore

  • chore: Delete bindings/ruby/cucumber.yml by @tisonkun in #4030
  • chore(website): Bump download link for 0.44.2 release by @Zheaoli in #4034
  • chore(website): Update the release tips by @Zheaoli in #4036
  • chore: add doap file by @tisonkun in #4038
  • chore(website): Add extra artifacts check process in release document by @Zheaoli in #4041
  • chore(bindings/dotnet): update cargo.lock and set up ci by @suyanhanx in #4084
  • chore(bindings/dotnet): build os detect by @suyanhanx in #4085
  • chore(bindings/ocaml): pinning OCaml binding opendal version for release by @Ranxy in #4086
  • chore: Specify opendal version for haskell binding by @Xuanwo in #4093
  • chore: Bump to version 0.45.0 to start release process by @morristai in #4088

New Contributors

Full Changelog: v0.44.2...v0.45.0

Contributors

qrilka, liurenjie1024, and 14 other contributors
Assets 2
Loading

v0.44.2

20 Jan 16:05
@Zheaoli Zheaoli
v0.44.2
This tag was signed with the committer’s verified signature.
Zheaoli Nadeshiko Manju
GPG key ID: 878F445D9C6CE65E
Learn about vigilant mode.
c5c6934
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.44.2

What's Changed

Added

  • feat: add behavior tests for blocking buffer reader by @WenyXu in #3872
  • feat(services): add pcloud support by @hoslo in #3892
  • feat(services/hdfs): Atomic write for hdfs by @shbhmrzd in #3875
  • feat(services/hdfs): add atomic_write_dir to hdfsconfig debug by @shbhmrzd in #3902
  • feat: add MongodbConfig by @zjregee in #3906
  • RFC-3898: Concurrent Writer by @WenyXu in #3898
  • feat(services): add yandex disk support by @hoslo in #3918
  • feat: implement concurrent MultipartUploadWriter by @WenyXu in #3915
  • feat: add concurrent writer behavior tests by @WenyXu in #3920
  • feat: implement concurrent RangeWriter by @WenyXu in #3923
  • feat: add concurrent and buffer parameters into FuzzInput by @WenyXu in #3921
  • feat(fuzz): add azblob as test service by @suyanhanx in #3931
  • feat(services/webhdfs): Implement write with append by @hoslo in #3937
  • feat(core/bench): Add benchmark for concurrent write by @Xuanwo in #3942
  • feat(oio): add block_write support by @hoslo in #3945
  • feat(services/webhdfs): Implement multi write via CONCAT by @hoslo in #3939
  • feat(core): Allow retry in concurrent write operations by @Xuanwo in #3958
  • feat(services/ghac): Add workaround for AWS S3 based GHES by @Xuanwo in #3985
  • feat: Implement path cache and refactor gdrive by @Xuanwo in #3975
  • feat(services): add hdfs native layout by @shbhmrzd in #3933
  • feat(services/s3): Return error if credential is empty after loaded by @Xuanwo in #4000
  • feat(services/gdrive): Use trash instead of permanently deletes by @Xuanwo in #4002
  • feat(services): add koofr support by @hoslo in #3981
  • feat(icloud): Add basic Apple iCloud Drive support by @bokket in #3980

Changed

  • refactor: Merge compose_{read,write} into enum_utils by @Xuanwo in #3871
  • refactor(services/ftp): Impl parse_error instead of From by @bokket in #3891
  • docs: very minor English wording fix in error message by @gabrielgrant in #3900
  • refactor(services/rocksdb): Impl parse_error instead of From by @suyanhanx in #3903
  • refactor: Re-organize the layout of tests by @Xuanwo in #3904
  • refactor(services/etcd): Impl parse_error instead of From by @suyanhanx in #3910
  • refactor(services/sftp): Impl parse_error instead of From by @G-XD in #3914
  • refactor!: Bump MSRV to 1.75 by @Xuanwo in #3851
  • refactor(services/redis): Impl parse_error instead of From by @suyanhanx in #3938
  • refactor!: Revert the bump of MSRV to 1.75 by @Xuanwo in #3952
  • refactor(services/onedrive): Add OnedriveConfig to implement ConfigDeserializer by @Borber in #3954
  • refactor(service/dropbox): Add DropboxConfig by @howiieyu in #3961
  • refactor: Polish internal types and remove not needed deps by @Xuanwo in #3964
  • refactor: Add concurrent error test for BlockWrite by @Xuanwo in #3968
  • refactor: Remove not needed types in icloud by @Xuanwo in #4021

Fixed

  • fix: Bump pyo3 to fix false positive of unnecessary_fallible_conversions by @Xuanwo in #3873
  • fix(core): Handling content encoding correctly by @Xuanwo in #3907
  • fix: fix RangeWriter incorrect next_offset by @WenyXu in #3927
  • fix(oio::BlockWrite): fix write_once case by @hoslo in #3953
  • fix: Don't retry close if concurrent > 1 to avoid content lost by @Xuanwo in #3957
  • fix(doc): fix rfc typos by @howiieyu in #3971
  • fix: Don't call wake_by_ref in OperatorFuture by @Xuanwo in #4003
  • fix: async fn resumed after initiate part failed by @Xuanwo in #4013
  • fix(pcloud,seafile): use get_basename and get_parent by @hoslo in #4020
  • fix(ci): remove pr author from review candidates by @dqhl76 in #4023

Docs

CI

  • build(deps): bump actions/download-artifact from 3 to 4 by @dependabot in #3885
  • build(deps): bump once_cell from 1.18.0 to 1.19.0 by @dependabot in #3880
  • build(deps): bump napi-derive from 2.14.2 to 2.14.6 by @dependabot in #3879
  • build(deps): bump url from 2.4.1 to 2.5.0 by @dependabot in #3876
  • build(deps): bump mlua from 0.8.10 to 0.9.2 by @oowl in #3890
  • ci: Disable supabase tests for our test org has been paused by @Xuanwo in #3908
  • ci: Downgrade artifact actions until regression addressed by @Xuanwo in #3935
  • ci: Refactor fuzz to integrate with test planner by @Xuanwo in #3936
  • ci: Pick random reviewers from committer list by @Xuanwo in #4001

Chore

Read more

Contributors

gabrielgrant, Xuanwo, and 17 other contributors
Assets 2
Loading

v0.44.1

06 Jan 08:50
@Xuanwo Xuanwo
v0.44.1
This tag was signed with the committer’s verified signature.
Xuanwo Xuanwo
GPG key ID: A61687076CEAD424
Learn about vigilant mode.
d4aab1e
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.44.1

What's Changed

Added

  • feat(service/memcached): Add MemCachedConfig by @ankit-pn in #3827
  • feat(service/rocksdb): Add RocksdbConfig by @ankit-pn in #3828
  • feat(services): add chainsafe support by @hoslo in #3834
  • feat(bindings/python): Build all available services for python by @Xuanwo in #3836
  • feat: Adding Atomicserver config by @k-aishwarya in #3845
  • feat(oio::read): implement the async buffer reader by @WenyXu in #3811
  • feat(oio::read): implement the blocking buffer reader by @WenyXu in #3860
  • feat: adapt the CompleteReader by @WenyXu in #3861
  • feat: add basic behavior tests for buffer reader by @WenyXu in #3862
  • feat: add fuzz reader with buffer tests by @WenyXu in #3866
  • feat(ofs): implement ofs based on fuse3 by @Inokinoki in #3857

Changed

  • refactor: simplify bindings_python.yml by @messense in #3837
  • refactor: Add edge test for aws assume role with web identity by @Xuanwo in #3839
  • refactor(services/webdav): Add WebdavConfig to implement ConfigDeserializer by @kwaa in #3846
  • refactor: use TwoWays instead of TwoWaysReader and TwoWaysWriter by @WenyXu in #3863

Fixed

  • fix: Add tests for listing recursively on not supported services by @Xuanwo in #3826
  • fix(services/upyun): fix list api by @hoslo in #3841
  • fix: fix a bypass seek relative bug in BufferReader by @WenyXu in #3864
  • fix: fix the bypass read does not sync the cur of BufferReader by @WenyXu in #3865

Docs

  • docs: Add Apache prefix for all bindings by @Xuanwo in #3829
  • docs: Add apache prefix for python docs by @Xuanwo in #3830
  • docs: Add branding in README by @Xuanwo in #3831
  • docs: Add trademark for Apache OpenDAL™ by @Xuanwo in #3832
  • docs: Add trademark sign for core by @Xuanwo in #3833
  • docs: Enable doc_auto_cfg when docs cfg has been enabled by @Xuanwo in #3835
  • docs: Address branding for haskell and C bindings by @Xuanwo in #3840
  • doc: add 0.44.0 release link to download.md by @dqhl76 in #3868

CI

  • ci: Remove workflows that not running or ready by @Xuanwo in #3842
  • ci: Migrate ftp to test planner by @Xuanwo in #3843

Chore

  • chore(bindings/java): Add name and description metadata by @tisonkun in #3838
  • chore(website): improve a bit trademark refs by @tisonkun in #3847
  • chore: Fix clippy warnings found in rust 1.75 by @Xuanwo in #3849
  • chore(bindings/python): improve ASF branding by @tisonkun in #3850
  • chore(bindings/haskell): improve ASF branding by @tisonkun in #3852
  • chore(bindings/c): make c binding separate workspace by @suyanhanx in #3856
  • chore(bindings/haskell): support co-log-0.6.0 && ghc-9.4 by @silver-ymz in #3858
  • chore: Bump to version 0.44.1 to start release process by @Xuanwo in #3869

New Contributors

Full Changelog: v0.44.0...v0.44.1

Contributors

messense, Xuanwo, and 10 other contributors
Assets 2
Loading

v0.44.0

31 Dec 03:00
@dqhl76 dqhl76
v0.44.0
This tag was signed with the committer’s verified signature.
dqhl76 Liuqing Yue
GPG key ID: 4F43CB2BC2E9EFE5
Learn about vigilant mode.
b0c6ea2
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.44.0

Upgrade Note

Rust core

Public API

Moka Service Configuration

  • The thread_pool_enabled option has been removed.

List Prefix Supported

After RFC: List Prefix landed, we have changed the behavior of list a path without /. OpenDAL used to return NotADirectory error, but now we will return the list of entries that start with given prefix instead.

Nodejs binding

Public API

Now, the list operation returns Array<Entry> instead of a lister.
Also, we removed scan, you can use list('some/path', {recursive: true})/listSync('some/path', {recursive: true}) instead of scan('some/path')/scanSync('some/path').

What's Changed

Added

Changed

  • refactor(service/etcd): use EtcdConfig in from_map by @G-XD in #3703
  • refactor(object_store): upgrade object_store to 0.7. by @youngsofun in #3713
  • refactor: List must support list without recursive by @Xuanwo in #3721
  • refactor: replace ftp tls impl as rustls by @oowl in #3760
  • refactor: Remove never used Stream poll_reset API by @Xuanwo in #3774
  • refactor: Polish operator read_with by @Xuanwo in #3775
  • refactor: Migrate gcs builder to config based by @Xuanwo in #3786
  • refactor(service/hdfs): Add HdfsConfig to implement ConfigDeserializer by @shbhmrzd in #3800
  • refactor(raw): add parse_header_to_str fn by @hoslo in #3804
  • refactor(raw): refactor APIs like parse_content_disposition by @hoslo in #3815
  • refactor: Polish http_util parse headers by @Xuanwo in #3817

Fixed

  • fix(oli): Fix cp -r command returns invalid path error by @kebe7jun in #3687
  • fix(website): folder name mismatch by @suyanhanx in #3707
  • fix(binding/java): fix SPECIAL_DIR_NAME by @G-XD in #3715
  • fix(services/dropbox): Workaround for dropbox limitations for create_folder by @Xuanwo in #3719
  • fix(ocaml_binding): sort actual & expected to pass ci by @Young-Flash in #3733
  • fix(ci): Make sure merge_local_staging handles all subdir by @Xuanwo in #3788
  • fix(services/gdrive): fix return value of get_file_id_by_path by @G-XD in #3801
  • fix(core): List root should not return itself by @Xuanwo in #3824

Docs

  • docs: add maturity model check by @suyanhanx in #3680
  • docs(website): show maturity model by @suyanhanx in #3709
  • docs(website): only VOTEs from PPMC members are binding by @G-XD in #3710
  • doc: add 0.43.0 release link to download.md by @G-XD in #3729
  • docs: Add process on nominating committers and ppmc members by @Xuanwo in #3740
  • docs: Deploy website to nightlies for every tags by @Xuanwo in #3739
  • docs: Remove not released bindings docs from top level header by @Xuanwo in #3741
  • docs: Add dependencies list for all packages by @Xuanwo in #3743
  • docs: Update maturity docs by @Xuanwo in #3750
  • docs: update the RFC doc by @suyanhanx in #3748
  • docs(website): polish deploy to nightlies by @suyanhanx in #3753
  • docs: add event calendar in community page by @dqhl76 in #3767
  • docs(community): polish events by @suyanhanx in #3768
  • docs(bindings/ruby): reflect test framework refactor by @tisonkun in #3798
  • docs(website): add service Huggingface to website by @morristai in #3812
  • docs: update release docs to add cargo-deny setup by @dqhl76 in #3821

CI

  • build(deps): bump cacache from 11.7.1 to 12.0.0 by @dependabot in #3690
  • build(deps): bump prometheus-client from 0.21.2 to 0.22.0 by @dependabot in #3694
  • build(deps): bump github/issue-labeler from 3.2 to 3.3 by @dependabot in #3698
  • ci: Add behavior test for b2 by @Xuanwo in #3714
  • ci(cargo): Add frame pointer support in build flag by @Zheaoli in #3772
  • ci: Workaround ring 0.17 build issue, bring aarch64 and armv7l back by @Xuanwo in #3781
  • ci: Support CI test for s3_read_on_wasm by @Zheaoli in #3813

Chore

  • chore: bump aws-sdk-s3 from 0.38.0 to 1.4.0 by @memoryFade in #3704
  • chore: Disable obs test for workaround by @Xuanwo in #3717
  • chore: Fix bindings CI by @Xuanwo in #3722
  • chore(binding/nodejs,website): Replace yarn with pnpm by @suyanhanx in #3730
  • chore: Bring persy CI back by @Xuanwo in #3751
  • chore(bindings/python): upgrade pyo3 to 0.20 by @messense in #3758
  • chore: remove unused binding feature file by @tisonkun in #3757
  • chore: Bump governor from 0.5.1 to 0.6.0 by @G-XD in #3761
  • chore: Split bindings/ocaml to separate workspace by @Xuanwo in #3792
  • chore: Split bindings/ruby to separate workspace by @ho-229 in #3794
  • chore(bindings/php): bump ext-php-rs to support latest php & separate workspace by @suyanhanx in #3799
  • chore: Address comments from hackernews by @Xuanwo in #3805
  • chore(bindings/ocaml): dep opendal point to core by @suyanhanx in #3814
  • chore: Bump to v0.44.0 to start release process by @dqhl76 in #3819

New Contributors

Full Changelog: http...

Read more

Contributors

messense, Xuanwo, and 16 other contributors
Assets 2
Loading

v0.43.0

08 Dec 10:43
@G-XD G-XD
v0.43.0
146e960
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.43.0

Upgrade Note

Rust Core

Public API

List Recursive

After RFC-3526: List Recursive landed, we have changed the list API to accept recursive instead of delimiter:

Users will need to change the following usage:

  • op.list_with(path).delimiter("") -> op.list_with(path).recursive(true)
  • op.list_with(path).delimiter("/") -> op.list_with(path).recursive(false)

delimiter other than "" and "/" is not supported anymore.

Stat a dir path

After RFC: List Prefix landed, we have changed the behavior of stat a dir path:

Here are the behavior list:

Case Path Result
stat existing dir abc/ Metadata with dir mode
stat existing file abc/def_file Metadata with file mode
stat dir without / abc/def_dir Error NotFound or metadata with dir mode
stat file with / abc/def_file/ Error NotFound
stat not existing path xyz Error NotFound

Services like s3, azblob can handle stat("abc/") correctly by check if there are objects with prefix abc/.

Raw API

Lister Align

We changed our internal lister implementation to align with the list public API for better performance and readability.

  • trait Page => List
  • struct Pager => Lister
  • trait BlockingPage => BlockingList
  • struct BlockingPager => BlockingLister

Every call to next will return an entry instead a page of entries. Also, we changed our async list api into poll based instead of async_trait.

Java binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

Node.js binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

Python binding

Breaking change

Because of a TLS lib issue, we temporarily disable the services-ftp feature.

C binding

There are no API changes.

What's Changed

Added

  • feat(bindings/C): Add opendal_operator_rename and opendal_operator_copy by @jiaoew1991 in #3517
  • feat(binding/python): Add new API to convert between AsyncOperator and Operator by @Zheaoli in #3514
  • feat: Implement RFC-3526: List Recursive by @Xuanwo in #3556
  • feat(service): add alluxio rest api support by @hoslo in #3564
  • feat(bindings/python): add OPENDAL_DISABLE_RANDOM_ROOT support by @Justin-Xiang in #3550
  • feat(core): add Alluxio e2e test by @hoslo in #3573
  • feat(service): alluxio support write by @hoslo in #3566
  • feat(bindings/nodejs): add retry layer by @suyanhanx in #3484
  • RFC: Concurrent Stat in List by @morristai in #3574
  • feat(service/hdfs): enable rename in hdfs service by @qingwen220 in #3592
  • feat: Improve the read_to_end perf and add benchmark vs_fs by @Xuanwo in #3617
  • feat: Add benchmark vs aws sdk s3 by @Xuanwo in #3620
  • feat: Improve the performance of s3 services by @Xuanwo in #3622
  • feat(service): support b2 by @hoslo in #3604
  • feat(core): Implement RFC-3574 Concurrent Stat In List by @morristai in #3599
  • feat: Implement stat dir correctly based on RFC-3243 List Prefix by @Xuanwo in #3651
  • feat(bindings/nodejs): Add capability support by @suyanhanx in #3654
  • feat: disable ftp for python and java binding by @ZutJoe in #3659
  • feat(bindings/nodejs): read/write stream by @suyanhanx in #3619

Changed

  • refactor(services/persy): migrate tot test planner by @G-XD in #3476
  • refactor(service/etcd): Add EtcdConfig to implement ConfigDeserializer by @Xuxiaotuan in #3543
  • chore(service/tikv): rename Backend to TikvBackend by @caicancai in #3545
  • refactor(services/azblob): add AzblobConfig by @acehinnnqru in #3553
  • refactor(services/cacache): migrate to test planner by @G-XD in #3568
  • refactor(services/sled): migrate to test planner by @G-XD in #3569
  • refactor(services/webhdfs): migrate to test planner by @G-XD in #3578
  • refactor(core): Rename all Page to List by @Xuanwo in #3589
  • refactor: Change List API into poll based and return one entry instead by @Xuanwo in #3593
  • refactor(services/tikv): migrate to test planner by @G-XD in #3587
  • refactor(service/redis): Migrate task to new task planner by @sunheyi6 in #3374
  • refactor(oio): Polish IncomingAsyncBody::bytes by @Xuanwo in #3621
  • refactor(services/rocksdb): migrate to test planner by @G-XD in #3636
  • refactor(services/azfile): Check if dir exists before create by @ZutJoe in #3652
  • refactor: Polish concurrent list by @Xuanwo in #3658

Fixed

Docs

CI

  • ci(services/libsql): add rust test threads limit by @G-XD in #3540
  • ci(services/redb): migrate to test planner by @suyanhanx in #3518
  • ci: Disable libsql behavior test until we or upstream address them by @Xuanwo in #3552
  • ci: Add new Python binding reviewer by @Zheaoli in #3560
  • ci(bindings/nodejs): add aarch64 build support by @suyanhanx in #3567
  • ci(planner): Polish the workflow planner code by @Zheaoli in #3570
  • ci(core): Add dry run for rc tags by @Xuanwo in #3624
  • ci: Disable persy until it has been fixed by @Xuanwo in #3631
  • ci: Calling cargo to make sure rust has been setup by @Xuanwo in #3633
  • ci: Fix etcd with tls and auth failed to start by @Xuanwo in #3637
  • ci(services/etcd): Use ALLOW_NONE_AUTHENTICATION as workaround by @Xuanwo in #3638
  • ci: dry run publish on rc tags for python binding by @everpcpc in #3645
  • ci: Add java linux arm64 build by @Xuanwo in #3660
  • ci(java/binding): Use zigbuild for glibc 2.17 support by @Xuanwo in #3664
  • ci(bindings/python): remove aarch support by @G-XD in #3674

Chore

  • chore(servies/sftp): Upgrade openssh-sftp-client to 0.14 by @sd44 in https://g...
Read more

Contributors

vincentdephily, jtescher, and 25 other contributors
Assets 2
Loading

v0.42.0

16 Nov 04:17
@silver-ymz silver-ymz
v0.42.0
This tag was signed with the committer’s verified signature.
silver-ymz Mingzhuo Yin
SSH Key Fingerprint: +yDj/QuXz5L9uiuMa+pwZ2C/vpdNJphkPuec2oD3Hlw Learn about vigilant mode.
9b423ae
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.42.0

Upgrade Note

Rust Core

MSRV Changed

OpenDAL bumps it's MSRV to 1.67.0.

S3 Service Configuration

  • The enable_exact_buf_write option has been deprecated and is superseded by BufferedWriter, introduced in version 0.40.

Oss Service Configuration

  • The write_min_size option has been deprecated and replaced by BufferedWriter, also introduced in version 0.40.
  • A new setting, allow_anonymous, has been added. Since v0.41, OSS will now return an error if credential loading fails. Enabling allow_anonymous to fallback to request without credentials.

Ghac Service Configuration

  • The enable_create_simulation option has been removed. We add this option to allow ghac simulate create empty file, but it's could result in unexpected behavior when users create a file with content length 1. So we remove it.

Wasabi Service Removed

wasabi service native support has been removed. Users who want to access wasabi can use our s3 service instead.

Java binding

There are no API changes.

Node.js binding

There are no API changes.

Python binding

Breaking change for layers

Operator and BlockingOperator won't accept layers anymore. Instead, we provide a layer API:

op = opendal.Operator("memory").layer(opendal.layers.RetryLayer())

We removed not used layers ConcurrentLimitLayer and ImmutableIndexLayer along with this change.

File and AsyncFile

OpenDAL removes Reader and AsyncReader classes, instead, we provide file-like object File and AsyncFile.

Open a file for reading in blocking way:

with op.open(filename, "rb") as r:
    content = r.read()

Open a file for reading in async way:

async with await op.open(filename, "rb") as r:
    content = await r.read()

Breaking change for Errors

We remove the old error classes and provide a couple of Exception based class for the error handling.

  1. opendal.Error is based class for all the exceptions now.
  2. opendal.exceptions.Unexpected is added.
  3. opendal.exceptions.Unsupported is added.
  4. opendal.exceptions.ConfigInvalid is added.
  5. opendal.exceptions.NotFound is added.
  6. opendal.exceptions.PermissionDenied is added.
  7. opendal.exceptions.IsADirectory is added.
  8. opendal.exceptions.NotADirectory is added.
  9. opendal.exceptions.AlreadyExists is added.
  10. opendal.exceptions.IsSameFile is added.
  11. opendal.exceptions.ConditionNotMatch is added.
  12. opendal.exceptions.ContentTruncated is added.
  13. opendal.exceptions.ContentIncomplete is added.
  14. opendal.exceptions.InvalidInput is added.

C binding

The naming convention for C binding has been altered.

Struct Naming

Renaming certain struct names for consistency.

  • opendal_operator_ptr => opendal_operator
  • opendal_blocking_lister => opendal_lister
  • opendal_list_entry => opendal_entry

API Naming

We've eliminated the blocking_ prefix from our API because C binding doesn't currently support async. In the future, we plan to introduce APIs such as opendal_operator_async_write.

  • opendal_operator_blocking_write => opendal_operator_write
  • opendal_operator_blocking_read => opendal_operator_read

What's Changed

Added

  • feat(binding/java): add rename support by @G-XD in #3238
  • feat(prometheus): add bytes metrics as counter by @flaneur2020 in #3246
  • feat(binding/python): new behavior testing for python by @laipz8200 in #3245
  • feat(binding/python): Support AsyncOperator tests. by @laipz8200 in #3254
  • feat(service/libsql): support libsql by @G-XD in #3233
  • feat(binding/python): allow setting append/buffer/more in write() call by @jokester in #3256
  • feat(services/persy): change blocking_x in async_x call to tokio::task::blocking_spawn by @Zheaoli in #3221
  • feat: Add edge test cases for OpenDAL Core by @Xuanwo in #3274
  • feat(service/d1): Support d1 for opendal by @realtaobo in #3248
  • feat(services/redb): change blocking_x in async_x call to tokio::task::blocking_spawn by @shauvet in #3276
  • feat: Add blocking layer for C bindings by @jiaoew1991 in #3278
  • feat(binding/c): Add blocking_reader for C binding by @jiaoew1991 in #3259
  • feat(services/sled): change blocking_x in async_x call to tokio::task::blocking_spawn by @shauvet in #3280
  • feat(services/rocksdb): change blocking_x in async_x call to tokio::task::blocking_spawn by @shauvet in #3279
  • feat(binding/java): make Metadata a POJO by @G-XD in #3277
  • feat(bindings/java): convey backtrace on exception by @tisonkun in #3286
  • feat(layer/prometheus): Support custom metric bucket for Histogram by @Zheaoli in #3275
  • feat(bindings/python): read APIs return memoryview instead of bytes to avoid copy by @messense in #3310
  • feat(service/azfile): add azure file service support by @dqhl76 in #3312
  • feat(services/oss): Add allow anonymous support by @Xuanwo in #3321
  • feat(bindings/python): build and publish aarch64 and armv7l wheels by @messense in #3325
  • feat(bindings/java): support duplicate operator by @tisonkun in #3330
  • feat(core): Add enabled for Scheme by @Xuanwo in #3331
  • feat(bindings/java): support get enabled services by @tisonkun in #3336
  • feat(bindings/java): Migrate behavior tests to new Workflow Planner by @Xuanwo in #3341
  • feat(layer/prometheus): Support output path as a metric label by @Zheaoli in #3335
  • feat(service/mongodb): Support mongodb service by @Zheaoli in #3355
  • feat: Make PrometheusClientLayer Clonable by @flaneur2020 in #3352
  • feat(service/cloudflare_kv): support cloudflare KV by @my-vegetable-has-exploded in #3348
  • feat(core): exposing Metadata::metakey() api by @G-XD in #3373
  • feat(binding/java): add list & remove_all support by @G-XD in #3333
  • feat: Add write_total_max_size in Capability by @realtaobo in #3309
  • feat(core): service add DBFS API 2.0 support by @morristai in #3334
  • feat(bindings/java): use random root for behavior tests by @tisonkun in #3408
  • feat(services/oss): Add start-after support for oss list by @wcy-fdu in #3410
  • feat(binding/python): Export full_capability API for Python binding by @Zheaoli in #3402
  • feat(test): Enable new test workflow planner for python binding by @Zheaoli in #3397
  • feat: Implement Lazy Reader by @Xuanwo in #3395
  • feat(binding/nodejs): upgrade test behavior and infra by @eryue0220 in #3297
  • feat(binding/python): Support Copy operation for Python binding by @Zheaoli in #3454
  • feat(bindings/python): Add layer API for operator by @Xuanwo in #3464
  • feat(bindings/java): add layers onto ops by @tisonkun in #3392
  • feat(binding/python): Support rename API for Python binding by @Zheaoli in #3467
  • feat(binding/python): Support remove_all API for Python binding by @Zheaoli in #3469
  • feat(core): fix token leak in OneDrive by @morristai in #3470
  • feat(core): service add OpenStack Swift support by @morristai in #3461
  • feat(bindings/python)!: Implement File and AsyncFile to replace Reader by @Xuanwo in #3474
  • feat(services): Implement ConfigDeserializer and add S3Config as example by @Xuanwo in #3490
  • feat(core): add OpenStack Swift e2e test by @morristai in #3493
  • feat(doc): add OpenStack Swift document for the website by @morristai in #3494
  • feat(services/sqlite): add SqliteConfig by @hoslo in #3497
  • feat(bindings/C): imp...
Read more

Contributors

flaneur2020, jokester, and 32 other contributors
Assets 2
Loading

v0.41.0

16 Oct 08:50
@suyanhanx suyanhanx
v0.41.0
This tag was signed with the committer’s verified signature.
suyanhanx Suyan
GPG key ID: 2AAECF0ABFC5F308
Learn about vigilant mode.
779750a
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.41.0

Upgrade Note

Rust Core

There are no public API and raw API changes.

Java binding

Breaking change for constructing operators

PR-3166 changes the API for constructing operators:

Previous:

new BlockingOperator(scheme, config);
new Operator(scheme, config);

Current:

BlockingOperator.of(scheme, config);
Operator.of(scheme, config);

Now, there is no public constructor for operators, but only factory methods. In this way, the APIs are free to do arbitrary verifications and preparations before constructing operators.

Node.js binding

There are no API changes.

Python binding

There are no API changes.

What's Changed

Added

  • feat: allow using prometheus-client crate with PrometheusClientLayer by @flaneur2020 in #3134
  • feat(binding/java): support info ops by @G-XD in #3154
  • test(binding/java): add behavior test framework by @G-XD in #3129
  • feat: Include starting offset for GHAC upload Content-Range by @huonw in #3163
  • feat(bindings/cpp): make ReaderStream manage the lifetime of Reader by @silver-ymz in #3164
  • feat: Enable multi write for ghac by @Xuanwo in #3165
  • feat: Add mysql support for OpenDAL by @Zheaoli in #3170
  • feat(service/postgresql): support connection pool by @Zheaoli in #3176
  • feat(services/ghac): Allow explicitly setting ghac endpoint/token, not just env vars by @huonw in #3177
  • feat(service/azdls): add append support for azdls by @dqhl76 in #3186
  • feat(bindings/python): Enable BlockingLayer for non-blocking services that don't support blocking by @messense in #3198
  • feat: Add write_can_empty in Capability and related tests by @Xuanwo in #3200
  • feat: Add basic support for bindings/go using CGO by @jiaoew1991 in #3204
  • feat(binding/java): add copy test by @G-XD in #3207
  • feat(service/sqlite): Support sqlite for opendal by @Zheaoli in #3212
  • feat(services/sqlite): Support blocking_get/set/delete in sqlite service by @Zheaoli in #3218
  • feat(oay): port WebdavFs to dav-server-fs-opendal by @Young-Flash in #3119

Changed

  • refactor(services/dropbox): Use OpWrite instead of passing all args as parameters by @ImSingee in #3126
  • refactor(binding/java): read should return bytes by @tisonkun in #3153
  • refactor(bindings/java)!: operator jni calls by @tisonkun in #3166
  • refactor(tests): reuse function to remove duplicate code by @zhao-gang in #3219

Fixed

  • fix(tests): Create test files one by one instead of concurrently by @Xuanwo in #3132
  • chore(ci): fix web identity token path for aws s3 assume role test by @everpcpc in #3141
  • fix(services/s3): Detect region returned too early when header is empty by @Xuanwo in #3187
  • fix: making OpenDAL compilable on 32hf platforms by @ClSlaid in #3188
  • fix(binding/java): decode Java’s modified UTF-8 format by @G-XD in #3195

Docs

CI

  • ci: Align tags with semver specs by @Xuanwo in #3136
  • ci: Migrate obs to databend labs sponsored bucket by @Xuanwo in #3137
  • build(bindings/java): support develop with JDK 21 by @tisonkun in #3140
  • ci: Migrate GCS to Databend Labs sponsored bucket by @Xuanwo in #3142
  • build(bindings/java): upgrade maven wrapper version by @tisonkun in #3167
  • build(bindings/java): support explicit cargo build target by @tisonkun in #3168
  • ci: Pin Kvrocks docker image to 2.5.1 to avoid test failure by @git-hulk in #3192
  • ci(bindings/ocaml): add doc by @Ranxy in #3208
  • build(deps): bump actions/checkout from 3 to 4 by @dependabot in #3222
  • build(deps): bump korandoru/hawkeye from 3.3.0 to 3.4.0 by @dependabot in #3223
  • build(deps): bump rusqlite from 0.25.4 to 0.29.0 by @dependabot in #3226

Chore

New Contributors

Full Changelog: v0.40.0...v0.41.0

Contributors

flaneur2020, zhao-gang, and 20 other contributors
Assets 2
Loading

v0.40.0

21 Sep 05:36
@Xuanwo Xuanwo
v0.40.0
This tag was signed with the committer’s verified signature.
Xuanwo Xuanwo
GPG key ID: A61687076CEAD424
Learn about vigilant mode.
9a775bd
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.40.0

Checkout our OwO #1 to know more about this release!

Upgrade Note

Public API

RFC-2578 Merge Append Into Write

RFC-2578 merges append into write and removes append API.

  • For writing a file at once, please use op.write() for convenience.
  • For appending a file, please use op.write_with().append(true) instead of op.append().

The same rule applies to writer() and writer_with().

RFC-2774 Lister API

RFC-2774 proposes a new lister API to replace current list and scan. And we add a new API list to return entries directly.

  • For listing a directory at once, please use list() for convenience.
  • For listing a directory recursively, please use list_with().delimiter("") or lister_with().delimiter("") instead of scan().
  • For listing in streaming, please use lister() or lister_with() instead.

RFC-2779 List With Metakey

RFC-2779 proposes a new op.list_with().metakey() API to allow list with metakey and removes op.metadata(&entry) API.

Please use op.list_with().metakey() instead of op.metadata(&entry), for example:

// Before
let entries: Vec<Entry> = op.list("dir/").await?;
for entry in entris {
  let meta = op.metadata(&entry, Metakey::ContentLength | Metakey::ContentType).await?;
  println!("{} {}", entry.name(), entry.metadata().content_length());
}

// After
let entries: Vec<Entry> = op
  .list_with("dir/")
  .metakey(Metakey::ContentLength | Metakey::ContentType).await?;
for entry in entris {
  println!("{} {}", entry.name(), entry.metadata().content_length());
}

RFC-2852: Native Capability

RFC-2852 proposes new native_capability and full_capability API to allow users to check if the underlying service supports a capability natively.

  • native_capability returns true if the capability is supported natively.
  • full_capability returns true if the capability is supported, maybe via a layer.

Most of time, you can use full_capability to replace capability call. But if to check if the capability is supported natively for better performance design, please use native_capability instead.

Buffered Writer

OpenDAL v0.40 added buffered writer support!

Users don't need to specify the content_length() for writer anymore!

- let mut w = op.writer_with("path/to/file").content_length(1024).await?;
+ let mut w = op.writer_with("path/to/file").await?;

Users can specify the buffer() to control the size we call underlying storage:

let mut w = op.writer_with("path/to/file").buffer(8 * 1024 * 1024).await?;

If buffer is not specified, we will call underlying storage everytime we call write. Otherwise, we will make sure to call underlying storage when buffer is full or close is called.

Raw API

RFC-3017 Remove Write Copy From

RFC-3017 removes copy_from API from the oio::Write trait. Users who implements services and layers by hand should remove this API.

What's Changed

Added

  • feat(service/etcd): support list by @G-XD in #2755
  • feat: setup the integrate with PHP binding by @godruoyi in #2726
  • feat(oay): Add read_dir by @Young-Flash in #2736
  • feat(obs): support loading credential from env by @everpcpc in #2767
  • feat: add async backtrace layer by @dqhl76 in #2765
  • feat: Add OCaml Binding by @Ranxy in #2757
  • feat(bindings/haskell): support logging layer by @silver-ymz in #2705
  • feat: Add FoundationDB Support for OpenDAL by @ArmandoZ in #2751
  • feat(oay): add write for oay webdav by @Young-Flash in #2769
  • feat: Implement RFC-2774 Lister API by @Xuanwo in #2787
  • feat(bindings/haskell): enhance original OpMonad to support custom IO monad by @silver-ymz in #2789
  • feat: Add into_seekable_read_by_range support for blocking read by @Xuanwo in #2799
  • feat(layers/blocking): add blocking layer by @yah01 in #2780
  • feat: Add async list with metakey support by @Xuanwo in #2803
  • feat(binding/php): Add basic io by @godruoyi in #2782
  • feat: fuzz test support read from .env by different services by @dqhl76 in #2824
  • feat(services/rocksdb): Add scan support by @JLerxky in #2827
  • feat: Add postgresql support for OpenDAL by @Xuanwo in #2815
  • feat: ci for php binding by @godruoyi in #2830
  • feat: Add create_dir, remove, copy and rename API for oay-webdav by @Young-Flash in #2832
  • feat(oli): oli stat should show path as specified by users by @sarutak in #2842
  • feat(services/moka, services/mini-moka): Add scan support by @JLerxky in #2850
  • feat(oay): impl some method for WebdavMetaData by @Young-Flash in #2857
  • feat: Implement list with metakey for blocking by @Xuanwo in #2861
  • feat(services/redis): add redis cluster support by @G-XD in #2858
  • feat(services/dropbox): read support range by @suyanhanx in #2848
  • feat(layers/logging): Allow users to control print backtrace or not by @Xuanwo in #2872
  • feat: add native & full capability by @yah01 in #2874
  • feat: Implement RFC-2758 Merge Append Into Write by @Xuanwo in #2880
  • feat(binding/ocaml): Add support for operator reader and metadata by @Ranxy in #2881
  • feat(core): replace field _pin with !Unpin as argument by @morristai in #2886
  • feat: Add retry for Writer::sink operation by @Xuanwo in #2896
  • feat: remove operator range_read and range_reader API by @oowl in #2898
  • feat(core): Add unit test for ChunkedCursor by @Xuanwo in #2907
  • feat(types): remove blocking operation range_read and range_reader API by @oowl in #2912
  • feat(types): add stat_with API for blocking operator by @oowl in #2915
  • feat(services/gdrive): credential manage by @suyanhanx in #2914
  • feat(core): Implement Exact Buf Writer by @Xuanwo in #2917
  • feat: Add benchmark for buf write by @Xuanwo in #2922
  • feat(core/raw): Add stream support for multipart by @Xuanwo in #2923
  • feat(types): synchronous blocking operator and operator's API by @oowl in #2924
  • feat(bindings/java): bundled services by @tisonkun in #2934
  • feat(core/raw): support stream body for mixedpart by @silver-ymz in #2936
  • feat(bindings/python): expose presign api by @silver-ymz in #2950
  • feat(bindings/nodejs): Implement presign test by @suyanhanx in #2969
  • docs(services/gdrive): update service doc by @suyanhanx in #2973
  • feat(bindings/cpp): init cpp binding by @silver-ymz in #2980
  • feat: gcs insert object support cache control by @fatelei in #2974
  • feat(bindings/cpp): expose all api returned by value by @silver-ymz in #3001
  • feat(services/gdrive): implement rename by @suyanhanx in #3007
  • feat(bindings/cpp): expose reader by @silver-ymz in #3004
  • feat(bindings/cpp): expose lister by @silver-ymz in #3011
  • feat(core): Avoid copy if input is larger than buffer_size by @Xuanwo in #3016
  • feat(service/gdrive): add gdrive list support by @Young-Flash in #3025
  • feat(services/etcd): Enable etcd connection pool by @Xuanwo in #3041
  • feat: Add buffer support for all services by @Xuanwo in #3045
  • feat(bindings/ja...
Read more

Contributors

jbampton, fatelei, and 34 other contributors
Assets 2
Loading