Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

doc: update VISION and behavior tests doc #2009

Merged
merged 1 commit into from
Apr 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/service_test_fs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ jobs:
- name: Test
shell: bash
working-directory: core
run: cargo test fs -- --show-output
run: cargo test services_fs -- --show-output
env:
RUST_BACKTRACE: full
RUST_LOG: debug
Expand Down
38 changes: 30 additions & 8 deletions core/tests/behavior/README.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
# Behavior Test
# Behavior Test for OpenDAL

Behavior Tests are used to make sure every service works correctly.
Behavior tests are used to make sure every service works correctly.

To support different testing backends simultaneously, we use `environment value` to carry the backend config.

## Setup

To run the behavior test, please copy `.env.example` to `.env` and change the values on need.
To run the behavior tests, please copy the `.env.example`, which is at project root, to `.env` and change the values on need.

Take `fs` for example. We need to change to enable behavior test on `fs` on `/tmp`.
Take `fs` for example, we need to change to enable behavior test on `fs` on `/tmp`.

```dotenv
OPENDAL_FS_TEST=false
Expand All @@ -22,18 +22,40 @@ OPENDAL_FS_TEST=on
OPENDAL_FS_ROOT=/tmp
```

Notice: If the env is not set, all behavior tests will be skipped by default.
Notice: If the env variables are not set, all behavior tests will be skipped by default.

## Run

Test all available backend.
Test all available backends.

```shell
cargo test
```

Test specific backend.
Test specific backend(such as `fs`).

```shell
cargo test fs
cargo test services_fs
```

As `cargo test` only run tests containing the following string in their names, we use `services-fs` to run all tests under `services::fs`.

To run all tests under `tests/behavior/write.rs` for `fs`, we use `services_fs_write`.

```shell
cargo test services_fs_write
```

You can also run specific test(such as `test_stat`) for specific backend.

```shell
cargo test services_fs_write::test_stat
```

To get the full name of a behavior module, please check the `mod` declaration in `tests/behavior/*.rs`.(It is always the same as the file name.)

## Debug

To debug a behavior test, you can use `RUST_LOG=debug RUST_BACKTRACE=full cargo test -- --show-output` to print the log with backtrace.

For more details, please visit [cargo test](https://doc.rust-lang.org/cargo/commands/cargo-test.html) or run the command `cargo test --help`.
4 changes: 2 additions & 2 deletions core/tests/behavior/read_only.rs
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ use sha2::Sha256;
macro_rules! behavior_read_test {
($service:ident, $($(#[$meta:meta])* $test:ident),*,) => {
paste::item! {
mod [<services_ $service:lower _read>] {
mod [<services_ $service:lower _read_only>] {
$(
#[tokio::test]
$(
Expand All @@ -41,7 +41,7 @@ macro_rules! behavior_read_test {
match op {
Some(op) if op.info().can_read() && !op.info().can_write() => $crate::read_only::$test(op).await,
Some(_) => {
log::warn!("service {} doesn't support read, ignored", opendal::Scheme::$service);
log::warn!("service {} doesn't support read_only, ignored", opendal::Scheme::$service);
Ok(())
},
None => {
Expand Down
16 changes: 8 additions & 8 deletions website/docs/vision.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,26 +11,26 @@ This is an overview of what the shape of OpenDAL looks like, but also somewhat z

## 1. Free from services

OpenDAL must enable users to access various storage services ranging from `s3` to `dropbox` via it's own native API. It should provide a unified API for accessing all these services.
OpenDAL must enable users to access various storage services ranging from `s3` to `dropbox` via its own native API. It should provide a unified API for accessing all these services.

### Examples

- Add support for [Google Drive](https://www.google.com/drive/): Good, it allows users to access and manage their data on the [Google Drive](https://www.google.com/drive/).
- Add support for oss via native API: Good, users can utilize Aliyun's RAM support.
- Add support for [supabase storage](https://supabase.com/docs/guides/storage): Good, users can visit supabase storage now!
- Add support for [Object Storage Service (OSS)](https://www.alibabacloud.com/product/object-storage-service) via native API: Good, users can utilize Aliyun's RAM support.
- Add support for [supabase storage](https://supabase.com/docs/guides/storage): Good, users can visit `supabase storage` now!


- Add support for [GCS](https://cloud.google.com/storage) via [XML API](https://cloud.google.com/storage/docs/xml-api/overview): Bad, [GCS](https://cloud.google.com/storage) has native [JSON API](https://cloud.google.com/storage/docs/json_api) which more powerful
- Add support for MySQL/PostgreSQL: Bad, relational DBMS provides data types such as BLOB, but they are often not used as a storage service.
- Add support for [Google Cloud Storage(GCS)](https://cloud.google.com/storage) via [XML API](https://cloud.google.com/storage/docs/xml-api/overview): Bad, [GCS](https://cloud.google.com/storage) has native [JSON API](https://cloud.google.com/storage/docs/json_api) which more powerful
- Add support for `MySQL/PostgreSQL`: Bad, relational DBMS provides data types such as BLOB, but they are often not used as a storage service.

## 2. Free from implementations

OpenDAL needs to separate the various implementation details of services and enables users to write identical logic for different services.

### Examples

- Add a new capability to indicate whether or not `presign` is supported: Good, users can now write logic based on the `can_presign` option.
- Add a `default_storage_class` configuration for the S3 service: Good, configuration is specific to the s3 service.
- Add a new capability to indicate whether `presign` is supported: Good, users can now write logic based on the `can_presign` option.
- Add a `default_storage_class` configuration for the S3 service: Good, configuration is specific to the S3 service.
- Add an option for `content_type` in the `write` operation: Good, it aligns with HTTP standards.


Expand Down Expand Up @@ -58,4 +58,4 @@ OpenDAL needs to implement features in zero cost way which means:
- Implement `seek` for Reader: Good, users can't write better `seek` support, they all need to pay the same cost.


- Add `Arc` for metadata: Bad, users may only need to use metadata once and never clone it. For those who do want this feature, they can add `Arc` themselves
- Add `Arc` for metadata: Bad, users may only need to use metadata once and never clone it. For those who do want this feature, they can add `Arc` themselves.