Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat(services/dropbox): impl create_dir and polish error handling #2600

Merged
merged 14 commits into from
Jul 6, 2023
Merged

feat(services/dropbox): impl create_dir and polish error handling #2600

merged 14 commits into from
Jul 6, 2023

Conversation

suyanhanx
Copy link
Member

@suyanhanx suyanhanx commented Jul 5, 2023 •
edited
Loading

This PR does the following:

  • Make the configuration root effective
  • Impl create_dir
  • polish error handling

@suyanhanx
test(services/dropbox): enable CI test for dropbox
c9f9dda 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
try
af49b45 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
try
ca691a9 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
fix
9b4c2b4 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
support create_dir
20009ff 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
fix root not affected
462fe4d 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
remove nextest
bf94f7b 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
fix
1fa9100 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
fix
0dbd132 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
tip for test run
f96beba 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx suyanhanx changed the title feat(CI): enable test for dropbox feat(services/dropbox): impl create_dir and polish error handling Jul 6, 2023
suyanhanx
suyanhanx commented Jul 6, 2023
View reviewed changes
Copy link
Member Author

@suyanhanx suyanhanx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For dropbox, it is very easy to trigger the too_many_write_operations error.
According to the official statement, this is not a traffic limitation, but too many objects on the same path are operated. So it can't pass the test_delete_stream test because there is too much concurrency.
After several attempts, the test passed when I changed the limit to 10.

https://github.com/apache/incubator-opendal/blob/ed90281742c16ec42fa7cb785f1527d359fa0fcd/core/tests/behavior/write.rs#L1030-L1054

.github/workflows/service_test_dropbox.yml Outdated Show resolved Hide resolved
@suyanhanx
make fmt happy
dd0d6d0 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx suyanhanx marked this pull request as ready for review July 6, 2023 08:41
@suyanhanx
make clippy happy
03881cc 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
Copy link
Member Author

The only test which failed:

image

@suyanhanx
Copy link
Member Author

I found that there has a delete_batch API available. I will follow up and try.
If it works, it will allow this service to pass all tests.

Xuanwo
Xuanwo reviewed Jul 6, 2023
View reviewed changes
core/src/services/dropbox/core.rs Outdated Show resolved Hide resolved
core/src/services/dropbox/builder.rs Outdated Show resolved Hide resolved
.github/workflows/service_test_dropbox.yml Outdated Show resolved Hide resolved
core/src/services/dropbox/backend.rs Outdated Show resolved Hide resolved
core/src/services/dropbox/core.rs Outdated Show resolved Hide resolved
core/src/services/dropbox/core.rs Show resolved Hide resolved
core/src/services/dropbox/error.rs Show resolved Hide resolved
@suyanhanx
remove CI config file
b202dc1 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx
polish
2135f1a 
Signed-off-by: suyanhanx <[email protected]>
@suyanhanx suyanhanx requested a review from Xuanwo July 6, 2023 11:26
Xuanwo
Xuanwo approved these changes Jul 6, 2023
View reviewed changes
Copy link
Member

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@Xuanwo Xuanwo merged commit a2634c4 into main Jul 6, 2023
@Xuanwo Xuanwo deleted the dropbox-test branch July 6, 2023 12:01
@suyanhanx suyanhanx mentioned this pull request Jul 6, 2023
Closed
5 tasks
ClSlaid added a commit to ClSlaid/opendal that referenced this pull request Jul 14, 2023
@ClSlaid
Bump to version 0.38.1
c758b2d 
* feat(binding/lua): add rename and create_dir operator function by @oowl in apache#2564
* feat(services/azblob): support sink by @suyanhanx in apache#2574
* feat(services/gcs): support sink by @suyanhanx in apache#2576
* feat(services/oss): support sink by @suyanhanx in apache#2577
* feat(services/obs): support sink by @suyanhanx in apache#2578
* feat(services/cos): impl sink by @suyanhanx in apache#2587
* feat(service): Support stat for Dropbox by @Zheaoli in apache#2588
* feat(services/dropbox): impl create_dir and polish error handling by @suyanhanx in apache#2600
* feat(services/dropbox): Implement refresh token support by @Xuanwo in apache#2604
* feat(service/dropbox): impl batch delete by @suyanhanx in apache#2606
* feat(CI): set Kvrocks test for service redis by @suyanhanx in apache#2613
* feat(core): object versioning APIs by @suyanhanx in apache#2614
* feat(oay): actually read configuration from `oay.toml` by @messense in apache#2615
* feat(services/webdav): impl sink by @suyanhanx in apache#2622
* feat(services/fs): impl Sink for Fs by @Ji-Xinyou in apache#2626
* feat(core): impl `delete_with` on blocking operator by @suyanhanx in apache#2633
* feat(bindings/C): add support for list in C binding by @Ji-Xinyou in apache#2448
* refactor(core): Add ErrorKind InvalidInput to indicate users input error by @dqhl76 in apache#2637
* fix(doc): fix codeblock rendering by @xxchan in apache#2592
* fix(service/minitrace): should set local parent by @andylokandy in apache#2620
* fix(service/minitrace): update doc by @andylokandy in apache#2621
* doc(bindings/haskell): add module document by @silver-ymz in apache#2566
* docs: Update license related comments by @Prashanth-Chandra in apache#2573
* docs: add hdfs namenode High Availability related troubleshoot by @wcy-fdu in apache#2601
* docs: polish release doc by @PsiACE in apache#2608
* docs(blog): add Apache OpenDAL(Incubating): Access Data Freely by @PsiACE in apache#2607
* docs(RFC): Object Versioning by @suyanhanx in apache#2602
* ci: Disable bindings/java deploy for now by @tisonkun in apache#2560
* ci: Disable the failed stage-release job instead by @tisonkun in apache#2561
* ci: add haddock generator for haskell binding by @silver-ymz in apache#2569
* ci(binding/lua): add luarocks package manager support by @oowl in apache#2558
* build(deps): bump predicates from 2.1.5 to 3.0.1 by @dependabot in apache#2583
* build(deps): bump tower-http from 0.4.0 to 0.4.1 by @dependabot in apache#2582
* build(deps): bump chrono from 0.4.24 to 0.4.26 by @dependabot in apache#2581
* build(deps): bump redis from 0.22.3 to 0.23.0 by @dependabot in apache#2580
* build(deps): bump cbindgen from 0.24.3 to 0.24.5 by @dependabot in apache#2579
* ci: upgrade hawkeye to v3 by @tisonkun in apache#2585
* ci(services/webdav): Setup integration test for nextcloud by @Xuanwo in apache#2631
* chore: add haskell binding link to website by @silver-ymz in apache#2571
* chore: fix cargo warning for resolver by @xxchan in apache#2590
* chore: bump log to 0.4.19 by @xxchan in apache#2591
* chore(deps): update deps to latest version by @suyanhanx in apache#2596
* chore: Add release 0.38.0 to download by @PsiACE in apache#2597
* chore(service/minitrace): automatically generate span name by @andylokandy in apache#2618

* @Prashanth-Chandra made their first contribution in apache#2573
* @andylokandy made their first contribution in apache#2618

**Full Changelog**: apache/opendal@v0.38.0...v0.38.1

Signed-off-by: ClSlaid <[email protected]>
@ClSlaid ClSlaid mentioned this pull request Jul 14, 2023
Merged
Xuanwo pushed a commit that referenced this pull request Jul 14, 2023
@ClSlaid
Bump to version 0.38.1 (#2638)
40e2a2a 
* Bump to version 0.38.1

* feat(binding/lua): add rename and create_dir operator function by @oowl in #2564
* feat(services/azblob): support sink by @suyanhanx in #2574
* feat(services/gcs): support sink by @suyanhanx in #2576
* feat(services/oss): support sink by @suyanhanx in #2577
* feat(services/obs): support sink by @suyanhanx in #2578
* feat(services/cos): impl sink by @suyanhanx in #2587
* feat(service): Support stat for Dropbox by @Zheaoli in #2588
* feat(services/dropbox): impl create_dir and polish error handling by @suyanhanx in #2600
* feat(services/dropbox): Implement refresh token support by @Xuanwo in #2604
* feat(service/dropbox): impl batch delete by @suyanhanx in #2606
* feat(CI): set Kvrocks test for service redis by @suyanhanx in #2613
* feat(core): object versioning APIs by @suyanhanx in #2614
* feat(oay): actually read configuration from `oay.toml` by @messense in #2615
* feat(services/webdav): impl sink by @suyanhanx in #2622
* feat(services/fs): impl Sink for Fs by @Ji-Xinyou in #2626
* feat(core): impl `delete_with` on blocking operator by @suyanhanx in #2633
* feat(bindings/C): add support for list in C binding by @Ji-Xinyou in #2448
* refactor(core): Add ErrorKind InvalidInput to indicate users input error by @dqhl76 in #2637
* fix(doc): fix codeblock rendering by @xxchan in #2592
* fix(service/minitrace): should set local parent by @andylokandy in #2620
* fix(service/minitrace): update doc by @andylokandy in #2621
* doc(bindings/haskell): add module document by @silver-ymz in #2566
* docs: Update license related comments by @Prashanth-Chandra in #2573
* docs: add hdfs namenode High Availability related troubleshoot by @wcy-fdu in #2601
* docs: polish release doc by @PsiACE in #2608
* docs(blog): add Apache OpenDAL(Incubating): Access Data Freely by @PsiACE in #2607
* docs(RFC): Object Versioning by @suyanhanx in #2602
* ci: Disable bindings/java deploy for now by @tisonkun in #2560
* ci: Disable the failed stage-release job instead by @tisonkun in #2561
* ci: add haddock generator for haskell binding by @silver-ymz in #2569
* ci(binding/lua): add luarocks package manager support by @oowl in #2558
* build(deps): bump predicates from 2.1.5 to 3.0.1 by @dependabot in #2583
* build(deps): bump tower-http from 0.4.0 to 0.4.1 by @dependabot in #2582
* build(deps): bump chrono from 0.4.24 to 0.4.26 by @dependabot in #2581
* build(deps): bump redis from 0.22.3 to 0.23.0 by @dependabot in #2580
* build(deps): bump cbindgen from 0.24.3 to 0.24.5 by @dependabot in #2579
* ci: upgrade hawkeye to v3 by @tisonkun in #2585
* ci(services/webdav): Setup integration test for nextcloud by @Xuanwo in #2631
* chore: add haskell binding link to website by @silver-ymz in #2571
* chore: fix cargo warning for resolver by @xxchan in #2590
* chore: bump log to 0.4.19 by @xxchan in #2591
* chore(deps): update deps to latest version by @suyanhanx in #2596
* chore: Add release 0.38.0 to download by @PsiACE in #2597
* chore(service/minitrace): automatically generate span name by @andylokandy in #2618

* @Prashanth-Chandra made their first contribution in #2573
* @andylokandy made their first contribution in #2618

**Full Changelog**: v0.38.0...v0.38.1

Signed-off-by: ClSlaid <[email protected]>

* refactor: bump npm version

Signed-off-by: ClSlaid <[email protected]>

* chore: include #2634

Signed-off-by: 蔡略 <[email protected]>

---------

Signed-off-by: ClSlaid <[email protected]>
Signed-off-by: 蔡略 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Xuanwo Xuanwo Xuanwo approved these changes

@PsiACE PsiACE Awaiting requested review from PsiACE PsiACE is a code owner

@ClSlaid ClSlaid Awaiting requested review from ClSlaid

@sundy-li sundy-li Awaiting requested review from sundy-li

Assignees
No one assigned
Labels
releases-note/feat
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@suyanhanx @Xuanwo