Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: Implement stat dir correctly based on RFC-3243 List Prefix #3651

Merged
merged 53 commits into from
Nov 23, 2023
Merged

feat: Implement stat dir correctly based on RFC-3243 List Prefix #3651

merged 53 commits into from
Nov 23, 2023

Conversation

Xuanwo
Copy link
Member

@Xuanwo Xuanwo commented Nov 22, 2023 •
edited
Loading

Part of #3247

This PR changed the behavior of stat as described in RFC-3243 List Prefix. In short:

  • We will return file not found if we are trying to stat file abc with abc/.
  • We will handle dir correctly by calling list with prefix.

This change will make sure we return correct dir metadata on s3 alike storage services.

I will update upgrade.md in following PRs since this PR is so large that any single change will trigger all CI to run.

@Xuanwo
polish stat docs
fc125ff 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Add behavior docs
bb1a6ac 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Add more test cases for stat
46b9040 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix kv adapter
fadf4e6 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix kv on new test cases
e8181fd 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Implement for s3
744c5cf 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix fs
435e548 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix azblob
501b718 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix azdls
592161b 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Merge remote-tracking branch 'origin/main' into fix-stat
7441aaa
@Xuanwo
Polish azfile
bc903fc 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix cos
b66c7cd 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix dropbox
238b081 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix ftp
8b3405f 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix gcs
eb953e6 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix gdrive
fb9c92f 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix ghac
6e778e5 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix hdfs
e9e9c7f 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix obs
cfdd0ea 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
FIx oss
e39939b 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
FIx test
44f9447 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Fix swift
e746e74 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
FIx webhdfs
c0af347 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo Xuanwo mentioned this pull request Nov 22, 2023
Closed
1 task
@Xuanwo Xuanwo marked this pull request as draft November 22, 2023 12:10
@Xuanwo
Fix ghac
697449c 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Address at complete layer side
3fc595e 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Address root
1af8ead 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo
Address alluxio
c7a26a6 
Signed-off-by: Xuanwo <[email protected]>
@Xuanwo Xuanwo marked this pull request as ready for review November 23, 2023 05:41
@Xuanwo
Copy link
Member Author

Xuanwo commented Nov 23, 2023

dropbox failed for too_many_write_operations.

suyanhanx
suyanhanx reviewed Nov 23, 2023
View reviewed changes
Copy link
Member

@suyanhanx suyanhanx left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This must be a breaking change. Please update the upgrade guide.

core/src/raw/oio/list/api.rs Show resolved Hide resolved
core/tests/behavior/write.rs Show resolved Hide resolved
core/src/types/operator/operator.rs Show resolved Hide resolved
@Xuanwo Xuanwo merged commit 0711e6c into main Nov 23, 2023
216 of 220 checks passed
@Xuanwo Xuanwo deleted the fix-stat branch November 23, 2023 08:27
@Xuanwo Xuanwo mentioned this pull request Nov 23, 2023
Closed
@CookiePieWw CookiePieWw mentioned this pull request Nov 23, 2023
Closed
@G-XD G-XD mentioned this pull request Nov 27, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@suyanhanx suyanhanx suyanhanx left review comments

@PsiACE PsiACE PsiACE approved these changes

@G-XD G-XD Awaiting requested review from G-XD

@ClSlaid ClSlaid Awaiting requested review from ClSlaid

@messense messense Awaiting requested review from messense messense is a code owner

@Zheaoli Zheaoli Awaiting requested review from Zheaoli Zheaoli is a code owner

@tisonkun tisonkun Awaiting requested review from tisonkun tisonkun is a code owner

Assignees
No one assigned
Labels
releases-note/feat
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Invalid metadata for dir objects in s3 is_exist always return true for key end with '/', in S3 service
3 participants
@Xuanwo @suyanhanx @PsiACE