Skip to content
This repository has been archived by the owner on Jul 24, 2024. It is now read-only.

storage: fix s3 storage walk-dir with empty sub-dir #713

Merged
merged 5 commits into from
Feb 3, 2021
Merged

storage: fix s3 storage walk-dir with empty sub-dir #713

merged 5 commits into from
Feb 3, 2021

Conversation

glorv
Copy link
Collaborator

@glorv glorv commented Jan 25, 2021

What problem does this PR solve?

Close #712

What is changed and how it works?

The s3 implement ListObjects will return no data if set the prefix to / instead of leave it empty.

NOTE: the current implementation is compatible with minio but no s3.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Code changes

Side effects

Related changes

  • Need to cherry-pick to the release branch

Release Note

  • Fix the bug that WalkDir for s3 storage returns nil if the target path is bucket name.

@glorv
Copy link
Collaborator Author

glorv commented Jan 25, 2021

@kennytm @overvenus PTAL how to add a unit/integration test for this kind of compatible issues.

@glorv
Copy link
Collaborator Author

glorv commented Jan 25, 2021

/run-all-tests

@glorv glorv added this to the v4.0.11 milestone Jan 25, 2021
@glorv glorv removed this from the v4.0.11 milestone Jan 29, 2021
@glorv glorv changed the title WIP: storage: fix s3 storage walk-dir with empty sub-dir storage: fix s3 storage walk-dir with empty sub-dir Jan 29, 2021
@glorv glorv removed the status/WIP label Jan 29, 2021
kennytm
kennytm reviewed Jan 31, 2021
View reviewed changes
Copy link
Collaborator

@kennytm kennytm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-srebot ti-srebot added the status/LGT1 LGTM1 label Jan 31, 2021
3pointer
3pointer reviewed Feb 3, 2021
View reviewed changes
Copy link
Collaborator

@3pointer 3pointer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@ti-srebot ti-srebot removed the status/LGT1 LGTM1 label Feb 3, 2021
@ti-srebot ti-srebot added the status/LGT2 LGTM2 label Feb 3, 2021
@3pointer 3pointer merged commit 605037c into pingcap:master Feb 3, 2021
ti-srebot pushed a commit to ti-srebot/br that referenced this pull request Feb 3, 2021
@glorv @ti-srebot
cherry pick pingcap#713 to release-4.0
ccdc8df 
Signed-off-by: ti-srebot <[email protected]>
@ti-srebot ti-srebot mentioned this pull request Feb 3, 2021
Merged
@ti-srebot
Copy link
Contributor

cherry pick to release-4.0 in PR #733

kennytm pushed a commit that referenced this pull request Feb 3, 2021
@ti-srebot @glorv
cherry pick #713 to release-4.0 (#733)
910cd73 
Signed-off-by: ti-srebot <[email protected]>

Co-authored-by: glorv <[email protected]>
@glorv glorv mentioned this pull request Feb 24, 2021
Merged
@ti-srebot ti-srebot mentioned this pull request Feb 25, 2021
Merged
@glorv glorv deleted the fix-s3 branch April 8, 2021 04:02
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@kennytm kennytm kennytm left review comments

@3pointer 3pointer 3pointer left review comments

@ti-srebot ti-srebot ti-srebot approved these changes

Assignees
No one assigned
Labels
component/external-storage needs-cherry-pick-release-4.0 status/LGT2 LGTM2 type/bug-fix
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

WalkDir return empty results when subdir is empty
5 participants
@glorv @ti-srebot @kennytm @3pointer @overvenus