-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
s3: support for directories #2619
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also, there is no tests here.
tests/unit/remote/test_s3.py
Outdated
|
||
|
||
def test_isdir(s3, remote): | ||
assert remote.isdir(remote.path_info / "data") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you could totally use parametrize
here instead of copypasting. 🙂
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes! Forgot about parametrize
👌
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Interesting, parametrize
is 1 and a half second slower 😓
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@MrOutis this is because all the fixtures are rebuilt every time. Use for loop inside single test instead. Moreover, I would reuse the same list for fixture and tests as far as that is possible.
@MrOutis Tests are still failing on py2 due to unicode error. Just a heads up 🙂 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks!
Fix: #1654
Setup:
Notes:
head_object
doesn't work with directories, only empty oneslist_objects
doesn't work with empty directoriesaws s3 cp
doesn't work with directories, you need to use--recursive
dvc run -d remote://
doesn't look up if cache is setup (as-o
), but directories uses cacheTODO:
walk
implementation towalk_files