Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure datasets can be downloaded from S3 #137

Closed
csala opened this issue May 8, 2019 · 0 comments · Fixed by #140
Closed

Ensure datasets can be downloaded from S3 #137

csala opened this issue May 8, 2019 · 0 comments · Fixed by #140
Assignees
Milestone

Comments

@csala
Copy link
Contributor

csala commented May 8, 2019

In the past ATM was able download datasets stored in S3, but at some point the functionality seems to have been broken and it's not working any more.

We should review it and make the necessary changes to have it working again.
We should also replace boto dependency with boto3, which is able to use of aws credentials from the user folder without him having to explicitly specify them.

We should, both for S3 and HTTP, we should download the data to a place relative to the current directory instead of inside the ATM code folders.

Finally, we should reorganize the code in a way that is easily extensible in the future if we want to add support for other data sources (FTP, SFTP, HFS...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants