Byte range support in S3 Transfer Manager #1215
Labels
automation-exempt
feature-request
This issue requests a feature.
needs-review
p3
This is a minor priority issue
Hi All!
I've been using the S3 Transfer Manager (http://boto3.readthedocs.io/en/latest/reference/customizations/s3.html) for quite some time and I'm really happy with the increased performance it gives when downloading/uploading large objects. The built in retry mechanism is also very helpful.
I just started a new project where I'll work with really large objects that I want to process in parts. The idea is to use byte ranges to fetch parts of an object using byte ranges, process it and then fetch the next part. These parts can be as large as several 100MB so multipart download would speed things up. I'm aware that byte range requests can be used with http://boto3.readthedocs.io/en/latest/reference/services/s3.html#S3.Client.get_object, but then I'd have to implement multipart and retry mechanisms myself (if I've understood things correctly).
Is there a way to use byte ranges with the S3 Transfer Manager that I haven't figured out yet? If not, are there any plans to support it in the future?
Thanks
Stefan
The text was updated successfully, but these errors were encountered: