Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resume from the breakpoint when upload failed with TransferUtilityMultiPartUpload #5455

Open
zhongruiBlink opened this issue Oct 26, 2024 · 4 comments
Labels
question General question storage

Comments

@zhongruiBlink
Copy link

Hi, I am using TransferUtilityMultiPartUpload to upload large video.

And my question is if there have any way, to resume upload from the breakpoint when the upload faild.

I try use the resume(), but I find the callback will show the task is failed imdientily.

@github-actions github-actions bot added pending-triage Issue is pending triage pending-maintainer-response Issue is pending response from an Amplify team member labels Oct 26, 2024
@edisooon edisooon added the question General question label Oct 27, 2024
@edisooon edisooon assigned edisooon and unassigned edisooon Oct 27, 2024
@edisooon edisooon added storage and removed pending-triage Issue is pending triage labels Oct 27, 2024
@edisooon
Copy link
Member

Hi @zhongruiBlink Thanks for submitting the issue! One of our team members will investigate and provide updates here.

@github-actions github-actions bot removed the pending-maintainer-response Issue is pending response from an Amplify team member label Oct 27, 2024
@ruisebas
Copy link
Member

Hi @zhongruiBlink! The resume method is meant to be used to restart an upload task that has been paused by explicitly calling the pause method. You cannot call it on failed tasks.

When doing a multipart upload, the TransferUtility attempts to retry failing parts up to the limit configured in TransferUtilityConfiguration, which defaults to 3.

@zhongruiBlink
Copy link
Author

Hi @ruisebas! Thanks for the information.

We're looking for a way to address this issue:

When we start an upload task with 100 chunks, if the upload fails after 50 chunks have been successfully uploaded, we currently have to restart from the first chunk. This means re-uploading those 50 chunks, which takes a lot of time. Is there any way to resume the upload starting from chunk 51?

@github-actions github-actions bot added the pending-maintainer-response Issue is pending response from an Amplify team member label Oct 29, 2024
@ruisebas
Copy link
Member

There is no way to resume an upload from a specific part if the upload task failed.
On failure, the transfer utility calls AbortMultipartUpload so that the server cleans up the temporary parts as well.

If you experience interruptions recurrently, you could try adjusting the AWSS3TransferUtilityConfiguration.retryLimit to a higher value.

@github-actions github-actions bot removed the pending-maintainer-response Issue is pending response from an Amplify team member label Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question General question storage
Projects
None yet
Development

No branches or pull requests

3 participants