-
Notifications
You must be signed in to change notification settings - Fork 24.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Increase chunk_size max to 256 Mb instead of 64 Mb #23489
Conversation
Increase chunk_size max to 256 Mb instead of 64 Mb
Isue=23446 Increase Azure max chunk size.
Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually? |
This is my first time doing a public repo pr, please let me know if I need to correct anything. |
@rjernst If you want to send to Azure repo a 1Gb file, with the current default, it will require 16 HTTP requests. With that change we can do it with 4 HTTP requests. |
Reading the documentation, it appears that 256mb would be the optimal chunk size corresponding to an Azure blob block. We haven't tested it so its an unknown, but it makes sense to go with the documented recommendation from Azure. I do think we should run tests on this against an actual Azure storage account to see if it makes a difference. |
If it is the recommended then +1 to change (with testing as @abeyad mentions). One thing to double check with that testing is how it performs with default settings (ie from a laptop as a new user would try it out). |
superseded by #32101 |
Increasing the max size of the chunk from 64mb to 256mb.
Closes #23446.