Skip to content

Commit

Permalink
[CHORE] Update default num_tries on S3Config to 25 (#1487)
Browse files Browse the repository at this point in the history
We were seeing some issues with timeouts and retries

Co-authored-by: Jay Chia <[email protected]@users.noreply.github.com>
  • Loading branch information
jaychia and Jay Chia authored Oct 12, 2023
1 parent 8ad7fda commit 84fcc7f
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion src/common/io-config/src/s3.rs
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,9 @@ impl Default for S3Config {
retry_initial_backoff_ms: 1000,
connect_timeout_ms: 10_000,
read_timeout_ms: 10_000,
num_tries: 5,
// AWS EMR actually does 100 tries by default for AIMD retries
// (See [Advanced AIMD retry settings]: https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark-emrfs-retry.html)
num_tries: 25,
retry_mode: Some("adaptive".to_string()),
anonymous: false,
verify_ssl: true,
Expand Down

0 comments on commit 84fcc7f

Please sign in to comment.