You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This task is to replace the blocking S3Client we use in all cases it makes sense to replace it with its non blocking version, i.e. in {RDD | Collection}{Readers | Writers}.
To make migration soft we may want to keep both clients (Sync and Async) available. i.e. it makes more sense for AttributeStore to use the blocking S3Client for now.
Leaving below the S3ClientProducer to support the idea.
objectS3ClientProducer {
@transient privatelazyvaloverrideConfig:ClientOverrideConfiguration= {
valretryCondition=OrRetryCondition.create(
RetryCondition.defaultRetryCondition(),
RetryOnErrorCodeCondition.create("RequestTimeout")
)
valbackoffStrategy=FullJitterBackoffStrategy.builder()
.baseDelay(Duration.ofMillis(50))
.maxBackoffTime(Duration.ofMillis(15))
.build()
valretryPolicy=RetryPolicy.defaultRetryPolicy()
.toBuilder
.retryCondition(retryCondition)
.backoffStrategy(backoffStrategy)
.build()
ClientOverrideConfiguration.builder()
.retryPolicy(retryPolicy)
.build()
}
@transient privatelazyvalclient:S3Client=S3Client.builder()
.overrideConfiguration(overrideConfig)
.build()
@transient privatelazyvalasyncClient:S3AsyncClient=S3AsyncClient.builder()
.overrideConfiguration(overrideConfig)
.build()
privatevarsummonClient: () =>S3Client= () => client
privatevarsummonAsyncClient: () =>S3AsyncClient= () => asyncClient
/** Set an alternative default function for summoning S3Clients */defset(getClient: () =>S3Client):Unit= summonClient = getClient
defsetAsync(getClient: () =>S3AsyncClient):Unit= summonAsyncClient = getClient
/** Get the current function registered as default for summong S3Clients */defget: () =>S3Client= summonClient
defgetAsync: () =>S3AsyncClient= summonAsyncClient
}
This task is to replace the blocking
S3Client
we use in all cases it makes sense to replace it with its non blocking version, i.e. in{RDD | Collection}{Readers | Writers}
.To make migration soft we may want to keep both clients (
Sync
andAsync
) available. i.e. it makes more sense forAttributeStore
to use the blockingS3Client
for now.Leaving below the
S3ClientProducer
to support the idea.Connects #2923, #2306
The text was updated successfully, but these errors were encountered: