Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ADLS Gen 2 Append and Lease Operations #31263

Conversation

ibrahimrabab
Copy link
Contributor

@ibrahimrabab ibrahimrabab commented Oct 4, 2022

Description

This PR aims to:

  1. Acquisition & renewal of lease will be available with append API from HDFS endpoint.

  2. Release of lease with flush on append.

  3. Acquire-Release Mode introduced in append for small file write scenarios. Auto-renew, Acquire & Acquire-Release Mode will be introduced for flush so that intermittent/slow flushes succeed.

More information on this feature can be found in the following spec

@ghost ghost added the Storage Storage Service (Queues, Blobs, Files) label Oct 4, 2022
@azure-sdk
Copy link
Collaborator

azure-sdk commented Oct 4, 2022

API change check

APIView has identified API level changes in this PR and created following API reviews.

azure-storage-file-datalake
azure-storage-file-share

/**
* @return the lease duration in seconds.
*/
public Integer getLeaseDuration() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd rather see Duration used here to allow this to be more flexible on usage and avoid time as a number ambiguity (is it milliseconds, is it seconds, is it something else)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears we are using int for lease duration in the DataLakeLeaseClient -

. I think we should be internally consistent throughout the SDK.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm good with using Integer if we want to maintain internal consistency. In the future if we get feedback about it another overload using Duration could always be added, similar to what was done with RequestRetryOptions (#17628).

/**
* @return the lease duration in seconds.
*/
public Integer getLeaseDuration() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It appears we are using int for lease duration in the DataLakeLeaseClient -

. I think we should be internally consistent throughout the SDK.

@ibrahimrabab ibrahimrabab merged commit 2622ec9 into Azure:feature/storage/stg85base Oct 18, 2022
@ibrahimrabab ibrahimrabab deleted the stg85base/appendLease branch October 18, 2022 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants