Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[storage][datalake] uploadFile does not abort when timeout #10316

Closed
ljian3377 opened this issue Jul 28, 2020 · 2 comments
Closed

[storage][datalake] uploadFile does not abort when timeout #10316

ljian3377 opened this issue Jul 28, 2020 · 2 comments
Assignees
Labels
Azure.Core Client This issue points to a problem in the data-plane of the library. Data Lake Storage Gen2

Comments

@ljian3377
Copy link
Member

it("uploadFile with chunkSize = FILE_UPLOAD_MAX_CHUNK_SIZE should succeed", async () => {

I expected the timeout aborter could abort the uploadFile operation. But it doesn't. The 4GB upload hasn't done yet after 6 minutes. We passed the abortSignal to underlying requests so what's going wrong? @XiaoningLiu @jeremymeng

abortSignal: options.abortSignal,

  it("uploadFile with chunkSize = FILE_UPLOAD_MAX_CHUNK_SIZE should succeed", async () => {
    recorder.skip("node", "Temp file - recorder doesn't support saving the file");
    const fileSize = FILE_UPLOAD_MAX_CHUNK_SIZE * 2 + MB;
    const tempFile = await createRandomLocalFile(tempFolderPath, fileSize / MB, MB);
    try {
      await fileClient.uploadFile(tempFile, {
        chunkSize: FILE_UPLOAD_MAX_CHUNK_SIZE,
        abortSignal: AbortController.timeout(20 * 1000) // takes too long to upload the file
      });
    } catch (err) {
      assert.equal(err.name, 'AbortError');
    }

    fs.unlinkSync(tempFile);
  }).timeout(timeoutForLargeFileUploadingTest);
azure:storage-file-datalake:info Request: {
  "streamResponseBody": false,
  "url": "https://jsv12.dfs.core.windows.net/filesystem159593998218302360/file159593998381509579?position=REDACTED&action=REDACTED",
  "method": "PATCH",
  "headers": {
    "_headersMap": {
      "content-length": "4194304000",
      "x-ms-version": "2019-12-12",
      "user-agent": "azsdk-js-storagedatalake/12.1.0 (NODE-VERSION v12.13.0; Windows_NT 10.0.19041)",
      "x-ms-client-request-id": "f3ad7dd1-2f1a-4a60-a064-06d0d0a6821d"
    }
  },
  "withCredentials": false,
  "timeout": 0,
  "keepAlive": true,
  "requestId": "f3ad7dd1-2f1a-4a60-a064-06d0d0a6821d"
}
azure:storage-file-datalake:info RetryPolicy: =====> Try=1 Primary
azure:storage-file-datalake:info Request: {
  "streamResponseBody": false,
  "url": "https://jsv12.dfs.core.windows.net/filesystem159593998218302360/file159593998381509579?position=REDACTED&action=REDACTED",
  "method": "PATCH",
  "headers": {
    "_headersMap": {
      "content-length": "1048576",
      "x-ms-version": "2019-12-12",
      "user-agent": "azsdk-js-storagedatalake/12.1.0 (NODE-VERSION v12.13.0; Windows_NT 10.0.19041)",
      "x-ms-client-request-id": "dac08a48-bb53-46f3-bb0a-e06cd04764b2"
    }
  },
  "withCredentials": false,
  "timeout": 0,
  "keepAlive": true,
  "requestId": "dac08a48-bb53-46f3-bb0a-e06cd04764b2"
}
azure:storage-file-datalake:info Response status code: 202
azure:storage-file-datalake:info Headers: {
  "_headersMap": {
    "content-length": "0",
    "date": "Tue, 28 Jul 2020 12:39:49 GMT",
    "server": "Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0",
    "x-ms-client-request-id": "dac08a48-bb53-46f3-bb0a-e06cd04764b2",
    "x-ms-request-id": "0cb03dee-501f-0071-14dc-6437a7000000",
    "x-ms-request-server-encrypted": "true",
    "x-ms-version": "2019-12-12"
  }
}
@ghost ghost added the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Jul 28, 2020
@ramya-rao-a ramya-rao-a added Client This issue points to a problem in the data-plane of the library. Data Lake Storage Gen2 labels Jul 28, 2020
@ghost ghost removed the needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. label Jul 28, 2020
@ramya-rao-a ramya-rao-a added this to the [2020] September milestone Jul 28, 2020
@ljian3377 ljian3377 modified the milestones: [2020] November, Backlog Oct 22, 2020
@jeremymeng
Copy link
Member

I think this is the same issue in #12029. We remove the abort event handler after receiving response, instead of waiting for streaming to complete.

@jeremymeng
Copy link
Member

timeout should trigger abortSignal now.

@xirzec xirzec removed this from the Backlog milestone May 17, 2022
@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Azure.Core Client This issue points to a problem in the data-plane of the library. Data Lake Storage Gen2
Projects
None yet
Development

No branches or pull requests

4 participants