Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Investigate flaky test - parallel/test-fs-rmdir-recursive #41201

Closed
mhdawson opened this issue Dec 16, 2021 · 3 comments
Closed

Investigate flaky test - parallel/test-fs-rmdir-recursive #41201

mhdawson opened this issue Dec 16, 2021 · 3 comments
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI. fs Issues and PRs related to the fs subsystem / file system.

Comments

@mhdawson
Copy link
Member

Test

parallel/test-fs-rmdir-recursive

Platform

Windows, arm/linux

Console output

Can't clean tmpdir: C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.236
Files blocking: [ 'rmdir-recursive-0', 'rmdir-recursive-4', 'rmdir-recursive-5' ]

assert.js:982
    throw newErr;
    ^

AssertionError [ERR_ASSERTION]: ifError got unwanted exception: ENOTEMPTY: directory not empty, rmdir 'C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.236\rmdir-recursive-0\d-3-2\d-2-2\d-1-2'
    at C:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:338:12
    at C:\workspace\node-test-binary-windows-js-suites\node\test\common\index.js:375:15
    at fs.js:179:23
    at CB (internal/fs/rimraf.js:59:5)
    at internal/fs/rimraf.js:151:18
    at CB (internal/fs/rimraf.js:59:5)
    at internal/fs/rimraf.js:151:18
    at CB (internal/fs/rimraf.js:59:5)
    at internal/fs/rimraf.js:151:18
    at CB (internal/fs/rimraf.js:59:5)
 {
  generatedMessage: false,
  code: 'ERR_ASSERTION',
  actual: [Error: ENOTEMPTY: directory not empty, rmdir 'C:\workspace\node-test-binary-windows-js-suites\node\test\.tmp.236\rmdir-recursive-0\d-3-2\d-2-2\d-1-2'] {
    errno: -4051,
    code: 'ENOTEMPTY',
    syscall: 'rmdir',
    path: 'C:\\workspace\\node-test-binary-windows-js-suites\\node\\test\\.tmp.236\\rmdir-recursive-0\\d-3-2\\d-2-2\\d-1-2'
  },
  expected: null,
  operator: 'ifError'
}

Build links

From latest reliability report

9 (#41083, #41022, #40980, #40899, #36696, #41046, #41146, #41149, #41173)

Additional information

There are also 5 instances of parallel/test-fs-rm failing in the reliability report. Often but not always in the same run as test-fs-rmdir-recursive. Maybe the tests have similar timing issues?

@mhdawson mhdawson added the flaky-test Issues and PRs related to the tests with unstable failures on the CI. label Dec 16, 2021
@Mesteery Mesteery added the fs Issues and PRs related to the fs subsystem / file system. label Dec 16, 2021
mhdawson added a commit to mhdawson/io.js that referenced this issue Jan 14, 2022
Refs: nodejs#41201

From recent reliability reports this is now the most
common failure by far in CI runs. Mark the test as
flaky until the issue is resolved.

Signed-off-by: Michael Dawson <[email protected]>
bcoe added a commit to bcoe/node-1 that referenced this issue Jan 15, 2022
Noticed that a few assertions were not being awaited, this could
potentially be leading to flakiness in tmp cleanup.

Refs: nodejs#41201
bcoe added a commit to bcoe/node-1 that referenced this issue Jan 16, 2022
Noticed that a few assertions were not being awaited, this could
potentially be leading to flakiness in tmp cleanup.

Refs: nodejs#41201
mhdawson pushed a commit that referenced this issue Jan 17, 2022
Noticed that a few assertions were not being awaited, this could
potentially be leading to flakiness in tmp cleanup.

Refs: #41201

PR-URL: #41545
Refs: #41201
Reviewed-By: Ian Sutherland <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Mestery <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Darshan Sen <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
thedull pushed a commit to thedull/node that referenced this issue Jan 18, 2022
Noticed that a few assertions were not being awaited, this could
potentially be leading to flakiness in tmp cleanup.

Refs: nodejs#41201

PR-URL: nodejs#41545
Refs: nodejs#41201
Reviewed-By: Ian Sutherland <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Mestery <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Darshan Sen <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
BethGriggs pushed a commit that referenced this issue Jan 25, 2022
Noticed that a few assertions were not being awaited, this could
potentially be leading to flakiness in tmp cleanup.

Refs: #41201

PR-URL: #41545
Refs: #41201
Reviewed-By: Ian Sutherland <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Mestery <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Darshan Sen <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Linkgoron pushed a commit to Linkgoron/node that referenced this issue Jan 31, 2022
Noticed that a few assertions were not being awaited, this could
potentially be leading to flakiness in tmp cleanup.

Refs: nodejs#41201

PR-URL: nodejs#41545
Refs: nodejs#41201
Reviewed-By: Ian Sutherland <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Mestery <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Darshan Sen <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
mhdawson added a commit that referenced this issue Jan 31, 2022
Refs: #41201

From recent reliability reports this is now the most
common failure by far in CI runs. Mark the test as
flaky until the issue is resolved.

Signed-off-by: Michael Dawson <[email protected]>

PR-URL: #41533
Reviewed-By: Ben Coe <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
ruyadorno pushed a commit that referenced this issue Feb 8, 2022
Refs: #41201

From recent reliability reports this is now the most
common failure by far in CI runs. Mark the test as
flaky until the issue is resolved.

Signed-off-by: Michael Dawson <[email protected]>

PR-URL: #41533
Reviewed-By: Ben Coe <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
@iansu
Copy link
Contributor

iansu commented Feb 18, 2022

@mhdawson did the fix @bcoe made in #41545 solve this problem?

@bcoe
Copy link
Contributor

bcoe commented Feb 21, 2022

@iansu mentioned in our tooling meeting, I think it might be worth serializing the test-fs-rmdir-recursive cb tests, as it might make it easier to determine where exactly the failure is happening ... I also have a hunch this might make the problem go away, if the underlying problem is two many fs operations happening in parallel on Windows.

danielleadams pushed a commit that referenced this issue Feb 26, 2022
Noticed that a few assertions were not being awaited, this could
potentially be leading to flakiness in tmp cleanup.

Refs: #41201

PR-URL: #41545
Refs: #41201
Reviewed-By: Ian Sutherland <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Mestery <[email protected]>
Reviewed-By: Mohammed Keyvanzadeh <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Rich Trott <[email protected]>
Reviewed-By: Darshan Sen <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
danielleadams pushed a commit that referenced this issue Mar 2, 2022
Refs: #41201

From recent reliability reports this is now the most
common failure by far in CI runs. Mark the test as
flaky until the issue is resolved.

Signed-off-by: Michael Dawson <[email protected]>

PR-URL: #41533
Reviewed-By: Ben Coe <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
danielleadams pushed a commit that referenced this issue Mar 3, 2022
Refs: #41201

From recent reliability reports this is now the most
common failure by far in CI runs. Mark the test as
flaky until the issue is resolved.

Signed-off-by: Michael Dawson <[email protected]>

PR-URL: #41533
Reviewed-By: Ben Coe <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
danielleadams pushed a commit that referenced this issue Mar 14, 2022
Refs: #41201

From recent reliability reports this is now the most
common failure by far in CI runs. Mark the test as
flaky until the issue is resolved.

Signed-off-by: Michael Dawson <[email protected]>

PR-URL: #41533
Reviewed-By: Ben Coe <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Luigi Pinca <[email protected]>
@marco-ippolito
Copy link
Member

lpinca added a commit to lpinca/node that referenced this issue Jan 25, 2024
nodejs-github-bot pushed a commit that referenced this issue Feb 1, 2024
Refs: #41201 (comment)
PR-URL: #51566
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
rdw-msft pushed a commit to rdw-msft/node that referenced this issue Feb 9, 2024
Refs: nodejs#41201 (comment)
PR-URL: nodejs#51566
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
targos pushed a commit that referenced this issue Feb 15, 2024
Refs: #41201 (comment)
PR-URL: #51566
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
marco-ippolito pushed a commit to marco-ippolito/node that referenced this issue Feb 19, 2024
Refs: nodejs#41201 (comment)
PR-URL: nodejs#51566
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
richardlau pushed a commit that referenced this issue Mar 25, 2024
Refs: #41201 (comment)
PR-URL: #51566
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
richardlau pushed a commit that referenced this issue Mar 25, 2024
Refs: #41201 (comment)
PR-URL: #51566
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
flaky-test Issues and PRs related to the tests with unstable failures on the CI. fs Issues and PRs related to the fs subsystem / file system.
Projects
None yet
Development

No branches or pull requests

5 participants