Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: fix flaky test-http2-session-timeout #20692

Closed
wants to merge 2 commits into from

Conversation

Trott
Copy link
Member

@Trott Trott commented May 13, 2018

Check actual expired time rather than relying on a number of calls to setTimeout() in test-http2-session-timeout more robust.

Fixes: #20628

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added the test Issues and PRs related to the tests. label May 13, 2018
@Trott Trott added the wip Issues and PRs that are still a work in progress. label May 13, 2018
@Trott Trott changed the title test: fix flaky test-http2-session-timeout [WIP] test: fix flaky test-http2-session-timeout May 13, 2018
@Trott Trott changed the title [WIP] test: fix flaky test-http2-session-timeout test: fix flaky test-http2-session-timeout May 13, 2018
@Trott Trott removed the wip Issues and PRs that are still a work in progress. label May 13, 2018
@Trott
Copy link
Member Author

Trott commented May 13, 2018

This version of the test is demonstrably more reliable than the current version when tested this way:

  • Copy the current version of the test to test/parallel.
  • Run the test with tools/test.py -j 16 --repeat 192 tools/test/parallal/test-<WHATEVER_YOU_COPIED_IT_TO>
  • Repeat with the new version of the test (copied to test/parallel).

You may need to adjust the value of the -j option to be not too large that both versions fail so often that it's hard to tell the difference and to not be so small that both versions fail so infrequently as to not make a different. 16 was a good value on my laptop. Here are the last three runs of each version of the test with that value (successes/failures):

Current version of test: 142/50, 159/33, 153/39
This PR's version of the test: 184/8, 191/1, 186/6

/cc @apapirovski

@Trott
Copy link
Member Author

Trott commented May 13, 2018

@@ -34,12 +34,14 @@ server.listen(0, common.mustCall(() => {
request.end();

request.on('end', () => {
if (attempts) {
setTimeout(() => makeReq(attempts - 1), callTimeout);
const diff = process.hrtime(startTime);
Copy link
Member

@apapirovski apapirovski May 13, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm -1 on this change. process.hrtime isn't cheap so it's more — not less — likely to create situations where this test fails on slow platforms.

Actually, scratch that, I see what's making this more reliable.

const diff = process.hrtime(startTime);
const milliseconds = (diff[0] * 1e3 + diff[1] / 1e6);
if (milliseconds < serverTimeout * 2) {
setTimeout(() => makeReq(), callTimeout);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: if I'm not wrong we can pass makeReq directly without wrapping it in another function.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Changed. Thanks!

@lpinca lpinca added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 13, 2018
@richardlau
Copy link
Member

#20692 (comment) suggests this doesn't fix the test but instead makes it less likely to fail, i.e., it's still flaky? If so can we reword the commit to something other than fix (maybe improve or make less flaky)?

Trott added 2 commits May 14, 2018 16:09
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

Fixes: nodejs#20628
@Trott
Copy link
Member Author

Trott commented May 14, 2018

@richardlau I went with "improve reliability" although I had to abbreviate the test name to keep it under 50 chars. (I removed test- which is implied by the test: for the subsystem. And the full name of the test appears in the main text.)

@Trott
Copy link
Member Author

Trott commented May 15, 2018

@apapirovski
Copy link
Member

Landed in 9e4ae56

apapirovski pushed a commit that referenced this pull request May 16, 2018
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

PR-URL: #20692
Fixes: #20628
Reviewed-By: Anatoli Papirovski <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: James M Snell <[email protected]>
MylesBorins pushed a commit that referenced this pull request May 22, 2018
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

PR-URL: #20692
Fixes: #20628
Reviewed-By: Anatoli Papirovski <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@addaleax addaleax mentioned this pull request May 22, 2018
kjin pushed a commit to kjin/node that referenced this pull request Aug 23, 2018
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

PR-URL: nodejs#20692
Fixes: nodejs#20628
Reviewed-By: Anatoli Papirovski <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: James M Snell <[email protected]>
kjin pushed a commit to kjin/node that referenced this pull request Sep 19, 2018
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

PR-URL: nodejs#20692
Fixes: nodejs#20628
Reviewed-By: Anatoli Papirovski <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: James M Snell <[email protected]>
kjin pushed a commit to kjin/node that referenced this pull request Oct 16, 2018
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

PR-URL: nodejs#20692
Fixes: nodejs#20628
Reviewed-By: Anatoli Papirovski <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: James M Snell <[email protected]>
BethGriggs pushed a commit that referenced this pull request Oct 17, 2018
Check actual expired time rather than relying on a number of calls to
setTimeout() in test-http2-session-timeout more robust.

Backport-PR-URL: #22850
PR-URL: #20692
Fixes: #20628
Reviewed-By: Anatoli Papirovski <[email protected]>
Reviewed-By: Trivikram Kamat <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@BethGriggs BethGriggs mentioned this pull request Oct 30, 2018
@Trott Trott deleted the one-less-timer branch January 13, 2022 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate flaky test-http2-session-timeout
8 participants