Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Revert "test: reduce runtime" #20927

Closed
wants to merge 1 commit into from
Closed

Conversation

Trott
Copy link
Member

@Trott Trott commented May 24, 2018

This reverts commit 352ae23.

The commit appears to have introduced unreliability in one of the two refactored tests in AIX in CI (see #20688 (comment)) and macOS (see #20888).

Not looking to fast track this or anything (unless this becomes more of a problem for people). Just want it opened in case this is still an issue in 48 hours. The benefits of the change are small (and might be offset by unintentional slow-down on some platforms, see #20688 (comment)).

/cc @danbev @MylesBorins @BridgeAR @nodejs/testing

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes

@nodejs-github-bot nodejs-github-bot added the test Issues and PRs related to the tests. label May 24, 2018
Copy link
Contributor

@danbev danbev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverting does allow test-async-wrap-pop-id-during-load.js to pass for me.

@BridgeAR
Copy link
Member

@danbev would you be so kind and check what stack size allows the test to pass for you? I tested this locally and it crashed with 46 on my machine and passed with 50. Seems like aix and others need more stack size. A fix should very likely also be to set the number to 75 (and that would still mean a much faster failing test than before).

@danbev
Copy link
Contributor

danbev commented May 24, 2018

would you be so kind and check what stack size allows the test to pass for you?

A stack size of 75 passes for me.

@gireeshpunathil
Copy link
Member

do we really need to revert? we understood and ratified the need for #20688 and we now also know the reason for the failure. Isn't it a matter of fixing it?

@BridgeAR BridgeAR mentioned this pull request May 24, 2018
4 tasks
@Trott
Copy link
Member Author

Trott commented May 24, 2018

do we really need to revert? we understood and ratified the need for #20688 and we now also know the reason for the failure. Isn't it a matter of fixing it?

@gireeshpunathil I think your questions/comments are addressed in the PR description:

Not looking to fast track this or anything (unless this becomes more of a problem for people). Just want it opened in case this is still an issue in 48 hours. The benefits of the change are small (and might be offset by unintentional slow-down on some platforms, see #20688 (comment)).

@Trott
Copy link
Member Author

Trott commented May 24, 2018

@refack
Copy link
Contributor

refack commented May 24, 2018

@Trott, as a possible future S.O.P. why not add test to flaky list instead of reverting? I kinda like the ⚠️ it's more informative than the 🔴 as it answers the user's question of "is this a real fail or a flake" while keeping the issue visible.

@Trott
Copy link
Member Author

Trott commented May 24, 2018

@Trott, as a possible future S.O.P. why not add test to flaky list instead of reverting? I kinda like the ⚠️ it's more informative than the 🔴 as it answers the user's question of "is this a real fail or a flake" while keeping the issue visible.

@refack Excellent and totally valid question. I have mixed feelings about marking things as flaky because I feel like it gives everyone a pass on ignoring the problems with the test. It furthers the idea that test failures are someone else's problem and I feel like the only way the project can keep scaling is if CI failures are everyone's problem. I especially dislike it for things that fail more than once in a very long while.

That said, that's all my opinion only and I'm not sure that opinion is widely shared.

@Trott
Copy link
Member Author

Trott commented May 28, 2018

#20940 landed, so this is (hopefully) not needed.

@Trott Trott closed this May 28, 2018
@Trott Trott deleted the rebert-er-revert branch January 13, 2022 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants