Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: refactor test_runner tests to change default reporter #54547

Merged
merged 2 commits into from
Aug 27, 2024

Conversation

cjihrig
Copy link
Contributor

@cjihrig cjihrig commented Aug 24, 2024

This commit updates the test runner tests in order to switch the default reporter from tap to spec. This commit can be backported, while changing the default reporter cannot.

This will also allow us to more slowly migrate the tests off of using the TAP reporter in some places. We should still test the TAP reporter though. In some cases the TAP reporter is beneficial for testing purposes - for example, verifying that tests are output with the correct test number ordering.

Refs: #54540

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Aug 24, 2024
@RedYetiDev
Copy link
Member

Note: this PR is part of #54548

@RedYetiDev RedYetiDev added the test_runner Issues and PRs related to the test runner subsystem. label Aug 24, 2024
Copy link

codecov bot commented Aug 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.32%. Comparing base (05bd3cf) to head (d14e5ca).
Report is 321 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #54547      +/-   ##
==========================================
- Coverage   87.33%   87.32%   -0.01%     
==========================================
  Files         649      649              
  Lines      182622   182623       +1     
  Branches    35037    35044       +7     
==========================================
- Hits       159494   159483      -11     
- Misses      16393    16401       +8     
- Partials     6735     6739       +4     

see 32 files with indirect coverage changes

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@cjihrig cjihrig added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 25, 2024
@cjihrig
Copy link
Contributor Author

cjihrig commented Aug 25, 2024

test/sequential/test-net-server-listen-ipv6-link-local.js appears to be failing in GitHub Actions. I don't think it is related to this change though because it's also failing in https://github.com/nodejs/node/actions/runs/10547441912/job/29220162937 (for #54190).

@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 25, 2024
@nodejs-github-bot
Copy link
Collaborator

@RedYetiDev
Copy link
Member

test/sequential/test-net-server-listen-ipv6-link-local.js appears to be failing in GitHub Actions. I don't think it is related to this change though because it's also failing in nodejs/node/actions/runs/10547441912/job/29220162937 (for #54190).

It's also failing in a few other places, like https://ci.nodejs.org/job/node-test-binary-windows-js-suites/29545/testReport/

@jakecastelli
Copy link
Member

The failure is caused by #54470 (just landed) breaks the test added in an earlier landed PR #54264, I tried to revert 54470 by #54554 🙏

@jakecastelli
Copy link
Member

Issue has been resolved, would you mind rebasing 🙏 I will try to follow up with the CI

This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: nodejs#54540
@cjihrig cjihrig added the request-ci Add this label to start a Jenkins CI on a PR. label Aug 26, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 26, 2024
@nodejs-github-bot
Copy link
Collaborator

@cjihrig cjihrig added request-ci Add this label to start a Jenkins CI on a PR. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. labels Aug 26, 2024
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 26, 2024
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

nodejs-github-bot commented Aug 26, 2024

@jakecastelli jakecastelli added the commit-queue Add this label to land a pull request using GitHub Actions. label Aug 27, 2024
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Aug 27, 2024
@nodejs-github-bot nodejs-github-bot merged commit 3d5357a into nodejs:main Aug 27, 2024
58 checks passed
@nodejs-github-bot
Copy link
Collaborator

Landed in 3d5357a

@cjihrig cjihrig deleted the reporter-tests branch August 27, 2024 01:18
RafaelGSS pushed a commit that referenced this pull request Aug 30, 2024
This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: #54540
PR-URL: #54547
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Jake Yuesong Li <[email protected]>
RafaelGSS pushed a commit that referenced this pull request Aug 30, 2024
This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: #54540
PR-URL: #54547
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Jake Yuesong Li <[email protected]>
@RafaelGSS RafaelGSS mentioned this pull request Aug 30, 2024
@targos targos added the dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. label Sep 22, 2024
louwers pushed a commit to louwers/node that referenced this pull request Nov 2, 2024
This commit updates the test runner tests in order to switch the
default reporter from tap to spec. This commit can be backported,
while changing the default reporter cannot.

Refs: nodejs#54540
PR-URL: nodejs#54547
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Jake Yuesong Li <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants