Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test_runner: emit test-only diagnostic warning #46540

Merged

Conversation

richiemccoll
Copy link
Contributor

@richiemccoll richiemccoll commented Feb 7, 2023

Fixes: #46448

This PR:

  • emits a test diagnostic when running tests with { only: true } or runOnlyTests(true) without the --test-only CLI flag.

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/test_runner

@nodejs-github-bot nodejs-github-bot added dont-land-on-v14.x needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem. labels Feb 7, 2023
@richiemccoll richiemccoll marked this pull request as ready for review February 7, 2023 10:02
lib/internal/test_runner/test.js Show resolved Hide resolved
lib/internal/test_runner/test.js Outdated Show resolved Hide resolved
test/message/test_runner_output.out Outdated Show resolved Hide resolved
@richiemccoll richiemccoll force-pushed the test-runner/emit-only-warning branch 2 times, most recently from 8292f1b to 514fd0b Compare February 9, 2023 12:11
Copy link
Member

@MoLow MoLow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, But I think we should emit a similar diagnostic when calling runOnly when no flag

@richiemccoll
Copy link
Contributor Author

richiemccoll commented Feb 10, 2023

I think we should emit a similar diagnostic when calling runOnly when no flag

@MoLow Are you referring to this scenario with runOnly below or something else? I've tested this locally and the diagnostic is there.

// running without the --test-only flag
const { test } = require('node:test');

test('top level test', async (t) => {
  t.runOnly(true);
  await t.test('this subtest is run');
});
TAP version 13
# Subtest: top level test
    # Subtest: this subtest is run
    ok 1 - this subtest is run
      ---
      duration_ms: 4.77543
      ...
    # 'only' and 'runOnly' require the --test-only command-line option.
    1..1
ok 1 - top level test
  ---
  duration_ms: 6.194772
  ...
1..1
# tests 1
# pass 1

@cjihrig
Copy link
Contributor

cjihrig commented Feb 11, 2023

I think we should emit a similar diagnostic when calling runOnly when no flag

I think @MoLow is saying to add the diagnostic when runOnly() is called, even if no tests are started. If this change is made, please ensure we don't end up in a situation where the diagnostic is included multiple times for the same test.

@richiemccoll
Copy link
Contributor Author

@cjihrig @MoLow Is that something you'd like to see as part of this PR or can that be tackled in a follow up commit?

@MoLow
Copy link
Member

MoLow commented Feb 13, 2023

Is that something you'd like to see as part of this PR or can that be tackled in a follow up commit?

A follow-up PR is ok

@MoLow
Copy link
Member

MoLow commented Feb 18, 2023

@richiemccoll this PR requires a rebase

@MoLow MoLow added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 20, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 20, 2023
@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@nodejs-github-bot
Copy link
Collaborator

@MoLow MoLow added the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 21, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Feb 21, 2023
@nodejs-github-bot nodejs-github-bot merged commit c90ea93 into nodejs:main Feb 21, 2023
@nodejs-github-bot
Copy link
Collaborator

Landed in c90ea93

MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
PR-URL: nodejs#46540
Fixes: nodejs#46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
PR-URL: nodejs#46540
Fixes: nodejs#46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@MoLow MoLow added the backport-open-v18.x Indicate that the PR has an open backport. label Feb 25, 2023
MoLow pushed a commit to MoLow/node that referenced this pull request Feb 25, 2023
PR-URL: nodejs#46540
Fixes: nodejs#46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
juanarbol pushed a commit that referenced this pull request Mar 3, 2023
PR-URL: #46540
Backport-PR-URL: #46839
Fixes: #46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
@juanarbol juanarbol mentioned this pull request Mar 3, 2023
juanarbol pushed a commit that referenced this pull request Mar 5, 2023
PR-URL: #46540
Backport-PR-URL: #46839
Fixes: #46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Mar 13, 2023
PR-URL: #46540
Fixes: #46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
targos pushed a commit that referenced this pull request Mar 14, 2023
PR-URL: #46540
Fixes: #46448
Reviewed-By: Colin Ihrig <[email protected]>
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-open-v18.x Indicate that the PR has an open backport. needs-ci PRs that need a full CI run. test_runner Issues and PRs related to the test runner subsystem.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test_runner: warn if only is used without properly enabling only tests
6 participants