Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bootstrap/libtest: print test name eagerly on failure even with verbose-tests=false / --quiet #113518

Merged
merged 2 commits into from
Mar 6, 2024

Conversation

jyn514
Copy link
Member

@jyn514 jyn514 commented Jul 10, 2023

Previously, libtest would wait until all tests finished running to print the progress, which made it
annoying to run many tests at once (since you don't know which have failed). Change it to print the
names as soon as they fail.

This makes it much easier to know which test failed without having to wait for compiletest to completely finish running. Before:

Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiFFiiiiiii
...

After:

Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii
[ui] tests/ui/associated-type-bounds/implied-in-supertrait.rs ... F

[ui] tests/ui/associated-type-bounds/return-type-notation/basic.rs#next_with ... F
iiiiiiiiiiiii
...

This serves a similar use case to the existing RUSTC_TEST_FAIL_FAST, but is on by default and as a result much more discoverable. We should consider unifying RUSTC_TEST_FAIL_FAST with the --no-fail-fast flag in the future for consistency and discoverability.

@rustbot
Copy link
Collaborator

rustbot commented Jul 10, 2023

r? @Mark-Simulacrum

(rustbot has picked a reviewer for you, use r? to override)

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Jul 10, 2023
@Kobzol
Copy link
Contributor

Kobzol commented Jul 12, 2023

This is great! It has been bugging me that the failures are not shown until the end. I had no idea about RUSTC_TEST_FAIL_FAST, of course.

@Kobzol
Copy link
Contributor

Kobzol commented Jul 13, 2023

I tried it locally and the immediate feedback is great, but it also really borks the nice table output of compiletest :/ In pytest, there's either the succint ...FF.... output, or a more verbose output like this:

test 1..... [PASSED]
test 2..... [FAILED]
test 3..... [PASSED]

But probably since we have so many tests, it would print too many lines and the failed tests would very quickly "scroll up".

@jyn514
Copy link
Member Author

jyn514 commented Jul 13, 2023

you can try out the more verbose output locally if you like, verbose-tests = true

@Kobzol
Copy link
Contributor

Kobzol commented Jul 13, 2023

With verbose tests, the failed tests indeed disappear too quickly I think. So this new output mode is definitely useful, to keep succint output, but also warn about test failures.

@jyn514 jyn514 added the A-testsuite Area: The testsuite used to check the correctness of rustc label Jul 14, 2023
@Mark-Simulacrum
Copy link
Member

Does this mean we're diverging from the libtest formatting? Or does libtest not have a "just dots" output?

I would prefer to avoid diverging if libtest does have such an output (i.e., maybe we can update that, since this seems like a reasonable change), but if this is already custom bootstrap code then r=me.

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 16, 2023
@jyn514
Copy link
Member Author

jyn514 commented Aug 21, 2023

rust-lang/team#1051

@jyn514 jyn514 closed this Aug 21, 2023
@jyn514 jyn514 changed the title bootstrap: print test name on failure even with verbose-tests=false bootstrap/libtest: print test name on failure even with verbose-tests=false Nov 5, 2023
@jyn514
Copy link
Member Author

jyn514 commented Nov 5, 2023

Does this mean we're diverging from the libtest formatting? Or does libtest not have a "just dots" output?

yes. i updated this to update libtest too.

@jyn514 jyn514 reopened this Nov 5, 2023
@jyn514 jyn514 changed the title bootstrap/libtest: print test name on failure even with verbose-tests=false bootstrap/libtest: print test name eagerly on failure even with verbose-tests=false / --quiet Nov 5, 2023
@jyn514
Copy link
Member Author

jyn514 commented Nov 5, 2023

r? @libs

@rustbot rustbot added the T-libs Relevant to the library team, which will review and decide on the PR/issue. label Nov 5, 2023
@rustbot rustbot assigned thomcc and unassigned Mark-Simulacrum Nov 5, 2023
@jyn514
Copy link
Member Author

jyn514 commented Nov 5, 2023

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. labels Nov 5, 2023
@jyn514
Copy link
Member Author

jyn514 commented Jan 25, 2024

r? libs

@rustbot rustbot assigned cuviper and unassigned thomcc Jan 25, 2024
Copy link
Member

@cuviper cuviper left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @rust-lang/testing-devex for test crate changes

library/test/src/formatters/terse.rs Outdated Show resolved Hide resolved
library/test/src/formatters/terse.rs Outdated Show resolved Hide resolved
@jyn514
Copy link
Member Author

jyn514 commented Feb 17, 2024

bump @cuviper :)

@bors
Copy link
Contributor

bors commented Feb 21, 2024

☔ The latest upstream changes (presumably #121400) made this pull request unmergeable. Please resolve the merge conflicts.

@cuviper
Copy link
Member

cuviper commented Mar 4, 2024

I think it's good, but needs a rebase.

This makes it much easier to know which test failed without having to wait for compiletest to completely finish running. Before:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiFFiiiiiii
...
```

After:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii
[ui] tests/ui/associated-type-bounds/implied-in-supertrait.rs ... F

[ui] tests/ui/associated-type-bounds/return-type-notation/basic.rs#next_with ... F
iiiiiiiiiiiii
...
```

This serves a similar use case to the existing RUSTC_TEST_FAIL_FAST, but is on by default and as a result much more discoverable.
We should consider unifying RUSTC_TEST_FAIL_FAST with the `--no-fail-fast` flag in the future for consistency and discoverability.
@rust-log-analyzer

This comment has been minimized.

Previously, libtest would wait until all tests finished running to print the progress, which made it
annoying to run many tests at once (since you don't know which have failed). Change it to print the
names as soon as they fail.

This also adds a test for the terse output; previously it was untested.
@oli-obk
Copy link
Contributor

oli-obk commented Mar 5, 2024

@bors r=cuviper

@bors
Copy link
Contributor

bors commented Mar 5, 2024

📌 Commit 8bfe9db has been approved by cuviper

It is now in the queue for this repository.

@bors bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 5, 2024
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Mar 6, 2024
bootstrap/libtest: print test name eagerly on failure even with `verbose-tests=false` / `--quiet`

Previously, libtest would wait until all tests finished running to print the progress, which made it
annoying to run many tests at once (since you don't know which have failed). Change it to print the
names as soon as they fail.

This makes it much easier to know which test failed without having to wait for compiletest to completely finish running. Before:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiFFiiiiiii
...
```

After:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii
[ui] tests/ui/associated-type-bounds/implied-in-supertrait.rs ... F

[ui] tests/ui/associated-type-bounds/return-type-notation/basic.rs#next_with ... F
iiiiiiiiiiiii
...
```

This serves a similar use case to the existing RUSTC_TEST_FAIL_FAST, but is on by default and as a result much more discoverable. We should consider unifying RUSTC_TEST_FAIL_FAST with the `--no-fail-fast` flag in the future for consistency and discoverability.
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 6, 2024
…iaskrgr

Rollup of 9 pull requests

Successful merges:

 - rust-lang#113518 (bootstrap/libtest: print test name eagerly on failure even with `verbose-tests=false` / `--quiet`)
 - rust-lang#119888 (Stabilize the `#[diagnostic]` namespace and `#[diagnostic::on_unimplemented]` attribute)
 - rust-lang#121752 (Detect unused struct impls pub trait)
 - rust-lang#121885 (Move generic `NonZero` `rustc_layout_scalar_valid_range_start` attribute to inner type.)
 - rust-lang#121926 (`f16` and `f128` step 3: compiler support & feature gate)
 - rust-lang#121959 (Removing absolute path in proc-macro)
 - rust-lang#122015 (Add better explanation for `rustc_index::IndexVec`)
 - rust-lang#122027 (Uplift some feeding out of `associated_type_for_impl_trait_in_impl` and into queries)
 - rust-lang#122038 (Fix linting paths with qself in `unused_qualifications`)

r? `@ghost`
`@rustbot` modify labels: rollup
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Mar 6, 2024
bootstrap/libtest: print test name eagerly on failure even with `verbose-tests=false` / `--quiet`

Previously, libtest would wait until all tests finished running to print the progress, which made it
annoying to run many tests at once (since you don't know which have failed). Change it to print the
names as soon as they fail.

This makes it much easier to know which test failed without having to wait for compiletest to completely finish running. Before:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiFFiiiiiii
...
```

After:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii
[ui] tests/ui/associated-type-bounds/implied-in-supertrait.rs ... F

[ui] tests/ui/associated-type-bounds/return-type-notation/basic.rs#next_with ... F
iiiiiiiiiiiii
...
```

This serves a similar use case to the existing RUSTC_TEST_FAIL_FAST, but is on by default and as a result much more discoverable. We should consider unifying RUSTC_TEST_FAIL_FAST with the `--no-fail-fast` flag in the future for consistency and discoverability.
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Mar 6, 2024
bootstrap/libtest: print test name eagerly on failure even with `verbose-tests=false` / `--quiet`

Previously, libtest would wait until all tests finished running to print the progress, which made it
annoying to run many tests at once (since you don't know which have failed). Change it to print the
names as soon as they fail.

This makes it much easier to know which test failed without having to wait for compiletest to completely finish running. Before:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiFFiiiiiii
...
```

After:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii
[ui] tests/ui/associated-type-bounds/implied-in-supertrait.rs ... F

[ui] tests/ui/associated-type-bounds/return-type-notation/basic.rs#next_with ... F
iiiiiiiiiiiii
...
```

This serves a similar use case to the existing RUSTC_TEST_FAIL_FAST, but is on by default and as a result much more discoverable. We should consider unifying RUSTC_TEST_FAIL_FAST with the `--no-fail-fast` flag in the future for consistency and discoverability.
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 6, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#113518 (bootstrap/libtest: print test name eagerly on failure even with `verbose-tests=false` / `--quiet`)
 - rust-lang#119888 (Stabilize the `#[diagnostic]` namespace and `#[diagnostic::on_unimplemented]` attribute)
 - rust-lang#121089 (Remove `feed_local_def_id`)
 - rust-lang#121926 (`f16` and `f128` step 3: compiler support & feature gate)
 - rust-lang#121959 (Removing absolute path in proc-macro)
 - rust-lang#122015 (Add better explanation for `rustc_index::IndexVec`)
 - rust-lang#122027 (Uplift some feeding out of `associated_type_for_impl_trait_in_impl` and into queries)
 - rust-lang#122038 (Fix linting paths with qself in `unused_qualifications`)

r? `@ghost`
`@rustbot` modify labels: rollup
matthiaskrgr added a commit to matthiaskrgr/rust that referenced this pull request Mar 6, 2024
bootstrap/libtest: print test name eagerly on failure even with `verbose-tests=false` / `--quiet`

Previously, libtest would wait until all tests finished running to print the progress, which made it
annoying to run many tests at once (since you don't know which have failed). Change it to print the
names as soon as they fail.

This makes it much easier to know which test failed without having to wait for compiletest to completely finish running. Before:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiFFiiiiiii
...
```

After:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii
[ui] tests/ui/associated-type-bounds/implied-in-supertrait.rs ... F

[ui] tests/ui/associated-type-bounds/return-type-notation/basic.rs#next_with ... F
iiiiiiiiiiiii
...
```

This serves a similar use case to the existing RUSTC_TEST_FAIL_FAST, but is on by default and as a result much more discoverable. We should consider unifying RUSTC_TEST_FAIL_FAST with the `--no-fail-fast` flag in the future for consistency and discoverability.
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 6, 2024
…iaskrgr

Rollup of 8 pull requests

Successful merges:

 - rust-lang#113518 (bootstrap/libtest: print test name eagerly on failure even with `verbose-tests=false` / `--quiet`)
 - rust-lang#117199 (Change the documented implicit value of `-C instrument-coverage` to `=yes`)
 - rust-lang#121190 (avoid overlapping privacy suggestion for single nested imports)
 - rust-lang#121382 (Rework `untranslatable_diagnostic` lint)
 - rust-lang#121833 (Suggest correct path in include_bytes!)
 - rust-lang#121959 (Removing absolute path in proc-macro)
 - rust-lang#122038 (Fix linting paths with qself in `unused_qualifications`)
 - rust-lang#122051 (cleanup: remove zero-offset GEP)

r? `@ghost`
`@rustbot` modify labels: rollup
bors added a commit to rust-lang-ci/rust that referenced this pull request Mar 6, 2024
…iaskrgr

Rollup of 7 pull requests

Successful merges:

 - rust-lang#113518 (bootstrap/libtest: print test name eagerly on failure even with `verbose-tests=false` / `--quiet`)
 - rust-lang#117199 (Change the documented implicit value of `-C instrument-coverage` to `=yes`)
 - rust-lang#121190 (avoid overlapping privacy suggestion for single nested imports)
 - rust-lang#121382 (Rework `untranslatable_diagnostic` lint)
 - rust-lang#121959 (Removing absolute path in proc-macro)
 - rust-lang#122038 (Fix linting paths with qself in `unused_qualifications`)
 - rust-lang#122051 (cleanup: remove zero-offset GEP)

r? `@ghost`
`@rustbot` modify labels: rollup
@bors bors merged commit 1b157a0 into rust-lang:master Mar 6, 2024
11 checks passed
@rustbot rustbot added this to the 1.78.0 milestone Mar 6, 2024
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Mar 6, 2024
Rollup merge of rust-lang#113518 - jyn514:streaming-failures, r=cuviper

bootstrap/libtest: print test name eagerly on failure even with `verbose-tests=false` / `--quiet`

Previously, libtest would wait until all tests finished running to print the progress, which made it
annoying to run many tests at once (since you don't know which have failed). Change it to print the
names as soon as they fail.

This makes it much easier to know which test failed without having to wait for compiletest to completely finish running. Before:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiFFiiiiiii
...
```

After:
```
Testing stage0 compiletest suite=ui mode=ui (x86_64-unknown-linux-gnu)

running 15274 tests
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii    88/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   176/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   264/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   352/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   440/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii   528/15274
iiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiii
[ui] tests/ui/associated-type-bounds/implied-in-supertrait.rs ... F

[ui] tests/ui/associated-type-bounds/return-type-notation/basic.rs#next_with ... F
iiiiiiiiiiiii
...
```

This serves a similar use case to the existing RUSTC_TEST_FAIL_FAST, but is on by default and as a result much more discoverable. We should consider unifying RUSTC_TEST_FAIL_FAST with the `--no-fail-fast` flag in the future for consistency and discoverability.
@jyn514 jyn514 deleted the streaming-failures branch March 7, 2024 00:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-libs Relevant to the library team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants