cargo test --test does not find any tests

[mqudsi]

Background: I'm improving fish's dynamic shell completions for cargo to do things like have cargo run --bin <TAB> give you a list of available binaries to choose from for tab completion.

It seems that cargo's dynamic test discovery is broken (or never implemented?).

e.g. while cargo bench --bench returns a list of all benchmarks along with the error message:

$ cargo bench --bench
error: "--bench" takes one argument.
Available benches:
    bench

cargo test --test always returns "no tests available" incorrectly:

$ cargo test --test
error: "--test" takes one argument.
No tests available.

while actually running cargo test shows that cargo has no problem enumerating tests:

mqudsi@ZBOOK /m/c/U/m/g/unquote> cargo test
    Finished test [unoptimized + debuginfo] target(s) in 0.12s
     Running target/debug/deps/unquote-d4809e2bcea39ff6

running 18 tests
... <SNIP>

[ehuss]

Do you have separate tests in your project? The --test flag only works with integration tests in the tests/ folder. If you do, is your project publicly available so I can look?

[mqudsi]

No, the tests are integrated in the project I tested it on.

The output of cargo test:

running 18 tests
test tests::empty_quotes ... ok
test tests::escaped_double_quote ... ok
test tests::escaped_single_quote ... ok
test tests::interpolated_double_quoted ... ok
test tests::leading_whitespace ... ok
test tests::nested_double_single_quote ... ok
test tests::nested_single_double_quote ... ok
test tests::interword_whitespace ... ok
test tests::one_double_quoted_word ... ok
test tests::one_single_quoted_word ... ok
test tests::one_spaced_double_quoted_word ... ok
test tests::one_spaced_single_quoted_word ... ok
test tests::one_word ... ok
test tests::only_whitespace ... ok
test tests::trailing_whitespace ... ok
test tests::two_double_quoted_words ... ok
test tests::two_single_quoted_words ... ok
test tests::two_words ... ok

but a single test can selectively be run via cargo test TEST_NAME:

mqudsi@ZBOOK /m/c/U/m/g/unquote> cargo test trailing_whitespace
    Finished test [unoptimized + debuginfo] target(s) in 0.13s
     Running target/debug/deps/unquote-d4809e2bcea39ff6

running 1 test
test tests::trailing_whitespace ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 17 filtered out

Meaning all the functionality I am seeking exists, but perhaps it is not exposed? How can I get a list of tests (not test files) without running them all?

[ehuss]

You can run the test executable with the --list option after it has been built, though I'm not sure how stable that output is. When you pass an argument to cargo test, that is handled by the test harness which handles filtering and such. You can run cargo test -- --help to get the help output from the harness to see its options.

Otherwise, there aren't really any easy ways to get the test names.

[ehuss]

I'll leave this open, though it really seems to cover a few different issues:

1. Documentation probably doesn't emphasize enough the difference between --test (an integration test), and a #[test] (a single "test" function).
2. Errors could probably be a little more detailed. For example, bare --test with no integration tests could explain that the flag is for integration tests.
3. Provide a better way to programmatically access the list of tests. This is more of a request for libtest, but possibly could use some integration with Cargo.

[epage]

Documentation probably doesn't emphasize enough the difference between --test (an integration test), and a #[test] (a single "test" function).

I think the new help output covers this

Target Selection:
      --lib               Test only this package's library unit tests
      --bins              Test all binaries
      --bin [<NAME>]      Test only the specified binary
      --examples          Test all examples
      --example [<NAME>]  Test only the specified example
      --tests             Test all tests
      --test [<NAME>]     Test only the specified test target
      --benches           Test all benches
      --bench [<NAME>]    Test only the specified bench target
      --all-targets       Test all targets (does not include doctests)

[epage]

Provide a better way to programmatically access the list of tests. This is more of a request for libtest, but possibly could use some integration with Cargo.

I feel like this would be best left to rust-lang/rust#49359