Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use llvm-args instead of RUST_TEST_THREADS, %Nm instead of NEXTEST_TEST_THREADS #279

Merged
merged 2 commits into from
Jun 2, 2023

Conversation

taiki-e
Copy link
Owner

@taiki-e taiki-e commented May 13, 2023

cargo-llvm-cov no longer sets the RUST_TEST_THREADS environment variable and uses llvm-args instead for workaround rust-lang/rust#91092.

TODO: skip passing llvm-args on newer nightly that includes rust-lang/rust#111469.

Closes #258
cc @Dushistov

@taiki-e
Copy link
Owner Author

taiki-e commented May 13, 2023

Ah, we should also pass -fprofile-update=atomic to clang.

let clang_flags = " -fprofile-instr-generate -fcoverage-mapping";

@Dushistov
Copy link

Dushistov commented May 13, 2023

It works and works correctly as far as I tested. But I don't see any time difference.
I tried functional and unit tests in one crate, like #258 .
Also I tried run tests for workspace with several crates there is no difference in work time for cargo llvm-cov nextest.

@taiki-e
Copy link
Owner Author

taiki-e commented May 14, 2023

As for NEXTEST_TEST_THREADS, I have replaced it with the way described in #258 (comment). I have confirmed that #258 is fixed, but have not yet been able to test whether the original problem is completely fixed.

$ cargo llvm-cov nextest
   Compiling a v0.1.0 (/Users/taiki/projects/tmp/a)
    Finished test [unoptimized + debuginfo] target(s) in 0.14s
    Starting 2 tests across 2 binaries
        PASS [   8.015s] a tests::test_8
        PASS [  30.015s] a::functional_tests test_30
------------
     Summary [  30.018s] 2 tests run: 2 passed, 0 skipped

@taiki-e taiki-e changed the title Use llvm-args instead of RUST_TEST_THREADS Use llvm-args instead of RUST_TEST_THREADS, %Nm instead of NEXTEST_TEST_THREADS May 14, 2023
@Dushistov
Copy link

Dushistov commented May 17, 2023

As for NEXTEST_TEST_THREADS, I have replaced it with the way described i

As result, on my real code base, I got such results:
before this PR: 556.727s
after this PR. : 182.539s

Great!

The coverage numbers are little different:

-Totals       67.03% (1545/2305)   79.55% (18853/23699)   67.27% (8344/12403) - (0/0)
+Totals       67.03% (1545/2305)   79.51% (18842/23699)   67.20% (8335/12403) - (0/0)

but I use https://github.com/BurntSushi/quickcheck that uses random generator,
so it is expected.

So, I finally can add all crates, not only important ones, in my workspace into coverage calculation for every CI run.

@svenstaro
Copy link

Could this be merged? From what I can tell, this is good to go, no?

@taiki-e taiki-e force-pushed the atomic-counter-update branch 2 times, most recently from ac365d9 to 04e192a Compare June 2, 2023 17:24
@taiki-e taiki-e merged commit ca7071c into main Jun 2, 2023
@taiki-e taiki-e deleted the atomic-counter-update branch June 2, 2023 17:33
@taiki-e
Copy link
Owner Author

taiki-e commented Jun 2, 2023

Published in v0.5.20. Thanks @Dushistov for testing this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cargo llvm-cov nextest no multithreading for funtional tests?
3 participants