macros: append generated test attribute so others are aware of it #6497

kezhuw · 2024-04-19T08:02:55Z

Motivation

Improve compatibility among multiple test proc macros.

Solution

Add #[::core::prelude::v1::test] to the end of test proc macros. This way, proc macros process later can choose to not generate #[::core::prelude::v1::test] to avoid duplicated test runs.
Forbid #[::core::prelude::v1::test] to avoid duplicated test runs as tokio::test is transforming an invalid test signature to one accepted by #[test]. This is consistent with what we currently treat #[test].

Links

This way other test macros can choose to not generate `#[::core::prelude::v1::test]` to avoid duplicated test runs. This commit also rejects existing `#[::core::prelude::v1::test]` just like how and why it rejects existing `#[test]`.

Darksonn · 2024-04-19T08:05:14Z

tokio-macros/Cargo.toml

@@ -24,7 +24,7 @@ proc-macro = true
 [dependencies]
 proc-macro2 = "1.0.60"
 quote = "1"
-syn = { version = "2.0", features = ["full"] }
+syn = { version = "2.0", features = ["full", "extra-traits"] }


What do you need this feature for? I prefer to avoid adding these features as they hurt compilation times.

impl PartialEq for Attribute. I guess I could duplicate code. Let me figure out.

I see. if it is possible, then try to perform the equality check manually. Thanks!

Darksonn · 2024-04-19T08:06:34Z

tests-build/tests/fail/macros_invalid_input.stderr

+error: second test attribute is supplied, try to order it after `tokio::test`
+  --> $DIR/macros_invalid_input.rs:49:1
+   |
+49 | #[::core::prelude::v1::test::test]
+   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+


What? The #[tokio::test] macro is first, so what do you mean by "after"?

I supposed this to be generated by other preceding test macros, e.g. #[test_log::test]. I guess I should reshape it or keep it origin. Any preference ?

How about "second test attribute is supplied, consider removing or ordering it after tokio::test" ?

My challenge with the wording here is that the code that triggers this error has #[tokio::test] first, and then afterwards, it has a #[::core::prelude::v1::test]. So suggesting that you move #[::core::prelude::v1::test] after #[tokio::test] doesn't make since it's already after #[tokio::test]. Perhaps you could say "before" or "above" instead of "after"?

I know, it is strange. It should be something similar to following.

❯ grep -n -B 2 -A 2 "with_append_test_attribute_and_test_args_and_panic_async" tests/mod.rs 84-#[test_log::test] 85-#[tokio::test] 86:async fn with_append_test_attribute_and_test_args_and_panic_async(x: i8, _y: i8) { 87- assert_eq!(x, 0); 88-} ❯ ❯ cargo test with_append_test_attribute_and_test_args_and_panic_async -- --nocapture error: second test attribute is supplied, consider removing it or ordering it after `tokio::test` --> tests/mod.rs:84:1 | 84 | #[test_log::test] | ^^^^^^^^^^^^^^^^^ | = note: this error originates in the attribute macro `test_log::test` (in Nightly builds, run with -Z macro-backtrace for more info) error: could not compile `test-log` (test "mod") due to 2 previous errors

Moving it "before" or "above" will result in duplicated test runs.

I guess we could:

Introduce dev dependency test-log. This ways the words should be more intuitively. Is it worth ?

Use the short version "second test attribute is supplied".

Other suggestions.

Hi @Darksonn, thank you for reviewing. I have updated this with a fixup commit targeting to your two concerns. Regarding the error words, I am open to any sentence, simply not good at 😮‍💨 .

Maybe just say "consider removing or changing the order of your test attributes".

…ware of it

…s are aware of it

…o others are aware of it

kezhuw · 2024-05-05T05:55:23Z

Hi @Darksonn, thank you for your reviewing, I have solved all review comments. Is there anything else left for this to move forward ? Should I do some squash/rebase work ?

Darksonn

Hi again, thanks for the ping. Sorry about the delay.

Darksonn · 2024-05-05T11:10:35Z

tokio-macros/src/entry.rs

+        syn::Meta::Path(path) => path,
+        _ => return false,
+    };
+    let segments = ["core", "prelude", "v1", "test"];


It's also available as a few other paths such as ::std::prelude::v1::test and ::core::prelude::rust_2021::test.

Would this be too paranoid for us ?

Given that Rust has three editions since v1 and new coming in future, together with combinations from "std", "core" and leading colon, this is a pretty large list.

Given the fact that test is there since day one, I would recommend not doing this. I checked test macros listed below and none use variants other than the two. I don't think test macros should generate other variants, if they do, let them fix it to avoid duplicated test runs but not us.

test-log, test-case, async-std::test, googletest::test uses ::core::prelude::v1::test.

quickcheck, test-strategy, rstest uses #[test]

proptest,tracing_test::traced_test does not generate any

Perhaps we can match ::[std|core]::prelude::*:test?

Updated. Additionally, leading colons are optional now as review comments below.

LGTM. If you can also add some of the variants to the tests, then I think this is ready to merge.

Darksonn · 2024-05-05T11:16:30Z

tests-build/tests/fail/macros_invalid_input.rs

+#[tokio::test]
+#[::core::prelude::v1::test]
+async fn test_has_generated_second_test_attr() {}


What about core without the leading colons?

tokio-macros/src/entry.rs

Darksonn

Thank you.

We decided to bump minimum required version of tokio to 1.34. Currently, the newest tokio version is 1.38, but some of the integration tests are eaten when testing with this specific verstion of tokio. Which is why, as of now, we decided not to support this version. The issue with version 1.38 is related to #[tokio::test] and #[ntest::timeout] attributes. Refs: - tokio-rs/tokio#6610 - becheran/ntest#28 - tokio-rs/tokio#6497

Darksonn reviewed Apr 19, 2024

View reviewed changes

fixup! macros: append generated test attribute so others are aware of it

81e10af

kezhuw mentioned this pull request Apr 19, 2024

fix: append #[::core::prelude::v1::test] only if it does not exist d-e-s-o/test-log#46

Open

kezhuw added 4 commits April 19, 2024 21:37

fixup! fixup! macros: append generated test attribute so others are a…

90afc4c

…ware of it

fixup! fixup! fixup! macros: append generated test attribute so other…

a9d0609

…s are aware of it

fixup! fixup! fixup! fixup! macros: append generated test attribute s…

a4828d4

…o others are aware of it

align macros_invalid_input.stderr

4f23227

mox692 added the A-tokio-macros Area: The tokio-macros crate label Apr 20, 2024

Darksonn reviewed May 5, 2024

View reviewed changes

kezhuw added 3 commits May 5, 2024 22:29

assert that we append test attribute instead of prepend

d920650

fix mismatch in macros_invalid_input.stderr

51c473c

reject more test attributes

6d2a30f

kezhuw requested a review from Darksonn May 5, 2024 15:45

add test for test path combinations

39c9a1d

Darksonn approved these changes May 5, 2024

View reviewed changes

Merge branch 'master' into test-proc-macros-cooperation

9426fa9

Darksonn enabled auto-merge (squash) May 5, 2024 16:19

Darksonn merged commit 6fcd9c0 into tokio-rs:master May 5, 2024
77 checks passed

kezhuw mentioned this pull request May 24, 2024

fix: improve compatibility among test proc macros frondeus/test-case#143

Open

This was referenced May 30, 2024

Prepare tokio-macros v2.3.0 #6600

Merged

chore: prepare Tokio v1.38.0 #6601

Merged

taiki-e mentioned this pull request Jun 1, 2024

The coverage-helper::test doesn't work with tokio::test taiki-e/coverage-helper#2

Open

mox692 mentioned this pull request Jun 3, 2024

regression: tokio::test in 1.38 eats my tests #6610

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

macros: append generated test attribute so others are aware of it #6497

macros: append generated test attribute so others are aware of it #6497

kezhuw commented Apr 19, 2024

Darksonn Apr 19, 2024

kezhuw Apr 19, 2024

Darksonn Apr 19, 2024

Darksonn Apr 19, 2024

kezhuw Apr 19, 2024

kezhuw Apr 19, 2024

Darksonn Apr 19, 2024 •

edited

Loading

kezhuw Apr 19, 2024

Darksonn Apr 19, 2024

kezhuw commented May 5, 2024

Darksonn left a comment

Darksonn May 5, 2024

kezhuw May 5, 2024

Darksonn May 5, 2024

kezhuw May 5, 2024

Darksonn May 5, 2024

Darksonn May 5, 2024

Darksonn left a comment

macros: append generated test attribute so others are aware of it #6497

macros: append generated test attribute so others are aware of it #6497

Conversation

kezhuw commented Apr 19, 2024

Motivation

Solution

Links

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Darksonn Apr 19, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kezhuw commented May 5, 2024

Darksonn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Darksonn left a comment

Choose a reason for hiding this comment

Darksonn Apr 19, 2024 •

edited

Loading