Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compiletest: tests using revisions treat files with just //~ ERROR has having no errors #55695

Closed
pnkfelix opened this issue Nov 5, 2018 · 3 comments
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.

Comments

@pnkfelix
Copy link
Member

pnkfelix commented Nov 5, 2018

The revision system lets compiletest encode different sets of errors for different invocations of the compiler.

and compiletest normally attempts to catch cases where you left out expected error messages by saying "if you have any //~ ERROR ... annotation in your file, then the set of such annotations needs to match the same set of errors that are actually emitted.

However, if you combine these features, then for each revision (call it rev) in the revisions list, at least one of your errors currently need to be written with the revision system in mind, e.g. //[rev]~ ERROR ...

If none of the error annotations include a revision (i.e., if they are all //~ ERROR ...), then compiletest thinks that the test has no error annotations, and thus treats it as a case where the compiler's error output is blindly accepted.

That seems bad.

@memoryruins
Copy link
Contributor

cc #55596

@pnkfelix pnkfelix added the A-testsuite Area: The testsuite used to check the correctness of rustc label Nov 8, 2018
pnkfelix added a commit to pnkfelix/rust that referenced this issue Nov 8, 2018
(The commit prior to this actual passes our test suite, "thanks"
to rust-lang#55695. But since I am aware of that bug, I took advantage of it
in choosing how to order my commit series...)
@JohnTitor JohnTitor added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label Apr 12, 2020
@varkor
Copy link
Member

varkor commented Oct 3, 2020

This is surely no longer the case, because ERROR and WARN annotations are now mandatory?

@tmiasko
Copy link
Contributor

tmiasko commented Feb 1, 2022

With annotation for warnings and errors being mandatory #66213, and additionally //~ ERROR annotations applying to all revisions #66524, this should be non-issue.

@tmiasko tmiasko closed this as completed Feb 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

No branches or pull requests

5 participants