Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report test as risky, if skipif does not has standard-output side-effect #6000

Open
staabm opened this issue Oct 19, 2024 · 0 comments
Open
Labels
feature/phpt End-to-end tests in PHPT format feature/test-runner CLI test runner type/enhancement A new idea that should be implemented

Comments

@staabm
Copy link
Contributor

staabm commented Oct 19, 2024

since #5998 we know which side-effects the PHPT skipif code has.

A skipif condition is expected to output a string starting with "skip", in case the test should be skipped.
this also means a SKIPIF condition which does not at all produce output, can never have a "skipping-effect" on the test-code.

-> we could report a test as "risky" when SKIPIF is declared but does not have such a expected standard output side-effect. I remember we talked about marking such test as "error" instead of "risky", but decided against it because some people use the skipif section to prepare something for the test-case (without the intention to skip it)

--TEST--
PHPT with --SKIPIF-- but without standard output side-effect is risky
--INI--
error_reporting=-1
--SKIPIF--
<?php declare(strict_types=1);
if (1 == 2) {
   exit(1);
}
--FILE--
<?php declare(strict_types=1);
echo "Hello, World!\n";
--EXPECT--
Hello, World!
@staabm staabm added the type/enhancement A new idea that should be implemented label Oct 19, 2024
@sebastianbergmann sebastianbergmann added feature/phpt End-to-end tests in PHPT format feature/test-runner CLI test runner labels Oct 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature/phpt End-to-end tests in PHPT format feature/test-runner CLI test runner type/enhancement A new idea that should be implemented
Projects
None yet
Development

No branches or pull requests

3 participants
@sebastianbergmann @staabm and others