Paratest is unstable and coverage retrieve different percentages each time I rerun tests

[EARodriguez2021]

Q	A
ParaTest version	6.4.1
PHPUnit version	9.5.11
PHP version	8.1.1

When I run tests, the percentage of the coverage is different

if I execute simple phpunit command, the percentage of my coverage is always the same. and after executing paratest, that percentage is different each time I rerun tests

[Slamdunk]

Hi, I am aware of the bug, but need the time to investigate it

[EARodriguez2021]

Thank you, I think this is a great library :)

[Slamdunk]

I finally made it to create a self-contained small enough repo that shows the bug.

@EARodriguez2021 @maks-rafalko can you confirm the issue you're having is similar to https://github.com/Slamdunk/phpunit_codecoverage_bug#readme ?

[EARodriguez2021]

Yes, @Slamdunk you're right, this is one of the main bugs that we experimented using different quantity of processes in the command executed because sometimes constants inside of an array are covered and suddenly same values are not covered without any logical reason. thank you for your valuable time you give us, we really appreciate it.

[Slamdunk]

Thread continues at sebastianbergmann/php-code-coverage#889

[EARodriguez2021]

Perfect, thanks

[Slamdunk]

So, after for 4 days of digging the bug, there's a workaround: autoload the class with the constants in the test bootstrap.
Taking the first example of https://github.com/Slamdunk/phpunit_codecoverage_bug#readme a simple

// tests_bootstrap.php
class_exists(AType::class);

will get you to a consistent report of both tested and untested lines.

I have an idea on how to avoid this manual step, but in the meantime that's what I have for you

[EARodriguez2021]

Excellent, thank you so much @Slamdunk that was a good solution

[Slamdunk]

Stay tuned: sebastianbergmann/php-code-coverage#892 will likely solve the issue altogether 💪

[Slamdunk]

v6.4.3 release now brings much more consistency between PHPUnit and ParaTest code coverage reports