Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky decorators on individual tests have no effect if --force-flaky argument is passed #209

Open
tdluk opened this issue Apr 16, 2024 · 1 comment

Comments

@tdluk
Copy link

tdluk commented Apr 16, 2024

From README.rst:

Pass --max-runs=MAX_RUNS and/or --min-passes=MIN_PASSES to control the behavior of flaky if --force-flaky is specified. Flaky decorators on individual tests will override these defaults.

However, flaky decorators on individual tests do not override these defaults.

To repro the issue, consider the following test file test.py:

import pytest

@pytest.mark.flaky(max_runs=10)
def test_foo():
    assert False

Then run pytest test.py --force-flaky and observe the test was run the default 2 times instead of the 10 specified by the decorator.

@tdluk
Copy link
Author

tdluk commented Apr 16, 2024

Problem appears to be here:

if not self._has_flaky_attributes(item):

This means an individual test's decorators won't be translated into flaky attributes, as the test's flaky attributes have already been set to the defaults (or values set in CLI args) here:

if self.force_flaky and not self._has_flaky_attributes(item):
self._make_test_flaky(
item,
self.max_runs,
self.min_passes,
)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant