Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed a flaky test #12

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Fixed a flaky test #12

wants to merge 1 commit into from

Conversation

suzzy777
Copy link

@suzzy777 suzzy777 commented Jan 29, 2022

What is the purpose of the change:

  • This PR is to fix a flaky test tests/test_api.py::TestAPI::test_missing_agent, which can fail after running any of the 9 polluters :
  1. tests/test_execute.py::TestExecute::test_valid_execute
  2. tests/test_get_attribute.py::TestGetAttribute::test_valid_bulk_request
  3. tests/test_get_attribute.py::TestGetAttribute::test_valid_request
  4. tests/test_list.py::TestList::test_invalid_path
  5. tests/test_list.py::TestList::test_valid_request
  6. tests/test_search.py::TestSearch::test_valid_search
  7. tests/test_set_attribute.py::TestSetAttribute::test_valid_bulk_write
  8. tests/test_set_attribute.py::TestSetAttribute::test_valid_request
  9. tests/test_version.py::TestVersion::test_valid_version, but passes when it is run in isolation.

Reproduce the test failure:

  • Run the following command:
    python -m pytest tests/test_execute.py::TestExecute::test_valid_execute tests/test_api.py::TestAPI::test_missing_agent
  • This command can be used with all (9) of the above mentioned polluters and the order dependent test - tests/test_api.py::TestAPI::test_missing_agent

Expected Result:

  • Test tests/test_api.py::TestAPI::test_missing_agent should pass when run after test tests/test_execute.py::TestExecute::test_valid_execute or any of the other 9 polluters.

Actual Result:

  • Test tests/test_api.py::TestAPI::test_missing_agent fails when it is run after any of the polluters.

Why the test fails:

  • The flaky test fails because the test is dependent on some state that's being changed by the polluters.

Fix:

  • The changes in this pull request cleans the state polluted by the polluter and makes the flaky test pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants