Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests are flakey and unstable #382

Open
runspired opened this issue Aug 4, 2022 · 1 comment
Open

Tests are flakey and unstable #382

runspired opened this issue Aug 4, 2022 · 1 comment

Comments

@runspired
Copy link
Contributor

tests will fail in unpredictable patterns, and this is somewhat hidden by CI timing out. Obervable in #381 where a readme change leads to failures.

@mixonic
Copy link
Contributor

mixonic commented Aug 4, 2022

There are two issues:

  • First, there are unpredictable failures in the test suite. These are papered over by a retry script which re-runs the whole suite on CI if there is a failure, and only gives up after several attempts.
  • Second, there is something in the test suite or runner which causes the github action to hang. This second item is what the timeout circumvents. Without a timeout, it simply hangs for hours.

mixonic added a commit that referenced this issue Aug 4, 2022
There is something in the vertical collection test suite which
periodically hangs the action runner. Extending the timeout to 15
minutes doesn't make it more likely we get a green build, instead it
increases the delay you need to wait before retrying.

Relates to #382
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants