You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now timeouts are counted starting from whatever point in time the test runner first picks them up. Having more cores then test can lead to all tests starting the timeout at the same time. With fewer cores they start after a (somewhat) random time.
This can be somewhat problematic as there is no clear and repeatable way to define how long timeouts actually are. a 60s timeout can mean "60s since the start of cargo test" or "60s since the last test finished" depending on the environment.
I wan to propose an alternative, serialize tests so that timeouts are predictable to "the prior test finished". I'm not 100% sure what the best implementation for this is. Or, perhaps even better, the timeout defines how long the test is allowed to take instead of how long the prior test is allowed to take (question here: is that overlapping with what cargo test already does?)
I think it would require somewhat of implementation of a custom test runner that collects all tests for a given key and runs them sequentially.
The text was updated successfully, but these errors were encountered:
Right now timeouts are counted starting from whatever point in time the test runner first picks them up. Having more cores then test can lead to all tests starting the timeout at the same time. With fewer cores they start after a (somewhat) random time.
This can be somewhat problematic as there is no clear and repeatable way to define how long timeouts actually are. a 60s timeout can mean "60s since the start of
cargo test
" or "60s since the last test finished" depending on the environment.I wan to propose an alternative, serialize tests so that timeouts are predictable to "the prior test finished". I'm not 100% sure what the best implementation for this is. Or, perhaps even better, the timeout defines how long the test is allowed to take instead of how long the prior test is allowed to take (question here: is that overlapping with what
cargo test
already does?)I think it would require somewhat of implementation of a custom test runner that collects all tests for a given key and runs them sequentially.
The text was updated successfully, but these errors were encountered: