Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rework timeouts so they are linked to waiting time of a test not total waiting time #76

Closed
Licenser opened this issue Aug 8, 2022 · 2 comments

Comments

@Licenser
Copy link
Contributor

Licenser commented Aug 8, 2022

Right now timeouts are counted starting from whatever point in time the test runner first picks them up. Having more cores then test can lead to all tests starting the timeout at the same time. With fewer cores they start after a (somewhat) random time.

This can be somewhat problematic as there is no clear and repeatable way to define how long timeouts actually are. a 60s timeout can mean "60s since the start of cargo test" or "60s since the last test finished" depending on the environment.

I wan to propose an alternative, serialize tests so that timeouts are predictable to "the prior test finished". I'm not 100% sure what the best implementation for this is. Or, perhaps even better, the timeout defines how long the test is allowed to take instead of how long the prior test is allowed to take (question here: is that overlapping with what cargo test already does?)

I think it would require somewhat of implementation of a custom test runner that collects all tests for a given key and runs them sequentially.

@palfrey
Copy link
Owner

palfrey commented Aug 10, 2022

Related to earlier question in #68

@palfrey
Copy link
Owner

palfrey commented Dec 12, 2022

#83 resolves this by removing timeouts entirely

@palfrey palfrey closed this as completed Dec 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants