Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests time out on Travis #11

Open
3 tasks
LourensVeen opened this issue Jan 21, 2019 · 1 comment
Open
3 tasks

Tests time out on Travis #11

LourensVeen opened this issue Jan 21, 2019 · 1 comment

Comments

@LourensVeen
Copy link
Contributor

The tests are very slow, and even slower on Travis, where they have now started to time out regularly. This should be improved.

  • Add measurement and get the timings of a test run. How long does it take to start and stop the containers? When does each test start and stop? Are things running in parallel or not? Output ideally to the console, so that we can test on Travis as well.
  • Try to speed up the tests, possibly by manually parallelising the scheduler tests?
  • Travis times out after not receiving any console output for 10 minutes. We have all the scheduler tests in a single file, which puts them on a single line, and it seems that there's a line buffer somewhere. Can we remove that buffer, or split the tests over multiple files so that Travis sees output more frequently?
@LourensVeen
Copy link
Contributor Author

We're not using Travis anymore, and the tests are running fine on GitHub Actions. They still take quite a while to run, about 15 minutes. Locally it's the same, which does slow down development.

The main things that take time seem to be building the test container, installing software (many versions of Python, dependencies for each, etc.), and then waiting for slurm to pick up and run jobs.

Some ideas:

  • We're using docker-compose to build the test image, and I think that uses the old fashioned build rather than buildx. Maybe we can build it separately, or maybe a newer docker-compose (we're still using the now-unsupported ancient Python version) will use buildx?
  • We're currently using 8 threads for the scheduler tests. CPU load isn't very high though on my laptop, because we're mostly waiting around for jobs to get picked up or results accounted. So maybe we can just increase the number of parallel tests greatly here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant