Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: Adds test-(slow|fast) options #3555

Merged
merged 10 commits into from
Aug 31, 2024

Conversation

dangotbanned
Copy link
Member

@dangotbanned dangotbanned commented Aug 23, 2024

While working on #3547, I've been running the test suite a bunch to identify bottlenecks.

This PR provides

  • tools to identify, skip or run only slow tests
  • some new, previously repeated mark decorators
  • misc refactoring of tests using the above
  • docs explaining intended usage

Impact?

  • Locally, this covers most tests that take 2-10 secs.
  • In total, 354/1671 tests are skipped
  • reduced runtime 40 -> 20 secs

Important

The default behaviour is unchanged:

# Runs all tests
>>> hatch test 

# Runs only slow tests
>>> hatch run test-slow

# Runs all tests (that are not marked @slow)
>>> hatch run test-fast

@dangotbanned dangotbanned changed the title test: Adds (slow|fast) test options test: Adds test-(slow|fast) test options Aug 23, 2024
@dangotbanned dangotbanned changed the title test: Adds test-(slow|fast) test options test: Adds test-(slow|fast) options Aug 23, 2024
@dangotbanned
Copy link
Member Author

These were some drive-by fixes of typing issues, fixing them here in case this stays in review for a while
They would show on mypy as well, but are hidden due to https://microsoft.github.io/pyright/#/mypy-comparison?id=type-checking-unannotated-code

@dangotbanned dangotbanned enabled auto-merge (squash) August 29, 2024 13:47
Copy link
Contributor

@binste binste left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really neat! :) I learned a lot while reading this. Also great to have more convenient decorators for skipping tests re pyarrow and vlc.

tests/utils/test_utils.py Show resolved Hide resolved
@dangotbanned
Copy link
Member Author

dangotbanned commented Aug 31, 2024

Really neat! :) I learned a lot while reading this. Also great to have more convenient decorators for skipping tests re pyarrow and vlc.

Thanks @binste!

I've found when reading the pytest docs that they have a lot of info - but can be overwhelming at times.
The docs I've added here are as much for my own benefit; as they are to simplify the test writing of others

@dangotbanned dangotbanned merged commit de58ec8 into vega:main Aug 31, 2024
13 checks passed
@dangotbanned dangotbanned deleted the tests-refac-perf-1 branch September 5, 2024 10:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants