Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up some tests #1657

Merged
merged 4 commits into from
Oct 23, 2023
Merged

Speed up some tests #1657

merged 4 commits into from
Oct 23, 2023

Conversation

dlakaplan
Copy link
Contributor

Trying to identify the tests that take the longest and make them faster. So far working on test_grid.py. Previously this took ~15min on my machine:

========================================================================================== 25 passed, 2 warnings in 993.05s (0:16:33) ==========================================================================================
pytest tests/test_grid.py  368.17s user 130.13s system 50% cpu 16:36.10 total

but now it takes 5 (just made the grids smaller):

========================================================================================== 25 passed, 2 warnings in 332.38s (0:05:32) ==========================================================================================
pytest tests/test_grid.py  197.18s user 61.64s system 77% cpu 5:35.38 total

@dlakaplan dlakaplan changed the title Speed up tests WIP: Speed up tests Oct 20, 2023
@codecov
Copy link

codecov bot commented Oct 20, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (334fab7) 68.49% compared to head (342ba46) 68.51%.
Report is 1 commits behind head on master.

❗ Current head 342ba46 differs from pull request most recent head 439b66e. Consider uploading reports for the commit 439b66e to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1657      +/-   ##
==========================================
+ Coverage   68.49%   68.51%   +0.02%     
==========================================
  Files         104      104              
  Lines       24267    24264       -3     
  Branches     4330     4329       -1     
==========================================
+ Hits        16622    16625       +3     
+ Misses       6558     6553       -5     
+ Partials     1087     1086       -1     

see 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@dlakaplan
Copy link
Contributor Author

Slowest 20 remaining tests:

============================= slowest 20 durations =============================
228.56s call     tests/test_tim_writing.py::test_flags
112.02s call     tests/test_toa_indexing.py::test_getitem_boolean
111.43s call     tests/test_toa_indexing.py::test_select
111.29s call     tests/test_toa_indexing.py::test_getitem_where
110.43s call     tests/test_toa_indexing.py::test_getitem_slice
48.87s call     tests/test_fake_toas.py::test_fake_DMfit
42.52s call     tests/test_toa_reader.py::test_load_multiple
42.46s call     tests/test_toa_reader.py::test_pickle_multiple
37.84s call     tests/test_grid.py::test_grid_3param_prefix_singleprocessor
36.54s call     tests/test_polycos.py::test_generate_polycos[72-9-400.0-gbt]
35.31s call     tests/test_grid.py::test_grid_3param_singleprocessor
34.52s call     tests/test_polycos.py::test_generate_polycos[72-9-1400.0-gbt]
34.51s call     tests/test_polycos.py::test_generate_polycos[72-9-400.0-coe]
34.29s call     tests/test_polycos.py::test_generate_polycos[72-9-400.0-ao]
34.16s call     tests/test_polycos.py::test_generate_polycos[72-9-1400.0-ao]
33.98s call     tests/test_polycos.py::test_generate_polycos[72-9-1400.0-coe]
26.25s call     tests/test_grid.py::test_grid_3param_prefix_multiprocessor
25.64s call     tests/test_toa_reader.py::test_renumber_subset_reordered
25.04s call     tests/test_toa_reader.py::test_merge_indices
24.99s call     tests/test_toa_reader.py::test_merge_indices_excised

@dlakaplan
Copy link
Contributor Author

Many of those are slow because they are going from tim files to toas and back for many permutations. Individual tests are fast but there are many of them. I'm wary of changing them,

@dlakaplan dlakaplan changed the title WIP: Speed up tests Speed up some tests Oct 23, 2023
@dlakaplan
Copy link
Contributor Author

I think that this can still be worked on more generally, but as a minor PR this saves ~10 minutes from the CI testing and can probably be merged as-is.

@abhisrkckl abhisrkckl merged commit c86140e into nanograv:master Oct 23, 2023
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants