👷 Disabled running computationally intensive unit tests via the CI in Debug build mode #486

Drewniok · 2024-07-29T11:27:01Z

Description

This PR moves all computationally intensive unit tests into a separate test suite called quality. These tests will only run in release mode within the CI pipeline, accelerating the workflow (> 20 % runtime improvement for macos CI).

Checklist:

The pull request only contains commits that are related to it.
I have added appropriate tests and documentation.
I have added a changelog entry.
I have created/adjusted the Python bindings for any new or updated functionality.
I have made sure that all CI jobs on GitHub pass.
The pull request introduces no new warnings and follows the project's style guidelines.

…y_tag

codecov · 2024-07-29T13:17:19Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.22%. Comparing base (8f61ecc) to head (4b3b9e1).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #486      +/-   ##
==========================================
- Coverage   98.22%   98.22%   -0.01%     
==========================================
  Files         216      217       +1     
  Lines       29164    29143      -21     
  Branches     1392     1391       -1     
==========================================
- Hits        28647    28625      -22     
- Misses        517      518       +1

Files	Coverage Δ
.../algorithms/simulation/sidb/operational_domain.hpp	`96.01% <100.00%> (-0.30%)`	⬇️
...fiction/technology/charge_distribution_surface.hpp	`99.36% <100.00%> (+<0.01%)`	⬆️
...t/algorithms/physical_design/design_sidb_gates.cpp	`100.00% <100.00%> (ø)`
test/algorithms/quality.cpp	`100.00% <100.00%> (ø)`
...tion/sidb/assess_physical_population_stability.cpp	`100.00% <100.00%> (ø)`
...lgorithms/simulation/sidb/critical_temperature.cpp	`100.00% <100.00%> (ø)`
...ion/sidb/determine_physically_valid_parameters.cpp	`100.00% <100.00%> (ø)`
test/algorithms/simulation/sidb/is_operational.cpp	`100.00% <ø> (ø)`
...maximum_defect_influence_position_and_distance.cpp	`100.00% <100.00%> (ø)`
test/algorithms/simulation/sidb/quickexact.cpp	`99.55% <100.00%> (-0.02%)`	⬇️

... and 1 file with indirect coverage changes

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8f61ecc...4b3b9e1. Read the comment docs.

test/algorithms/simulation/sidb/quickexact.cpp

@@ -1639,8 +1623,7 @@
    }
 }

-TEMPLATE_TEST_CASE("Special test cases", "[quickexact]", (sidb_100_cell_clk_lyt_siqad),
-                   (cds_sidb_100_cell_clk_lyt_siqad))
+TEMPLATE_TEST_CASE("Special test cases", "[quickexact]", sidb_100_cell_clk_lyt_siqad)


test/algorithms/simulation/sidb/assess_physical_population_stability.cpp

+TEMPLATE_TEST_CASE("Bestagon AND gate", "[assess-physical-population-stability]", sidb_cell_clk_lyt_siqad,
+                   cds_sidb_cell_clk_lyt_siqad)


test/algorithms/simulation/sidb/critical_temperature.cpp

+TEMPLATE_TEST_CASE("Test critical_temperature function", "[critical-temperature]", sidb_100_cell_clk_lyt_siqad,
+                   cds_sidb_100_cell_clk_lyt_siqad)


marcelwa · 2024-07-30T06:52:18Z

@Drewniok running expensive tests in Release mode only, is a worthwhile addition. Thanks a lot! I don't understand why they had to be moved to a separate file. By assigning them an additional tag [quality] on top of their original tag, we can achieve the same result I think. This way, everything can stay in place and thus logically sorted.

Drewniok · 2024-07-30T06:56:40Z

@Drewniok running expensive tests in Release mode only, is a worthwhile addition. Thanks a lot! I don't understand why they had to be moved to a separate file. By assigning them an additional tag [quality] on top of their original tag, we can achieve the same result I think. This way, everything can stay in place and thus logically sorted.

That's a good point! I think, I just did it like in mockturtle. But I agree, it's better to remove the separate file again. Thanks!

…y_tag

test/algorithms/simulation/sidb/assess_physical_population_stability.cpp

-    "Bestagon CROSSING gate input 11, using siqad coordinates", "[assess-physical-population-stability]",
-    (cell_level_layout<sidb_technology, clocked_layout<cartesian_layout<siqad::coord_t>>>),
-    (charge_distribution_surface<cell_level_layout<sidb_technology, clocked_layout<cartesian_layout<siqad::coord_t>>>>))
+TEST_CASE("Bestagon CX gate input 11", "[assess-physical-population-stability], [quality]")


marcelwa

Many thanks! 🙏

Drewniok added 3 commits July 29, 2024 13:20

✅ Move all computationally expensive unit tests to quality.cpp

4d5522b

🎨 small fix.

df55738

👷 execute quality unit test only in release mode.

81367c6

Drewniok added the performance Performance improvements label Jul 29, 2024

Drewniok self-assigned this Jul 29, 2024

Drewniok marked this pull request as draft July 29, 2024 11:27

pre-commit-ci bot and others added 6 commits July 29, 2024 11:27

🎨 Incorporated pre-commit fixes

7014682

👷 small fix.

1f882a1

🎨 Incorporated pre-commit fixes

dd018eb

👷 small fix.

729d060

Merge remote-tracking branch 'origin/add_quality_tag' into add_qualit…

06609b6

…y_tag

👷 small fix.

4b3b9e1

Drewniok marked this pull request as ready for review July 29, 2024 12:28

Drewniok requested a review from marcelwa July 29, 2024 12:28

Drewniok changed the title ~~👷 Move computationally intensive unit tests to a separate test file~~ 👷 Moved computationally intensive unit tests to a separate test file Jul 29, 2024

github-advanced-security bot found potential problems Jul 29, 2024

View reviewed changes

Drewniok and others added 3 commits July 30, 2024 09:22

🎨 use an additional tag instead of a separate file

59f6b7f

Merge remote-tracking branch 'origin/add_quality_tag' into add_qualit…

78c544b

…y_tag

🎨 Incorporated pre-commit fixes

6019b02

github-advanced-security bot found potential problems Jul 30, 2024

View reviewed changes

marcelwa approved these changes Jul 30, 2024

View reviewed changes

marcelwa added the github_actions Pull requests that update GitHub Actions code label Jul 30, 2024

marcelwa changed the title ~~👷 Moved computationally intensive unit tests to a separate test file~~ 👷 Disabled running computationally intensive unit tests via the CI in Debug build mode Jul 30, 2024

marcelwa merged commit 8dbfa45 into cda-tum:main Jul 30, 2024
57 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

👷 Disabled running computationally intensive unit tests via the CI in Debug build mode #486

👷 Disabled running computationally intensive unit tests via the CI in Debug build mode #486

Drewniok commented Jul 29, 2024 •

edited

Loading

codecov bot commented Jul 29, 2024

marcelwa commented Jul 30, 2024

Drewniok commented Jul 30, 2024

marcelwa left a comment

		TEMPLATE_TEST_CASE("Bestagon AND gate", "[assess-physical-population-stability]", sidb_cell_clk_lyt_siqad,
		cds_sidb_cell_clk_lyt_siqad)

		TEMPLATE_TEST_CASE("Test critical_temperature function", "[critical-temperature]", sidb_100_cell_clk_lyt_siqad,
		cds_sidb_100_cell_clk_lyt_siqad)

👷 Disabled running computationally intensive unit tests via the CI in Debug build mode #486

👷 Disabled running computationally intensive unit tests via the CI in Debug build mode #486

Conversation

Drewniok commented Jul 29, 2024 • edited Loading

Description

Checklist:

codecov bot commented Jul 29, 2024

Codecov Report

marcelwa commented Jul 30, 2024

Drewniok commented Jul 30, 2024

marcelwa left a comment

Choose a reason for hiding this comment

Drewniok commented Jul 29, 2024 •

edited

Loading