Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add toggle for Graph Snapshot Testing #361

Merged
merged 2 commits into from
Jun 19, 2024
Merged

Add toggle for Graph Snapshot Testing #361

merged 2 commits into from
Jun 19, 2024

Conversation

gowerc
Copy link
Collaborator

@gowerc gowerc commented Jun 19, 2024

No corresponding issue.

Unit tests were failing on internal CICD build due to snapshot discrepancies, as far as I can tell this is minor white space stuff on graphs due to differences in package versions (I've posted the diff below). To address this I've added a toggle for graph snapshots which is enabled for the GitHub CICD but disabled for the in house builds.

I opted for a wrapper around the expect* function rather than skipping the test as this way it will still through an error on the internal build if the plot generation code fails its only the final comparison that is suppressed.

> diff  \
    survival-plot-no-wrap-no-ci-km-ggplot2.new.svg  \
    survival-plot-no-wrap-no-ci-km-ggplot2.svg
    
21c21
< <rect x='0.00' y='0.00' width='720.00' height='576.00' style='stroke-width: 1.07; stroke: #FFFFFF; fill: #FFFFFF;' />
---
> <rect x='0.000000000000064' y='0.00' width='720.00' height='576.00' style='stroke-width: 1.07; stroke: #FFFFFF; fill: #FFFFFF;' />

@gowerc gowerc requested a review from gravesti June 19, 2024 14:31
Copy link
Contributor

github-actions bot commented Jun 19, 2024

badge

Code Coverage Summary

Filename                           Stmts    Miss  Cover    Missing
-------------------------------  -------  ------  -------  --------------------------------
R/brier_score.R                      166       0  100.00%
R/DataJoint.R                         76       2  97.37%   264, 270
R/DataLongitudinal.R                 119       1  99.16%   244
R/DataSubject.R                       85       1  98.82%   142
R/DataSurvival.R                      98       0  100.00%
R/defaults.R                          10       6  40.00%   18-57, 84
R/generics.R                          31       4  87.10%   51, 395, 414, 449
R/Grid.R                              27       1  96.30%   173
R/GridEven.R                          32       0  100.00%
R/GridEvent.R                         22       0  100.00%
R/GridFixed.R                         30       0  100.00%
R/GridGrouped.R                       52       0  100.00%
R/GridManual.R                        23       3  86.96%   78-80
R/GridObserved.R                      20       0  100.00%
R/GridPopulation.R                    31       4  87.10%   61, 69-71
R/GridPrediction.R                    36       6  83.33%   83, 91-95
R/JointModel.R                       123       8  93.50%   141-143, 193, 197, 239, 285, 291
R/JointModelSamples.R                 59       0  100.00%
R/link_generics.R                     13       3  76.92%   59, 76, 91
R/Link.R                              62       4  93.55%   200-203
R/LinkComponent.R                     19       3  84.21%   88, 120-121
R/LongitudinalGSF.R                   82       0  100.00%
R/LongitudinalModel.R                 19       0  100.00%
R/LongitudinalQuantities.R            71       0  100.00%
R/LongitudinalRandomSlope.R           41       5  87.80%   93-97
R/LongitudinalSteinFojo.R             75       5  93.33%   143-147
R/Parameter.R                         14       0  100.00%
R/ParameterList.R                     42       1  97.62%   184
R/Prior.R                            236       8  96.61%   481, 577, 589-607
R/Promise.R                           25       0  100.00%
R/Quantities.R                        60       0  100.00%
R/QuantityGeneratorPopulation.R       22       0  100.00%
R/QuantityGeneratorPrediction.R       48       0  100.00%
R/QuantityGeneratorSubject.R          19       0  100.00%
R/settings.R                          12      12  0.00%    55-69
R/SimGroup.R                           5       0  100.00%
R/SimJointData.R                      72       1  98.61%   103
R/SimLongitudinal.R                    5       2  60.00%   22, 40
R/SimLongitudinalGSF.R                53       0  100.00%
R/SimLongitudinalRandomSlope.R        42       0  100.00%
R/SimLongitudinalSteinFojo.R          50       0  100.00%
R/SimSurvival.R                      104       0  100.00%
R/StanModel.R                         15       0  100.00%
R/StanModule.R                       176       6  96.59%   192-193, 235, 246, 383, 411
R/SurvivalExponential.R               10       0  100.00%
R/SurvivalLoglogistic.R               11       0  100.00%
R/SurvivalModel.R                     19       0  100.00%
R/SurvivalQuantities.R               170       1  99.41%   125
R/SurvivalWeibullPH.R                 11       0  100.00%
R/utilities.R                        143       1  99.30%   13
R/zzz.R                               28      25  10.71%   3, 8-12, 14-35, 47
TOTAL                               2814     113  95.98%

Diff against main

Filename      Stmts    Miss  Cover
----------  -------  ------  --------
TOTAL             0       0  +100.00%

Results for commit: a36dbdc

Minimum allowed coverage is 80%

♻️ This comment has been updated with latest results

Copy link
Contributor

github-actions bot commented Jun 19, 2024

Unit Tests Summary

  1 files  140 suites   7m 54s ⏱️
131 tests 128 ✅ 3 💤 0 ❌
976 runs  973 ✅ 3 💤 0 ❌

Results for commit a36dbdc.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Jun 19, 2024

Unit Test Performance Difference

Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
compile 💚 $3.12$ $-1.79$ $0$ $0$ $0$ $0$
extract_quantities 💔 $9.87$ $+1.86$ $0$ $0$ $0$ $0$
model_multi_chain 💚 $35.39$ $-2.07$ $0$ $0$ $0$ $0$
Additional test case details
Test Suite $Status$ Time on main $±Time$ Test Case
Grid 💔 $25.20$ $+1.45$ Grid_objects_work_with_QuantityGenerator_and_QuantityCollapser
GridPrediction 💔 $157.23$ $+1.73$ GridPrediction_works_as_expected_for_Survival_models
SurvivalExponential 💔 $13.97$ $+1.54$ SurvivalExponential_can_recover_true_parameter_including_covariates_
SurvivalLoglogistic 💚 $17.28$ $-1.42$ sim_os_loglogistic_is_consistant_with_flexsurv
SurvivalQuantities 💔 $11.76$ $+1.88$ SurvivalQuantities_and_autoplot.SurvivalQuantities_works_as_expected
brierScore 💔 $34.89$ $+1.57$ brierScore_SurvivalQuantities_returns_same_results_as_survreg
compile 💚 $3.12$ $-1.79$ compileStanModel_doesn_t_error_if_the_directory_doesn_t_exist
extract_quantities 💔 $9.87$ $+1.86$ extract_quantities_works_as_expected
model_multi_chain 💚 $35.39$ $-2.07$ Can_recover_known_distribution_parameters_from_random_slope_model_when_using_multiple_chains
stan_functions 💔 $16.14$ $+2.34$ GSF_SLD_function_works_as_expected

Results for commit 14a09bd

♻️ This comment has been updated with latest results.

Copy link
Collaborator

@gravesti gravesti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one suggestion. Maybe you already considered it? @gowerc

tests/testthat/test-survival_plot.R Show resolved Hide resolved
@gowerc gowerc enabled auto-merge (squash) June 19, 2024 15:54
@gowerc gowerc merged commit 885eaad into main Jun 19, 2024
21 checks passed
@gowerc gowerc deleted the xxx-fix-graph-snapshots branch June 19, 2024 16:17
@gowerc gowerc mentioned this pull request Jun 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants