Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix SF Quantities #370

Merged
merged 9 commits into from
Jun 26, 2024
Merged

Fix SF Quantities #370

merged 9 commits into from
Jun 26, 2024

Conversation

gowerc
Copy link
Collaborator

@gowerc gowerc commented Jun 25, 2024

Closes #368

Ok so the actual bug fix was trivial was just a case of fixing a simple typo e.g. gsf -> sf. What was a bit more involved though was updating the unit tests to actually catch this error. It would have been easy to just compile all of the models but this would have added a ton of run time especially if we want to test each permutation of model / quantity generation.

Instead I decided to abstract the code that generates the StanModule for the quantities into its own function so that we can just grab that object before its actually compiled. I then put together a mock JointModelSamples object to which I then use to generate the corresponding StanModule and then run the stan parser to make sure theres no obviously variable errors.

I then added the unit test to our other longitudinal models to make sure no other bugs like this exist.

A bit of a red herring in these changes was that my original implementation tried to expand on the write_stan() function. I adapted the function to take a connection stream to which I was going to pipe the output to a variable. I abandoned this implementation for testing this PR but I thought having the function take a stream was a nice change so left it in as I had already written the code (hope thats ok).

@gowerc gowerc requested a review from gravesti June 25, 2024 15:29
@gowerc
Copy link
Collaborator Author

gowerc commented Jun 25, 2024

@gravesti - The "files changed" tab on GitHub here is miss-leading; as I've moved a chunk of code (without modification) between 2 functions the diff is just showing the difference between the code chunks immediately before the chunk I moved. I would potentially recommend viewing the complete file and not just the diff.

Copy link
Contributor

github-actions bot commented Jun 25, 2024

badge

Code Coverage Summary

Filename                           Stmts    Miss  Cover    Missing
-------------------------------  -------  ------  -------  --------------------------------
R/brier_score.R                      167       0  100.00%
R/DataJoint.R                         76       2  97.37%   264, 270
R/DataLongitudinal.R                 119       1  99.16%   244
R/DataSubject.R                       85       1  98.82%   142
R/DataSurvival.R                      98       0  100.00%
R/defaults.R                          10       6  40.00%   18-57, 84
R/generics.R                          33       4  87.88%   52, 396, 416, 471
R/Grid.R                              27       1  96.30%   173
R/GridEven.R                          32       0  100.00%
R/GridEvent.R                         22       0  100.00%
R/GridFixed.R                         30       0  100.00%
R/GridGrouped.R                       52       0  100.00%
R/GridManual.R                        23       3  86.96%   78-80
R/GridObserved.R                      20       0  100.00%
R/GridPopulation.R                    31       4  87.10%   61, 69-71
R/GridPrediction.R                    36       6  83.33%   83, 91-95
R/JointModel.R                       126      10  92.06%   146-151, 201, 205, 247, 293, 299
R/JointModelSamples.R                 66       0  100.00%
R/link_generics.R                     13       3  76.92%   59, 76, 91
R/Link.R                              62       4  93.55%   200-203
R/LinkComponent.R                     19       3  84.21%   88, 120-121
R/LongitudinalClaretBruno.R           83       6  92.77%   156-160, 175
R/LongitudinalGSF.R                   83       0  100.00%
R/LongitudinalModel.R                 19       0  100.00%
R/LongitudinalQuantities.R            71       0  100.00%
R/LongitudinalRandomSlope.R           42       5  88.10%   98-102
R/LongitudinalSteinFojo.R             76       5  93.42%   149-153
R/Parameter.R                         14       0  100.00%
R/ParameterList.R                     42       1  97.62%   184
R/Prior.R                            236       8  96.61%   481, 577, 589-607
R/Promise.R                           25       0  100.00%
R/Quantities.R                        60       0  100.00%
R/QuantityGeneratorPopulation.R       22       0  100.00%
R/QuantityGeneratorPrediction.R       48       0  100.00%
R/QuantityGeneratorSubject.R          19       0  100.00%
R/settings.R                          12      12  0.00%    55-69
R/SimGroup.R                           5       0  100.00%
R/SimJointData.R                      72       1  98.61%   103
R/SimLongitudinal.R                    5       2  60.00%   22, 40
R/SimLongitudinalClaretBruno.R        52       0  100.00%
R/SimLongitudinalGSF.R                53       0  100.00%
R/SimLongitudinalRandomSlope.R        42       0  100.00%
R/SimLongitudinalSteinFojo.R          50       0  100.00%
R/SimSurvival.R                      104       0  100.00%
R/StanModel.R                         15       0  100.00%
R/StanModule.R                       176       6  96.59%   192-193, 235, 246, 383, 411
R/SurvivalExponential.R               10       0  100.00%
R/SurvivalLoglogistic.R               11       0  100.00%
R/SurvivalModel.R                     19       0  100.00%
R/SurvivalQuantities.R               170       1  99.41%   125
R/SurvivalWeibullPH.R                 11       0  100.00%
R/utilities.R                        144       2  98.61%   13, 335
R/zzz.R                               28      25  10.71%   3, 8-12, 14-35, 47
TOTAL                               2966     122  95.89%

Diff against main

Filename                     Stmts    Miss  Cover
-------------------------  -------  ------  --------
R/JointModel.R                  +1      +2  -1.54%
R/JointModelSamples.R           +3       0  +100.00%
R/LongitudinalSteinFojo.R        0      -1  +1.32%
R/utilities.R                   +1      +1  -0.69%
TOTAL                           +5      +2  -0.06%

Results for commit: 627b514

Minimum allowed coverage is 80%

♻️ This comment has been updated with latest results

Copy link
Collaborator

@gravesti gravesti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, @gowerc
just one bug I think

R/JointModel.R Outdated Show resolved Hide resolved
@gowerc
Copy link
Collaborator Author

gowerc commented Jun 26, 2024

Updated to add the same test for the new claretBruno model which was merged in this morning

Copy link
Contributor

github-actions bot commented Jun 26, 2024

Unit Tests Summary

    1 files    154 suites   6m 44s ⏱️
  136 tests   133 ✅ 3 💤 0 ❌
1 021 runs  1 017 ✅ 4 💤 0 ❌

Results for commit 627b514.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Jun 26, 2024

Unit Test Performance Difference

Test Suite $Status$ Time on main $±Time$ $±Tests$ $±Skipped$ $±Failures$ $±Errors$
JointModelSamples 💚 $32.29$ $-1.47$ $0$ $0$ $0$ $0$
compile 💚 $6.56$ $-5.45$ $0$ $0$ $0$ $0$
extract_quantities 💔 $0.87$ $+2.38$ $0$ $0$ $0$ $0$
model_multi_chain 💚 $7.88$ $-2.29$ $0$ $0$ $0$ $0$
Additional test case details
Test Suite $Status$ Time on main $±Time$ Test Case
Grid 💚 $17.25$ $-4.32$ Grid_objects_work_with_QuantityGenerator_and_QuantityCollapser
GridPrediction 💚 $138.55$ $-4.87$ GridPrediction_works_as_expected_for_Survival_models
JointModelSamples 💚 $32.29$ $-1.47$ print_works_as_expected_for_JointModelSamples
LongitudinalClaretBruno 💔 $3.24$ $+1.19$ LongitudinalClaretBruno_works_as_expected_with_default_arguments
LongitudinalClaretBruno 👶 $+0.01$ Quantity_models_pass_the_parser
LongitudinalGSF 💔 $2.12$ $+1.47$ LongitudinalGSF_works_as_expected_with_default_arguments
LongitudinalGSF 👶 $+0.02$ Quantity_models_pass_the_parser
LongitudinalQuantiles 💚 $49.51$ $-1.97$ Test_that_LongitudinalQuantities_works_as_expected
LongitudinalRandomSlope 💔 $15.26$ $+3.74$ Print_method_for_LongitudinalRandomSlope_works_as_expected
LongitudinalRandomSlope 👶 $+0.00$ Quantity_models_pass_the_parser
LongitudinalSteinFojo 👶 $+0.03$ Quantity_models_pass_the_parser
SurvivalLoglogistic 💚 $9.53$ $-4.49$ sim_os_loglogistic_is_consistant_with_flexsurv
SurvivalQuantities 💔 $24.48$ $+5.64$ SurvivalQuantities_and_autoplot.SurvivalQuantities_works_as_expected
brierScore 💔 $3.84$ $+1.83$ brierScore_SurvivalQuantities_returns_same_results_as_survreg
compile 💚 $6.56$ $-5.45$ compileStanModel_doesn_t_error_if_the_directory_doesn_t_exist
extract_quantities 💔 $0.87$ $+2.38$ extract_quantities_works_as_expected
model_multi_chain 💚 $7.88$ $-2.29$ Can_recover_known_distribution_parameters_from_random_slope_model_when_using_multiple_chains
stan_functions 💔 $25.55$ $+1.17$ GSF_SLD_function_works_as_expected

Results for commit 8581191

♻️ This comment has been updated with latest results.

gravesti
gravesti previously approved these changes Jun 26, 2024
@gowerc gowerc merged commit a28d91b into main Jun 26, 2024
24 checks passed
@gowerc gowerc deleted the 368-fix-sf-quantities branch June 26, 2024 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

SF code refers to GSF variables
2 participants