Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make pipeline inference tests more focused #475

Conversation

SamuelBrand1
Copy link
Collaborator

This PR rationalises the tests in the pipeline to make it easier to identify the error in the forecast step.

Closes #474

From this branch

 function make_inference(pipeline)
        truthdata_dg_task = do_truthdata(pipeline)
        truthdata = fetch.(truthdata_dg_task)
        do_inference(truthdata[1], pipeline)
    end

        pipeline = SmoothOutbreakPipeline(; ndraws = 20, nchains = 1, testmode = true)
        inference_results_tsk = make_inference(pipeline)
        inference_results = fetch.(inference_results_tsk)

inference_results[1]["forecast_results"]
#InexactError(:Int64, Int64, 26256623853999718400)

Which focuses the issue towards wild sampling (probably due to very badly specified priors).

Copy link
Contributor

github-actions bot commented Oct 4, 2024

Try this Pull Request!

Open Julia and type:

import Pkg
Pkg.activate(temp=true)
Pkg.add(url="https://github.com/CDCgov/Rt-without-renewal", rev="fix-pipeline-test", subdir="EpiAware")
using EpiAware

@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.02%. Comparing base (f93efeb) to head (b47309e).

Additional details and impacted files
@@                                        Coverage Diff                                        @@
##           404-upgrade-the-testmode-pipeline-script-to-just-pipeline-script     #475   +/-   ##
=================================================================================================
  Coverage                                                             90.02%   90.02%           
=================================================================================================
  Files                                                                    57       57           
  Lines                                                                   742      742           
=================================================================================================
  Hits                                                                    668      668           
  Misses                                                                   74       74           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@seabbs seabbs merged commit 657ee3c into 404-upgrade-the-testmode-pipeline-script-to-just-pipeline-script Oct 4, 2024
9 of 10 checks passed
@seabbs seabbs deleted the fix-pipeline-test branch October 4, 2024 13:53
github-merge-queue bot pushed a commit that referenced this pull request Oct 7, 2024
…dictive + full (#445)

* remove most compat from pipeline

* Update test_pipelinefunctions.jl

* Fix the truthdata output to be missing or Int

* change to daily increments in latent processes

* Change default AD mode to ReverseDiff{true}

* Create changelog.md

* reformat

* change filename

* Update simulate.jl

* add a testmode to pipelinetypes

* tighten typing

* fix constructor

* fix constructor

* pipeline helper functions

* unit tests and end-to-end tests

* remove old scripts

* Fix y_t type

* bring inference prefixing into line with truthdata prefixing

* reformat

* interim commit

* rename and collect plot functions

* tidy up tests

* basic plots to Makie

* new plot tests and test reorganisation

* Update runtests.jl

* reformat

* 455 plotting methods of prior predictive (#462)

* add priorpredictive method to make_inference

* Add pipeline priorpredictive boolean

* reformat

* remove inference_method kwarg because can be dispatched on

* remove specialisation on forecast results

add missing handling as well

* move inference step into own function

and give fail cover with error report

* remove passing inference_method

* add a latent model name to InferenceConfig

* Util for setting up PI levels

* prior pred plot

* export prior_predictive_plot

* remove dead end-to-end test

in favour of direct test of prior_predictive_plot

* Make pipeline inference tests more focused (#475)

* fix tests and capture forecasting failures

* delete stale end-to-end test

* Full priorpred check pipeline script (#476)

* Hotfix: Doc link to main

* create xexpy function with x = 1 typestable (#473)

* rename data folder

* catch bug where objects were getting saved to root dir

* fix bug where being saved to root

* add priorpredictive mode to the inference config struct

* do_truthdata behaviour modifed by prior predictive

* send priorpredictive mode in generate_inference

* update tests

* prior predictive pipeline

* reformat

---------

Co-authored-by: Sam Abbott <[email protected]>

* Add oneexpy to pipeline (#480)

* Changed saved prior predictive data (#478)

* change to pipeline being part of InferenceConfig

This also splits figure saving from data saving

* fix passed var

* change serialisation target dir

* update tests and config usage to pass pipeline object

---------

Co-authored-by: Sam Abbott <[email protected]>

* Update run_priorpred_pipeline.jl

* Only save strings rather than Exception objects

---------

Co-authored-by: Sam Abbott <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants