Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update CI that compare sampled quantities #419

Closed
SamuelBrand1 opened this issue Aug 14, 2024 · 2 comments
Closed

Update CI that compare sampled quantities #419

SamuelBrand1 opened this issue Aug 14, 2024 · 2 comments
Labels
enhancement New feature or request EpiAware

Comments

@SamuelBrand1
Copy link
Collaborator

          I got irritated by a stochastic CI failure on how different the sample variance is from sampling from `SafeNegativeBinomial` vs `NegativeBinomial` so I made it more principled test based on the std of the variance estimator.

Originally posted by @SamuelBrand1 in #418 (comment)

The idea here is that we should relate a "bad" failure that indicates a high probability of a miscoded sampler to the properties of the estimator being tested. We've had intermittent failure on doing this for NegativeBinomial sampling (iirc) but thats to be expected because the variance of each draw is fairly high and therefore the std of the population variance estimator is fairly high and is competitive against the $O(1/\sqrt{n})$ factor.

@SamuelBrand1 SamuelBrand1 added enhancement New feature or request EpiAware labels Aug 14, 2024
@seabbs
Copy link
Collaborator

seabbs commented Aug 15, 2024

sounds good agree

@SamuelBrand1
Copy link
Collaborator Author

I think we've covered this problem so closing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request EpiAware
Projects
None yet
Development

No branches or pull requests

2 participants