Imrpove BRLA get_min_sample_size() efficiency
#31
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
the brla get_min_sample_size() binary search gets very expensive for large contests because their total ballots were increasing the search space. After generating many minimum sample sizes for brla, I found the highest minimum sample size was 17 (for a 0.01% risk limit up to 10,000,000 ballots). So I hard-coded the right bound in the binary search to 30 (just in case) which significantly reduces the search space for all contest making this initialization much more efficient for large contests. I also added a test file to ensure the minimums are all still correct. The test file only goes to 500,000.
Codecov Report
@@ Coverage Diff @@
## master #31 +/- ##
==========================================
- Coverage 89.08% 88.74% -0.34%
==========================================
Files 18 18
Lines 1374 1386 +12
Branches 172 175 +3
==========================================
+ Hits 1224 1230 +6
- Misses 117 119 +2
- Partials 33 37 +4
Continue to review full report at Codecov.
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I had a thought about this a while ago and the conversation yesterday about minimum sample sizes reminded me about it.
The
get_min_sample_size()method inbrlagets very expensive for contests with large total ballot counts because it was increasing the binary search space and causing many unnecessary expensive distribution computations. I generated the following minimum sample sizes to determine if we could create a more informed search policy.As I expected, the minimum sample sizes from
brlado not have a large range. I hardcoded the binary search to look between 1 and 30 (instead of the maximum ballots to draw) which greatly improved the initialization time for large contests. I added a minimum sample size test file for the above data. I include all the risk limits, but only contests up to 500,000 total ballots.Updates