Fine-tune the sample size for CMRR.py #136
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
So here's the idea: the amount of time it takes to run CMRR for small values of days_to_simulate is very small. Which means that we can increase the sample size without having to worry about CMRR taking too long to annoy the user. Instead of writing a whole bunch of else-if statements, I made a function that calculates the "best" sample size. Specifically, it calculates what sample size would make CMRR run roughly as long as it owuld with sample_size=4 and days_to_simulate=365. sample_size=4 and days_to_simulate=365 is used as the default, and then if days_to_simulate is lower, sample size is increased to make the total run time approximately constant.
Do you verify it? |
|
Only "on paper". I can't verify it without having a version of Anki that actually implements it. |
|
OK. I will test it later. |
|
I tested it in Python and it seems that my approximation provides a decent fit, but it could be better. I improved the coefficients. |
|
Here are the times with the new method for calculating the sample size: |
I don't know how I managed to delete "best" twice today
|
Ok, I changed the coefficients and the output value for I'm sure that this is ready to be merged now. |
|
Sorry to bother you. Maybe you need to update the workload graph again. I found that I forgot to set
|
|
Well, give me the raw data again. And make it start from 0.50. |
If I revert this PR, the width of the green area will be larger than current one, right? If so, I won't revert it. |
|
Alright. Release a new version of the Google Colab optimizer so I can test and see if graphs look ok. |
|
You can modify this code to install fsrs-optimizer from your GitHub repo and branch. 
Reference: https://stackoverflow.com/questions/20101834/pip-install-from-git-repo-branch |
|
The raw data: |
|
|
Don't mind me, I'm just trying things
|
Alright, should be good, you can merge it. Don't forget to add the new method of calculating the sample size to the Rust version. |
Ensure that nothing is plotted above the box with the graph
Merged
via the queue into
open-spaced-repetition:main
with commit Sep 9, 2024
aef1af1
1 check passed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
So here's the idea: the amount of time it takes to run CMRR for small values of days_to_simulate is very small. Which means that we can increase the sample size without having to worry about CMRR taking too long to annoy the user.
Instead of writing a whole bunch of else-if statements, I made a function that calculates the "best" sample size. Specifically, it calculates what sample size would make CMRR run roughly as long as it would with sample_size=4 and days_to_simulate=365.
sample_size=4 and days_to_simulate=365 is used as the default, and then if days_to_simulate is lower, sample size is increased to make the total run time approximately constant for days_to_simulate <=365.
You can run something like:
to see what sample size values it outputs for each value of days_to_simulate.
Don't forget to add this to FSRS-rs and merge it into Anki.