660/truncate feature values #661
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified lines are covered by tests ✅
📢 Thoughts on this report? Let us know!. |
tamarl08
reviewed
Oct 4, 2023
tamarl08
reviewed
Oct 4, 2023
tamarl08
reviewed
Oct 4, 2023
tamarl08
reviewed
Oct 4, 2023
damien2012eng
force-pushed
the
660/truncate_feature_values
branch
from
3299a63
to
7217c05
Compare
tamarl08
reviewed
Oct 4, 2023
tamarl08
reviewed
Oct 4, 2023
tamarl08
reviewed
Oct 4, 2023
|
Looks good @damien2012eng ! just added a couple of docstring nitpicks. |
tamarl08
approved these changes
Oct 4, 2023
desilinguist
requested changes
Oct 5, 2023
There was a problem hiding this comment.
@damien2012eng looks pretty good but it's still missing a critical piece. Do you remember how we check that the value of standardize_features specified for rsmexplain and in the original rsmtool experiment match and, if not, we raise a warning? We need to do this same for truncate_outliers as well.
Instead of duplicating code, may be factor out that code as a function in rsmexplain.py and use it for both fields?
damien2012eng
force-pushed
the
660/truncate_feature_values
branch
from
b301453
to
091fbfa
Compare
added 8 commits
October 5, 2023 15:40
damien2012eng
force-pushed
the
660/truncate_feature_values
branch
from
091fbfa
to
4d31f66
Compare
tamarl08
reviewed
Oct 5, 2023
tamarl08
reviewed
Oct 5, 2023
desilinguist
reviewed
Oct 5, 2023
Use a simple `for` loop to verify and overwrite the values of `standardize_features` and `truncate_outliers` for rsmexplain.
Add new tests to check that `truncate_outliers` values are correctly overwritten when necessary.
desilinguist
approved these changes
Oct 5, 2023
|
@damien2012eng don't forget to fix the file @tamarl08 pointed out. Otherwise this is ready to merge. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I verified the ASAP dataset using RSMTool toolset. The column mechanics has outlier values. Here is an example result (Response 181) for truncating and w/o truncating:
Truncating outlier: 3.213 (Original data) and 3.248 (preprocessed data)
W/O Truncating outlier: 3.213 (for both original data and preprocessed data)
I also verified the outliers range for the Mechanics: [3.248, 5.648]. Because the value 3.213 is lower than the lower bound, it is truncated to be the value of the lower bound.
Closes #660