36 make pandasstandardscaler for hugs #38
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Inital upload of a standard scaler that supports pandas dataframes as input. Maybe it should be renamed to DataFrame scaler to match the parser_to_dataframe.py file...
Using scikit-learn's StandardScaler as a base. Saving and loading is a bit tricky as the internal state of the scikit-learn implementation isn't easily saved. It looks like you can save it's internal __dict__ and then set the attributes on load, but I'm not sure if this is robust...
The implementation avoids using scikit-learn entirely for a number of reasons including no option for axis changes and no option for changing the epsilon value when dealing with small variances.
Saving and loading configurations are now done by utility functions in the utils folder. This simplifies the modules and allows for unit testing on the config I/O functions outside of any module. Utility functions try to properly handle FileNotFound and FileExists errors.
[save/load]_config --> [save/load]_yaml_config to avoid confusion if we want another version for JSON or something else.
Adding lots of additional documentation. Some functions aren't fully documented, but this is a very good start.
sgoldenCS
requested review from
dlersch,
Kishanrajput,
schr476 and
ahmedmohammed107
Unittests still work and changes seem to mostly be single quotes to double and removing spaces.
This reverts commit f759eba.
|
Reviewed code. looks good to me. going to approve. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Creates a StandardScaler for Pandas DataFrames with unittests for both the scaler and a few new I/O utility functions for YAML configuration files.
Also, includes a small update to the Parser2DataFrame to include the new I/O behavior. All unittests still work. This probably should have been its own issue/pull, but the changes were very minimal.
This will close #36 as complete.
We may want to think about whether we should rename run() and reverse() with transform() and inverse_transform() which matches Scikit-Learn.