Tests passing, merge with devel_python3 #4
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
But is ~10x slower, likely due to seeking through file
Changed the implementation of Region_Reader to yield headers and seqs This manages to cut the memory (somehow from 60 to 1 MB) and the runtime from 2 minutes to 10 seconds (last commit at 13 minutes).
Changed filter2 to use numpy and the new yielded regions Execution is ~10s and uses 1.5 MB memory In addition to floating point precision differences, noticed difference in sorting of alt ids when the values are equal going form python 2 and 3. Handled differences in comparison scripts which format to 10 digits and sort ids when values are equal.
Changed operation of threshold scan to limit number of read throughs of the region files.
Limited changes as original method was fairly fast
Conflicts: code/analyze/filter_1_main.py code/analyze/filter_2_main.py code/analyze/filter_helpers.py code/analyze/id_regions_main.py code/analyze/predict.py code/analyze/predict_main.py code/analyze/summarize_region_quality.py Unit tests running, but failing on new implementation of args/gp
Mocked and encoded constants enough to get things passing
Added a yaml version of the config to replace global params and setup args files. Helper script clean_config performs lookups of referenced entries in config. Have not added to main methods yet.
Started moving predict methods into a Predictor class to clean up the long argument lists for performing a prediction. Need to test new code and get old tests passing again with minimal object
Changed the implementation of predict_main.py into click with support for the new yaml configuration file. Refactored predict into two main objects to simplify the main code. Added a README.md
Modified behavior with missing files to match previous implementation (continue). Currently matching original implementation on chromosome 1.
Added log file option and config. When set, a progress bar is displayed on the console with click.
Added class for adding region ids and integrated with the main click method. Created new class to hold configuration and handle setting logic, as that was heavily reused between main methods.
Move the validation code to the corresponding main objects Seeing that positions is required for summarize, went back and made that required.
Refactored summarize region quality main and supporting module with cleaner implementation. Added to main click method and all supporting unit tests.
Combined two filter steps, helper functions, and threshold sweep into a single file for further refactoring.
Continued refactoring of main methods onto filtering. Part of the changes saw a modification to the configuration object to simplify setting code into a more uniform interface. Have started checking flake8 on entire project, fixing occasionally.
Finished refactor and testing of summarize_strain_states Finished formatting code consistent with FLAKE8
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Had to revert some argument parsing to make unit test passing faster. Will work on replacing arg parsing with click/yaml.