Use joblib for parallelism in regress_out #1695
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ivirshup
changed the title
|
@meeseeksdev backport to 1.7.x |
meeseeksmachine
pushed a commit
to meeseeksmachine/scanpy
that referenced
this pull request
Mar 3, 2021
ivirshup
added a commit
that referenced
this pull request
Mar 3, 2021
Co-authored-by: Isaac Virshup <[email protected]>
Zethson
pushed a commit
that referenced
this pull request
Mar 15, 2021
* Use joblib for parallism in regress_out * release note * fix link in release notes * Add todo for resource test
ivirshup
added a commit
that referenced
this pull request
Mar 18, 2021
* add flake8 pre-commit Signed-off-by: Zethson <[email protected]> * fix pre-commit Signed-off-by: Zethson <[email protected]> * add E402 to flake8 ignore Signed-off-by: Zethson <[email protected]> * revert neighbors Signed-off-by: Zethson <[email protected]> * fix flake8 Signed-off-by: Zethson <[email protected]> * address review Signed-off-by: Zethson <[email protected]> * fix comment character in .flake8 Signed-off-by: Zethson <[email protected]> * fix test Signed-off-by: Zethson <[email protected]> * black Signed-off-by: Zethson <[email protected]> * review round 2 Signed-off-by: Zethson <[email protected]> * review round 3 Signed-off-by: Zethson <[email protected]> * readded double comments Signed-off-by: Zethson <[email protected]> * Ignoring E262 & reverted comment Signed-off-by: Zethson <[email protected]> * using self for obs_tidy Signed-off-by: Zethson <[email protected]> * Restore setup.py * rm call of black test (#1690) * Fix print_versions for python<3.8 (#1691) * add codecov so we can have a badge to point to (#1693) * Attempt server-side search (#1672) * Fix paga_path (#1047) Fix paga_path Co-authored-by: Isaac Virshup <[email protected]> * Switch to flit This reverts commit d645790 * add setup.py while leaving it ignored * Update install instructions * Circumvent new pip check (see pypa/pip#9628) * Go back to regular pip (#1702) * Go back to regular flit Co-authored-by: Isaac Virshup <[email protected]> * codecov comment (#1704) * Use joblib for parallelism in regress_out (#1695) * Use joblib for parallism in regress_out * release note * fix link in release notes * Add todo for resource test * Add sparsificiation step before sparse-dependent Scrublet calls (#1707) * Add sparsificiation step before sparse-dependent Scrublet calls * Apply sparsification suggestion Co-authored-by: Isaac Virshup <[email protected]> * Fix imports Co-authored-by: Isaac Virshup <[email protected]> * Fix version on Travis (#1713) By default, Travis does `git clone --depth=50` which means the version can’t be detected from the git tag. * `sc.metrics` module (add confusion matrix & Geary's C methods) (#915) * Add `sc.metrics` with `gearys_c` Add a module for computing useful metrics. Started off with Geary's C since I'm using it and finding it useful. I've also got a fairly fast way to calculate it worked out. Unfortunatly my implementation runs into some issues with some global configs set by umap (see lmcinnes/umap#306), so I'm going to see if that can be resolved before changing it. * Add sc.metrics.confusion_matrix * Better tests and output for confusion_matrix * Workaround umap<0.4 and increase numerical stability of gearys_c * Work around lmcinnes/umap#306 by not calling out to kernel function. That code has been kept, but commented out. * Increase numerical stability by casting data to system width. Tests were failing due to instability. * Split up gearys_c tests * Improved unexpected error message * gearys_c working again. Sadly, a bit slower * One option for doc strings * Simplify implementation to use single dispatch * release notes * Fix clipped images in docs (#1717) * Cleanup normalize_total (#1667) * Cleanup normalize_total * Add modification tests and copy kwarg for normalize_total * Test that 'layers' argument is deprecated * Added more mutation checks for normalize_total * release note * Error message * deprecate scvi (#1703) * deprecate scvi * Update .azure-pipelines.yml Co-authored-by: Isaac Virshup <[email protected]> * remove :func: links to scvi in release notes * remove tildes in front of scvi in release notes * Update docs/release-notes/1.5.0.rst Co-authored-by: Michael Jayasuriya <[email protected]> Co-authored-by: Isaac Virshup <[email protected]> * updated ecosystem.rst to add triku (#1722) * Minor addition to contributing docs (#1726) * Preserve category order when groupby is a list (#1735) Preserve category order when groupby is a list * Asymmetrical diverging colormaps and vcenter (#1551) Add vcenter and norm arguments to plotting functions * add flake8 pre-commit Signed-off-by: Zethson <[email protected]> * add E402 to flake8 ignore Signed-off-by: Zethson <[email protected]> * revert neighbors Signed-off-by: Zethson <[email protected]> * address review Signed-off-by: Zethson <[email protected]> * black Signed-off-by: Zethson <[email protected]> * using self for obs_tidy Signed-off-by: Zethson <[email protected]> * rebased Signed-off-by: Zethson <[email protected]> * rebasing Signed-off-by: Zethson <[email protected]> * rebasing Signed-off-by: Zethson <[email protected]> * rebasing Signed-off-by: Zethson <[email protected]> * add flake8 to dev docs Signed-off-by: Zethson <[email protected]> * add autopep8 to pre-commits Signed-off-by: Zethson <[email protected]> * add flake8 ignore docs Signed-off-by: Zethson <[email protected]> * add exception todos Signed-off-by: Zethson <[email protected]> * add ignore directories Signed-off-by: Zethson <[email protected]> * reinstated lambdas Signed-off-by: Zethson <[email protected]> * fix tests Signed-off-by: Zethson <[email protected]> * fix tests Signed-off-by: Zethson <[email protected]> * fix tests Signed-off-by: Zethson <[email protected]> * fix tests Signed-off-by: Zethson <[email protected]> * fix tests Signed-off-by: Zethson <[email protected]> * Add E741 to allowed flake8 violations. Co-authored-by: Isaac Virshup <[email protected]> * Add F811 flake8 ignore for tests Co-authored-by: Isaac Virshup <[email protected]> * Fix mask comparison Co-authored-by: Isaac Virshup <[email protected]> * Fix mask comparison Co-authored-by: Isaac Virshup <[email protected]> * fix flake8 config file Signed-off-by: Zethson <[email protected]> * readded autopep8 Signed-off-by: Zethson <[email protected]> * import Literal Signed-off-by: Zethson <[email protected]> * revert literal import Signed-off-by: Zethson <[email protected]> * fix scatterplot pca import Signed-off-by: Zethson <[email protected]> * false comparison & unused vars Signed-off-by: Zethson <[email protected]> * Add cleaner level determination Co-authored-by: Isaac Virshup <[email protected]> * Fix comment formatting Co-authored-by: Isaac Virshup <[email protected]> * Add smoother dev documentation Co-authored-by: Isaac Virshup <[email protected]> * fix flake8 Signed-off-by: Zethson <[email protected]> * Readd long comment Co-authored-by: Isaac Virshup <[email protected]> * Assuming X as array like Co-authored-by: Isaac Virshup <[email protected]> * fix flake8 Signed-off-by: Zethson <[email protected]> * fix flake8 config Signed-off-by: Zethson <[email protected]> * reverted rank_genes Signed-off-by: Zethson <[email protected]> * fix disp_mean_bin formatting Co-authored-by: Isaac Virshup <[email protected]> * fix formatting Signed-off-by: Zethson <[email protected]> * add final todos Signed-off-by: Zethson <[email protected]> * boolean checks with is Signed-off-by: Zethson <[email protected]> * _dpt formatting Signed-off-by: Zethson <[email protected]> * literal fixes Signed-off-by: Zethson <[email protected]> * links to leafs Signed-off-by: Zethson <[email protected]> * revert paga variable naming Co-authored-by: Philipp A <[email protected]> Co-authored-by: Sergei Rybakov <[email protected]> Co-authored-by: Isaac Virshup <[email protected]> Co-authored-by: Jonathan Manning <[email protected]> Co-authored-by: mjayasur <[email protected]> Co-authored-by: Michael Jayasuriya <[email protected]> Co-authored-by: Alex M. Ascensión <[email protected]> Co-authored-by: Gökçen Eraslan <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fixes #1396
Branch which I had sitting on my computer, but forgot to open a PR for (doh).
Uses joblib to parallelize
regress_outinstead ofmultiprocessingin order to prevent oversubscription of threads.Not sure how to test this.
Should I also allow passing more arguments to joblib?