Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Faster indexing #456

Closed
wants to merge 7 commits into from
Closed

[WIP] Faster indexing #456

wants to merge 7 commits into from

Conversation

luizirber
Copy link
Member

@luizirber luizirber commented Mar 21, 2018

  • Split nodes into internal and leaves
  • allow adding leaves without updating internal nodes (trigger it later)

Checklist

  • Is it mergeable?
  • make test Did it pass the tests?
  • make coverage Is the new code covered?
  • Did it change the command-line interface? Only additions are allowed
    without a major version increment. Changing file formats also requires a
    major version number increment.
  • Was a spellchecker run on the source code and documentation after
    changes were made?

@codecov-io
Copy link

codecov-io commented Mar 21, 2018

Codecov Report

Merging #456 into master will decrease coverage by 0.68%.
The diff coverage is 82.28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #456      +/-   ##
==========================================
- Coverage   89.26%   88.58%   -0.69%     
==========================================
  Files          27       27              
  Lines        4238     4414     +176     
  Branches       39       39              
==========================================
+ Hits         3783     3910     +127     
- Misses        455      504      +49
Impacted Files Coverage Δ
sourmash/__main__.py 96% <ø> (ø) ⬆️
sourmash/commands.py 85.66% <10.25%> (-3.43%) ⬇️
sourmash/sbt_storage.py 87.87% <90.9%> (+2.48%) ⬆️
sourmash/sbt.py 86.22% <94.57%> (+0.51%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d69b79...863c255. Read the comment docs.

luizirber and others added 7 commits July 7, 2019 20:22
refactor SBT to keep internal nodes and leaves in separate dicts
make nodes and missing nodes 'private'
fill_up implemented and working
Disable updates during indexing
small cleanups
* start support for loading IPFS data without a local node

* start prepare command

* implement prepare cmd

* add sbt prepare test data

* more tests, fix migrate

* working on preloading

* add stub unload method

* fix test
@luizirber luizirber mentioned this pull request Jul 8, 2019
5 tasks
@luizirber luizirber closed this Mar 27, 2022
@luizirber luizirber deleted the refactor/faster_indexing branch March 27, 2022 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants