Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Splitstore Garbage Collection #6728

Closed
wants to merge 33 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
93427a4
BlockstoreMover trait
vyzo Jul 15, 2021
cedc4ee
implement BlockstoreMover in badger
vyzo Jul 11, 2021
6189eaa
use pooled slices for the copy
vyzo Jul 11, 2021
8ecd541
add MoveTo test
vyzo Jul 11, 2021
fc5a780
fix lotus-shed
vyzo Jul 11, 2021
93be5a9
fix lint
vyzo Jul 11, 2021
7d1b0be
fix copy: flush discards the transaction
vyzo Jul 11, 2021
d5b78a5
clean up empty subdirectories in splitstore path after move
vyzo Jul 11, 2021
18f9e58
follow symbolic links when deleting old dbs
vyzo Jul 12, 2021
03ecc15
add some more logging around move
vyzo Jul 12, 2021
49735c1
don't heap allocate the cond, just set L
vyzo Jul 13, 2021
e50cf69
use BlockstoreMover in splitstore gc
vyzo Jul 15, 2021
a8d2792
implemente PruneChain
vyzo Jul 14, 2021
cb807c1
sort dead objects; it's a matter of safety
vyzo Jul 14, 2021
e2f1ddd
fix semantics of retain state predicate when the value is 0
vyzo Jul 14, 2021
e26fa35
add v1 api for chain pruning
vyzo Jul 14, 2021
ea6c310
add chain prune cli
vyzo Jul 14, 2021
a6a2f97
deal with json-rpc types; where integers become floats
vyzo Jul 14, 2021
e19e1c3
show type of coldstore when trait cast fails
vyzo Jul 14, 2021
658b688
unwrap coldstore
vyzo Jul 14, 2021
9a48f35
improve error message when compacting is set
vyzo Jul 14, 2021
7615f43
optimize walkChainDeep to use a lax version of walkObjectIncomplete
vyzo Jul 14, 2021
73a28c0
make gen
vyzo Jul 15, 2021
d516461
do online gc when no gc method has been specified
vyzo Jul 15, 2021
6e0a5d7
update chain prune cli
vyzo Jul 15, 2021
77374f6
make docsgen-cli
vyzo Jul 15, 2021
c51cada
update README
vyzo Jul 14, 2021
1788558
more pertinent log
vyzo Jul 15, 2021
1dee206
parallelize walk/marking in walkChainDeep
vyzo Jul 15, 2021
a9bece1
print progress report in the deep walk
vyzo Jul 15, 2021
b142cd2
more frequent progress report
vyzo Jul 15, 2021
279e469
close the work channel to wait for workers
vyzo Jul 15, 2021
f3151aa
drop the sort, takes forever
vyzo Jul 15, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions api/api_full.go
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,10 @@ type FullNode interface {
// If oldmsgskip is set, messages from before the requested roots are also not included.
ChainExport(ctx context.Context, nroots abi.ChainEpoch, oldmsgskip bool, tsk types.TipSetKey) (<-chan []byte, error) //perm:read

// ChainPrune prunes the stored chain state and garbage collects; only supported if you are
// using the splitstore
ChainPrune(ctx context.Context, opts map[string]interface{}) error //perm:admin

// MethodGroup: Beacon
// The Beacon method group contains methods for interacting with the random beacon (DRAND)

Expand Down
14 changes: 14 additions & 0 deletions api/mocks/mock_full.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions api/proxy_gen.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading