Splitstore Garbage Collection

[vyzo]

Implements online coldstore chain pruning and garbage collection with the lotus chain prune command.

See #6577 for discussion.

From the README:
The coldstore can be pruned and garbage collection using the lotus chain prune command.
Note that the command initiates pruning, but runs asynchronously as it can take a long time to
complete.

By default, pruning keeps all chain reachable object; the user however has the option to specify
a retention policy for old state roots and message receipts with the --retention option.
The value is an integer with the following semantics:

• If it is -1 then all state objects reachable from the chain will be retained in the coldstore.
  This is the (safe) default.
• If it is 0 then no state objects that are unreachable within the compaction boundary will
  be retained in the coldstore.
  This effectively throws away all old state roots and it is maximally effective at reclaiming space.
• If it is a positive integer, then it's the number of finalities past the compaction boundary
  for which chain-reachable state objects are retained.
  This allows you to keep some older state roots in case you need to reset your head outside
  the compaction boundary or perform historical queries.

During pruning, unreachable objects are deleted from the coldstore. In order to reclaim space,
you also need to specify a gargage collection policy, with two possible options:

• The --online-gc flag performs online garbage collection; this is fast but does not reclaim all
  space possible.
• The --moving-gc flag performs moving garbage collection, where the coldstore is moved,
  copying only live objects. If your coldstore lives outside the .lotus directory, e.g. with
  a symlink to a different file system comprising of cheaper disks, then you can specify the
  directory to move to with the --move-to option.
  This reclaims all possible space, but it is slow and also requires disk space to house the new
  coldstore together with the old coldstore during the move.

[vyzo]

rebased for changes in the base branch.

[vyzo]

rebased on master.

[vyzo]

I'll rebase and resolve the gen conflicts when we are ready to merge.

[Stebalien]

This needs integration tests. I.e., tests where we trigger GC while computing a tipset or at least something that approximates computing a tipset (i.e. a bunch of reads, then vm.Copy). Ideally:

1. Start GC before computing, and during.
2. End GC during and after.

VM execution doesn't just write blocks, it makes a bunch of assumptions about what we have and what we don't have.

[vyzo]

Agreed; for now I am testing in a live node, but we definitely want a proper integration test.
I propose we add it to the Harness test effort outlined in #6725

How does it sounds to you?
It something we can reasonably tackle without time pressure after the v1.11.1 release code freeze.

[Stebalien]

It's really a waste of time to review until there are tests. Testing on a node isn't likely to trigger the conditions we care about.

[Stebalien]

(also, let's please break this into two PRs: one that refactors (splits files), then one that makes the actual change)

[vyzo]

Agreed with the sentiment, following our sync discussion.
I'll cherry-pick the refactor out for easy review, and then the plan is to write a good enough integration test.
If we can get it done before the deadline, great.
If not, we don't have to ship it now, and people who are willing to take the risk can always use the pr directly.

[BlocksOnAChain]

@vyzo should we close this one since it's related to 1.11.1 which is out?
FYI: @jennijuju

[vyzo]

no, its in draft for a reason.

…

On Fri, Sep 10, 2021, 21:08 Dragan Z ***@***.***> wrote: @vyzo <https://github.com/vyzo> should we close this one since it's related to 1.11.1 which is out? FYI: @jennijuju <https://github.com/jennijuju> — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#6728 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAI4SWD2CEXRMYXTCKXWG3UBJCS7ANCNFSM5AFLT4NQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[magik6k]

Done in #9056