lnd: first class pruned node support

[Roasbeef]

Much of the work described in this issue will be done in btcwallet, but as it will be a major feature of lnd I chose to track it here instead.

Today, it isn't safe to run lnd with a pruned node, as it's possible that while lnd is down, bitcoind prunes blocks that we still need in order to complete a rescan which allows us to detect when we've received funds on chain, or if a channel has been spent. As a result, we need to be able control when blocks are pruned in order to ensure that they're only pruned once we no longer need them. With this feature, then we can safely support pruned nodes.

The only other case to be concerned about is being able to validate very old channels. However there isn't much risk in this as a node should quickly hear about new channels, and in the future channel announcements will also have SPV proofs attached for them. The final case to consider is performing a full historical rescan (via dropwtxmgr). For this case to work properly, we'll require a version of bitcoind that maintains the BIP 157 filters so we can scan them locally, and then fetch blocks from the p2p network.

Steps To Completion

• Extend the main configuration of btcwallet to be able to define a manual prune mode. This mode should only work with the bitcoind chain backend (for now). We'll likely want to introduce a new interace (in addition to the existing primary interface) that only the bitcoind backend should implement. The wallet can then use a type assertion to detect if the backend supports manual pruning or not:

type ChainPruner interface {
    PruneBlock(chainhash.Hash) error
}

In order to start bitcoind in this mode, -prune should be set to 1. The RPC call will fail if the node isn't started in manual pruned mode (or a pruned mode at all).

• In the main chain notifications loop, once a block is connected, we should prune the set of blocks a distance of N blocks away from this newly connected block. Pruning should use the above interface, and will likely require creating a raw JSON request as btcd doesn't have this new call.

• The pruned size can either take a number of blocks, or a size target. Implementation wise, from our PoV, the number of blocks is easier, and the size based pruning will require us to look on disk to tell how much space is occupied. Alternatively, we can just extrapolate the size target based on the max possible block size.

• The amount of blocks to prune will depend on how many blocks the user wants to maintain. This should be configurable via a new command line parameter within lnd, possibly something along the lines of bitcoind.pruned, bitcoind.prunesize. The format activates pruned move, and the latter sets the size target in MB.

[lukechilds]

@Roasbeef as a temporary workaround for people who would like to use lnd with the self validating guarantees of their own pruned bitcoind, would it be sufficient to run lnd backed by Neutrino and just poll the lnd tip blockhash once a minute and check that against pruned bitcoind?

If the blockhash differs then the script should notify me that there may be some sort of attack going on.

[mandelmonkey]

@lukechilds hey, this is exactly what we are doing with Nayuta Core, its bitcoind running in pruned node on an android phone with LND, at first we connected LND to the backend but had issues with channels due to the pruned data
so now we use neutrino and compare hashes, if there is a difference we alert the user and let them specify their own neutrino server to connect to if they wish

[lukechilds]

@mandelmonkey thanks for getting back to me, great to hear other people are using this technique with success.

Great work on Nayuta!