possible rework to kernel sync+validation

[antiochp]

Been thinking a bit about how we fast-sync -

• sync all the headers (chunks of 500 at a time)
• then big txhashset.zip download, containing -
  • output MMR (possibly pruned, effectively UTXO only)
  • rangeproof MMR (possibly pruned, effectively UTXO only)
  • kernel MMR (full kernel history, not pruned)
• then recent full blocks

I think there is an argument to be made that the kernel MMR sync is more similar to the header sync (i.e. we need all of them and we need to validate the full history) than the output+rangeproof MMR sync.

Currently we validate the kernel history by rewinding back from latest block header and validating the kernel sums at each block height.

I wonder if we could sync kernels differently -

1. sync them in a similar way to block headers (in chunks of n kernels for some reasonably large n).
2. we only need to sync the kernels themselves, se can rebuild the MMR hash file from the underlying data as we do not prune these nodes currently (this has been discussed in rough terms in some earlier PR).
3. kernel history validation may be simpler (and more efficient) if we then validate them as we receive them in increasing block height order, with no need to rewind each time

Related #1218

Possible revised sync and validation process -

• sync headers
  • validate them as we process them
• sync kernels
  • validate them as we process them (each header gives us a MMR size to validate sums with)
• sync txhashset.zip (containing only output and rangeproof MMRs)

Related -
#804
#952

[hashmap]

@antiochp is it still valid?

[antiochp]

Let's close this - we have some ideas around how to improve this, but we can open more focused issues if/when we prioritize working on this.