Dynamic block pruning depth

[nazar-pc]

In Subspace we have a need to make pruning depth dynamic (in terms of blocks). This is because initial sync will happen from our distributed storage network (DSN) and doesn't require archival nodes, but we need to be able to run keep-up sync for blocks that are not in DSN yet, number of which changes depending on block contents (and thus size).

The workaround I'll be using for now is to not finalize blocks beyond some depth, but maybe the pruning itself can be made dynamic? Looks like replacing BlocksPruning::Some(u32) with BlocksPruning::Some(Arc<AtomicU32>) might already be sufficient.

[bkchr]

The workaround I'll be using for now is to not finalize blocks beyond some depth,

THis doesn't sound like a workaround, but more like the solution? Your consensus should ensure that it can only finalize things when they are available. This is basically also what is done by Polkadot and its Parachains.

[nazar-pc]

It is available either way, the challenge here is that blocks at certain depth are archived and end up in DSN, but if node switches to keep-up sync, it may take some time for it to actually catch up to the tip and there is a chance it will be unable to pull some of the blocks from nodes using Substrate sync mechanism. I wanted to ensure we have more slack in terms of blocks that are available on non-archival nodes.

More generally finality seems to be used for multiple different purposes in Substrate, not all of which have the same nature. The primary reason we finalize blocks in our chain at all is such that block pruning works, on chains without finality block pruning is non-functional, which ideally wouldn't be the case.