blockchain: Use skip list for ancestor traversal.

[davecgh]

There are several places when processing blocks and headers that require the ability to traverse the chain to ancestors of a given block such as tallying votes and versions to determine threshold states, finding forks, and calculating PoW difficulty, ticket price, and sequence locks.

Currently most traversal of this type is linear and therefore does not scale well as the chain height grows. In addition, the ability to quickly locate ancestors, potentially deep in history, will be needed when decoupling the connection code from the download logic as well as several planned future optimizations.

Consequently, this significantly optimizes ancestor traversal by introducing a deterministic skip list and adds tests to ensure proper functionality.

The following is a before and after comparison of ancestor traversal for a large number of nodes:

benchmark           old ns/op     new ns/op   delta
------------------------------------------------------
BenchmarkAncestor   67901581      158         -100.00%

benchmark           old allocs   new allocs   delta
------------------------------------------------------
BenchmarkAncestor   0            0            +0.00%

benchmark           old bytes    new bytes    delta
------------------------------------------------------
BenchmarkAncestor   0            0            +0.00%

The following is a scatter plot of the number of steps to traverse back to a height of 1 for all heights up to 2^14 along with a logarithmic regression to help visualize the behavior.

[matheusd]

Very nice :)

I wanted to get some intuition from how the skip list was actually behaving, so I came up with this visualization:

clob = lambda x: x & (x-1)                # Clear Lowest One Bit
slh = lambda x: clob(clob(x))             # Skip List Height
fmt = lambda x: print(format(x, '#034b')) # pretty print in binary

def ancestor(orig, target):
  x = orig
  fmt(x)
  c = 0
  while x != target:
    if slh(x) >= target:
      x = slh(x)
    else:
      x = x -1
    c += 1
    fmt(x)
  print("Count: %d" % c)

ancestor(0xffffffff, 1)

Here's an online jupyter notebook (hopefully it doesn't go away too soon): https://notebooks.gesis.org/binder/jupyter/user/ipython-ipython-in-depth-j8yn9m22/notebooks/binder/skip%20list.ipynb

[davecgh]

I updated the PR description to include a scatter plot of the number of steps to traverse back to a height of 1 for all heights up to 2^14 along with a logarithmic regression to help visualize the behavior.