[Ready 1/2] Fork choice rewrite (#865)

* initial fork-choice refactor

* Add fork_choice test for "no votes"

* Initial test with voting: fix handling of unknown validators and parent blocks

* Fix tiebreak of votes

* Cleanup debugging traces

* Complexify the vote test

* fakeHash use the bigEndian repr of number + fix tiebreak for good

* Stash changes: found critical bug in nimcrypto `==` and var openarray

* Passing fork choice tests with varying votes

* Add FFG fork choice scenario + fork choice to the test suite

* Not sure why lmdb / rocksdb reappeared in rebase

* Add sanity checks to .nimble file + integrate fork choice tests to the test DB and test timing

* Cleanup debugging echos

* nimcrypto fix #864 as been merged, remove TODO comment

* Turn fork choice exception-free

* Cleanup "result" to ensure early return is properly used

* Add a comment on private/public error code vs Result

* result -> results following #866

* Address comments:
- raises: [Defect] doesn't work -> TODO
- process_attestation cannot fail
- try/except as expression pending Nim v1.2.0
- cleanup TODOs

* re-enable all sanity checks

* tag no raise for process_attestation

* use raises defect everywhere in fork choice and fix process_attestation test

AllTests-mainnet.md

@@ -46,6 +46,14 @@ OK: 2/2 Fail: 0/2 Skip: 0/2
 + Multiaddress to ENode                                                                      OK
 ```
 OK: 2/2 Fail: 0/2 Skip: 0/2
+## Fork Choice + Finality  [Preset: mainnet]
+```diff
++ fork_choice - testing finality #01                                                         OK
++ fork_choice - testing finality #02                                                         OK
++ fork_choice - testing no votes                                                             OK
++ fork_choice - testing with votes                                                           OK
+```
+OK: 4/4 Fail: 0/4 Skip: 0/4
 ## Honest validator
 ```diff
 + Attestation topics                                                                         OK
@@ -234,4 +242,4 @@ OK: 4/4 Fail: 0/4 Skip: 0/4
 OK: 8/8 Fail: 0/8 Skip: 0/8
 
 ---TOTAL---
-OK: 145/148 Fail: 3/148 Skip: 0/148
+OK: 149/152 Fail: 3/152 Skip: 0/152

AllTests-minimal.md

@@ -73,6 +73,14 @@ OK: 2/2 Fail: 0/2 Skip: 0/2
 + Multiaddress to ENode                                                                      OK
 ```
 OK: 2/2 Fail: 0/2 Skip: 0/2
+## Fork Choice + Finality  [Preset: minimal]
+```diff
++ fork_choice - testing finality #01                                                         OK
++ fork_choice - testing finality #02                                                         OK
++ fork_choice - testing no votes                                                             OK
++ fork_choice - testing with votes                                                           OK
+```
+OK: 4/4 Fail: 0/4 Skip: 0/4
 ## Honest validator
 ```diff
 + Attestation topics                                                                         OK
@@ -261,4 +269,4 @@ OK: 4/4 Fail: 0/4 Skip: 0/4
 OK: 8/8 Fail: 0/8 Skip: 0/8
 
 ---TOTAL---
-OK: 160/163 Fail: 3/163 Skip: 0/163
+OK: 164/167 Fail: 3/167 Skip: 0/167

beacon_chain.nimble

@@ -52,8 +52,12 @@ task test, "Run all tests":
   # price we pay for that.
 
   # Minimal config
+  buildBinary "proto_array", "beacon_chain/fork_choice/", "-d:const_preset=minimal"
+  buildBinary "fork_choice", "beacon_chain/fork_choice/", "-d:const_preset=minimal"
   buildBinary "all_tests", "tests/", "-d:chronicles_log_level=TRACE -d:const_preset=minimal"
   # Mainnet config
+  buildBinary "proto_array", "beacon_chain/fork_choice/", "-d:const_preset=mainnet"
+  buildBinary "fork_choice", "beacon_chain/fork_choice/", "-d:const_preset=mainnet"
   buildBinary "all_tests", "tests/", "-d:const_preset=mainnet"
 
   # Generic SSZ test, doesn't use consensus objects minimal/mainnet presets
@@ -69,4 +73,3 @@ task test, "Run all tests":
   # State sim; getting into 4th epoch useful to trigger consensus checks
   buildBinary "state_sim", "research/", "", "--validators=1024 --slots=32"
   buildBinary "state_sim", "research/", "-d:const_preset=mainnet", "--validators=1024 --slots=128"
-

beacon_chain/fork_choice/README.md

@@ -0,0 +1,5 @@
+# Fork choice implementations
+
+References:
+- https://github.com/ethereum/eth2.0-specs/blob/v0.10.1/specs/phase0/fork-choice.md
+- https://github.com/protolambda/lmd-ghost