Feature - Cooperative Program Loading

[Lichtso]

Problem

Currently transaction batches race in parallel to load, verify and compile missing cache entries.
Instead they should be coordinating and split these tasks to distribute the workload across the threads and to avoid redundant work.

Summary of Changes

• Verification-less reloading had to be disabled again, it can be reintroduced later.
• Refactors LoadedPrograms::extract() to use a mutable parameters instead of returning them in order to support multiple iterations.
• Adds explicit SecondLevel structure to LoadedPrograms to have a per address cooperative loading lock.
• Adds a conditional variable waiting section to prevent a TX batch from spinning when waiting without a loading task of its own.
• Corrects the fork graph in some tests which is important now, otherwise a TX batch would not find the programs it just loaded itself.

Principle

A TX batch in the Bank starts out by putting all the programs it requests into the "missing" set and an empty TX batch local cache. It then iteratively calls LoadedPrograms::extract() which in every iteration transfers some programs from the "missing" set to the TX batch local cache, according to what it found in the global cache. Additionally, during each iteration it tries to select a still missing entry (which no other TX batch has selected yet) and load it itself. In the next iteration it publishes its loading result to the global cache and LoadedPrograms::extract() will use that to complete the TX batch local cache in turn.

[codecov]

Codecov Report

Merging #34407 (6586075) into master (268a8d8) will decrease coverage by 0.1%.
Report is 1539 commits behind head on master.
The diff coverage is 92.2%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #34407      +/-   ##
==========================================
- Coverage    81.8%    81.8%    -0.1%     
==========================================
  Files         767      819      +52     
  Lines      209271   221004   +11733     
==========================================
+ Hits       171331   180934    +9603     
- Misses      37940    40070    +2130