Offchain Phragmén BREAKING.

[kianenigma]

Fix #4410

---

Currently, Kusama is suffering from severe stalls at the end of each era. Seemingly validators keeep missing their slot, which makes whatever we do in new_era a suspect. Phragmén election is one of them. I have made some attempts (which I hope to finish soon as reusable tools) to benchmark the wasm code but hadn't got any decisive conclusion yet. Nonetheless, using https://github.com/kianenigma/offline-phragmen I could verify that the current Kusama network (~200 candidates, ~600 voters) takes around 0.5 seconds just to compute the election result in native. In wasm, and taking the overlooked overhead of state lookup, hashing and decoding, it is enough to prioritise offchain phragmen as it is obviously hindering the block's limited 2 seconds.

In the offchain model:

• A certain number of blocks before the era is expected to end, the staking module enables a flag that allows for solutions to be submitted.
• All validators who opt to run the offchain code start computing phragmen and submit the solution back to the chain.
• Any other account can also submit a solution.
• The staking module evaluates and stores only the best result.
• If for any reason, at the end of the era, no solutions have been submitted, we fallback to the current system: on-chain phragmen.

To be Done

In this PR

• A way for staking to know when era ends.
• Main skeleton of choosing between on/off chain solutions
• Signing the transaction from offchain @tomusdrw (waiting for Add documentation to SubmitSignedTransaction and actually make it work #4200)
• Solution reduction
• Solution encoding and decoding
• Solution evaluation
• Allow submission from also unsigned (for validators)
• clean TODOS
• A metric ton of tests.
• Real testing on dev chain.
• Benchmark the runtime side of things.
• Update hashes if needed https://github.com/paritytech/substrate/pull/5226/files

In follow ups:

• Staking should freeze and not allow any changes to sensitive data when election window is open. (already done)
• Clean the reduce code. Add index cache (+ other optimizations) Add benchmarks and keep track of performance. This is not critical as that code will run offchain.
• Run equalize() again.
• rewards? punishment?

Discussions

• The key that validators sign the transaction with would be the ephemeral session key AFAIK, how would they pay for it? Update: Unsigned support added

• Maybe disable equalisation and only do seq-phragmen in fallback mode. Update: already done

• im-online is assuming that each session has a fixed length by assuming slot == block. Maybe we should do the same? @andresilva

• Encoding and Reduction research https://hackmd.io/JOn9x98iS0e0DPWQ87zGWg?view

---

polkadot companion: paritytech/polkadot#940
polkadot companion: paritytech/polkadot#940

[bkchr]

Why not an unsigned transaction?

[kianenigma]

Why not an unsigned transaction?

Because we want normal accounts (not just a validator) or entities to also be able to submit a solution. I can very well see a tool that does this for example (if we allocate a small reward to a good solution)

[bkchr]

Why don't we support unsigned and signed? I still don't see how you want to have funds in the account associated to your session keys.

[kianenigma]

Why don't we support unsigned and signed? I still don't see how you want to have funds in the account associated to your session keys.

It is one of the discussion items and I am not sure either. But:

• Only unsigned is not an option. Supporting both, maybe. Validators can submit unsigned then.
• We can hack around it by tweaking the TakeFee to make some exceptions in on certain transaction, if they come from certain account (but it might not look good :D). But we allow the signed extension to actually examine the origin, here's a use case.

[tomusdrw]

Good start!

The key that validators sign the transaction with would be the ephemeral session key AFAIK, how would they pay for it?

I think we should allow both Unsigned transactions coming from validators and Signed transactions from external nodes. Possibly the external nodes are accepted only as an improvement
over what validators have proposed.

speaking of that, maybe this whole approach of X blocks before the era ends for the window of offchain is simply too complicated. We could just say: in the last session the offchain is allowed.

Why make any requirements like that? Aren't we going to accept the best result anyway? So by definition the results computed ealier would be worse or even invalid.

[kianenigma]

Good start!

The key that validators sign the transaction with would be the ephemeral session key AFAIK, how would they pay for it?

I think we should allow both Unsigned transactions coming from validators and Signed transactions from external nodes. Possibly the external nodes are accepted only as an improvement
over what validators have proposed.

speaking of that, maybe this whole approach of X blocks before the era ends for the window of offchain is simply too complicated. We could just say: in the last session the offchain is allowed.

Why make any requirements like that? Aren't we going to accept the best result anyway? So by definition the results computed ealier would be worse or even invalid.

Suggestion of accepting both signed and unsigned is indeed an option. Also recommended by @bkchr. We'll see about it.

Regarding the timespan and always allowing solutions: I initially disagree. The votes and validations might, and most likely will, change during an era. A solution submitted toward the beginning is most likely a waste to even compute and verify since the votes will change.

I am more in favour of doing something moderate in between: the X-block window that I implemented now is also a bit too strict. Making the window wider to something like the last quarter of the era is more sensible. We could do this either at granularity of blocks or sessions.

[burdges]

Would we reward validators for their solutions too?

We looked for VRF tricks to prove ownership of Phragmén solutions, but if I recall Phragmén does not use randomness in the right way for this. Assuming this, if you want solutions to be owned then you must commit to them in advance and reveal after the commit finalizes. We do not need this now but it might be something we'd want in 2021 or so.

I'm only mentioning this in case it impacts what code goes into the runtime, but presumably the runtime could always be modified to do commits.

[kianenigma]

@bkchr @thiolliere you could have a look at the phragmen/compact part already as it is isolated and can be reviewed separately.

[kianenigma]

Ok recap:

• No need for Signature, key, signing etc. OCW will submit an unsigned call without any additional information.
• This transaction will live in the pool until a node is designated to be the author. Then it will get validated. It is for the longevity of this transaction to be only until the end of the era, and
• ValidateUnsigned will validate it only if Source::Local and will not propagate it.

One thing that I don't get is: Now that we will soon have source in ValidateUnsigned, this is not relevant anymore, right?

The alternative is to have validate_unsigned actually check the solution, and then accept from external sources (though nobody would actually be publishing).

ValidateUnsigned can just validate the call being the correct one and the source being local. The rest can be done in the execution phase.

[tomusdrw]

The alternative is to have validate_unsigned actually check the solution, and then accept from external sources (though nobody would actually be publishing).

ValidateUnsigned can just validate the call being the correct one and the source being local. The rest can be done in the execution phase.

I believe so, yes. Make sure to check the source as the first thing (i.e. reject if source is External accept if it's Local or InBlock). Validating the entire solution for everyone is a potential DoS vector. You could spam unsigned transactions with seemingly valid solutions that just don't bring any improvement when included on chain, yet we have to validate and execute them to learn that.

[kianenigma]

@gavofyork this is the only remaining part that I told you about. I propose adding the panic/slash in a second phase. The code change would be simple and we merely panic here if check_and_replace_solution was anything other than Ok(()).

Given

• Validators wanting to behave correctly,
• Our offchain worker code working correctly,

submit_election_solution_unsigned will only ever be called if a solution is an improvement to what we already have on chain (because we check the score in ValidateUnsigned), hence it must never panic unless if a validator attempted to put a non-improving solution on chain.

[kianenigma]

@jacogr from the end user's perspective, maybe an optimisation idea for the UI is to show a warning in the Account Actions page of Staking whenElectionStatus::Open(_), saying that you cannot submit any transactions until the end of the era.

[kianenigma]

polkadot fix here: paritytech/polkadot#941