signet/miner: Add --prefer-gbt-cli option

[ajtowns]

Adds a --prefer-gbt-cli option. Setting this allows you to specify an alternative bitcoin-cli invocation that will (only) be used for preferentially obtaining a block template. The logic for mining a block then becomes:

1. obtain preferential block template
2. mine this template to produce a candidate block
3. submit this candidate block using the regular bitcoin-cli invocation
4. if this fails, obtain a block template using the regular bitcoin-cli invocation
5. mine and submit that block template

The idea here is that you can point the signet miner at both a core node and an inquisition node, but only use the inquisition node for generating block template (using --prefer-gbt-cli to point at the inquisition node). This means that if there is an unexpected inconsistency in the consensus logic between the two nodes that causes inquisition to generate blocks that aren't accepted by core, then submitblock will fail at step 3, and only core compatible blocks will be generated.

[ajtowns]

Open question: should anything be done to detect/recover from the --prefer-gbt-cli having more restrictive relay rules, resulting in txs building up in the main node's mempool but never getting included in the prefer-gbt-cli template? I argue not in this post, but I can think of two plausible alternatives:

• mine, say, 10% of blocks just using core (using --multiminer), so the txs make it through eventually. This just papers over bugs rather than highlighting them, which doesn't seem helpful though.
• before mining the prefered gbt, ask core for a template, and compare the coinbasevalue between the two: if core's version is higher, that probably indicates a problem, so issue a warning and mine using core. But it could just mean someone's successfully done a pinning attack against the preferred node, and that there isn't anything actually wrong, so may be too vulnerable to false positives.

[michaelfolkson]

My assumption (and this may be a shortsighted so bear with me) is that a signet miner would be expected to run the latest version of bitcoin-inquisition. Hence a signet miner running an old version of bitcoin-inquisition and as a result their blocks being rejected/not being signed by the block signers and transactions in their mempool not being relayed isn't a major concern. It is not as if we're trying to replicate the "miner can competitively mine using any Core or bitcoin-inquisition version" here.

resulting in txs building up in the main node's mempool but never getting included in the prefer-gbt-cli template

Given the prior stated assumption, when updating to the latest version of bitcoin-inquisition your mempool would flush any existing transactions that no longer meet the latest consensus rules and policy rules. That sounds like a much cleaner and less chaotic solution than somehow trying to mine x% of blocks using a different set of consensus rules, policy rules than the latest version of bitcoin-inquisition.

A Core signet node will only care if there is an effective hard fork and will ignore any soft forks on bitcoin-inquisition. If there is an effective hard fork on bitcoin-inquisition there is a risk of the Core signet node stalling and needing to run bitcoin-inquisition to get their blockchain moving again.

[ajtowns]

Hence a signet miner running an old version of bitcoin-inquisition and as a result their blocks being rejected/not being signed by the block signers

There isn't any separation between miners and block signers. If there were one, signing occurs before applying proof of work, anyway so that outcome isn't possible.

A Core signet node will only care if there is an effective hard fork and will ignore any soft forks on bitcoin-inquisition. If there is an effective hard fork on bitcoin-inquisition there is a risk of the Core signet node stalling and needing to run bitcoin-inquisition to get their blockchain moving again.

That is exactly what this patch prevents. Any "hard fork block", that is a block that the core signet node sees as invalid will be rejected by the core signet node when the block is passed into submitblock, so that block will never be seen on the network; instead a new block template will be generated from core, then signed and mined, then submitted to core and relayed across the network.

[michaelfolkson]

There isn't any separation between miners and block signers. If there were one, signing occurs before applying proof of work, anyway so that outcome isn't possible.

I thought it was submit block template to block signer, get block signature, then find proof of work? Theoretically possible to have the separation between miners and block signers even if today the 2 block signers are the only miners?

That is exactly what this patch prevents. Any "hard fork block", that is a block that the core signet node sees as invalid will be rejected by the core signet node when the block is passed into submitblock, so that block will never be seen on the network; instead a new block template will be generated from core, then signed and mined, then submitted to core and relayed across the network.

So in the case of a change to a soft fork implementation being pushed to bitcoin-inquisition and it being an effective hard fork the above would mean transactions applying the updated rules of the soft fork implementation (impacted by the hard fork) would never be mined? I don't think there's any way to square the circle supporting effective hard forks in this repo whilst not changing the Core codebase. Just start again with a (new) different opcode or different sighash flag if the implementation of say OP_CTV or SIGHASH_ANYPREVOUT needs to be changed post being activated on bitcoin-inquisition?

[ajtowns]

There isn't any separation between miners and block signers. If there were one, signing occurs before applying proof of work, anyway so that outcome isn't possible.

I thought it was submit block template to block signer, get block signature, then find proof of work? Theoretically possible to have the separation between miners and block signers even if today the 2 block signers are the only miners?

Block signers obtain a template from their node by calling getblocktemplate, it's not provided by some third party. If you want to let other people create block templates, you'll probably need some way of validating the template prior to signing it, which I don't think exists (if you send it to submitblock, you'll just get errors of the form "bad-signet-blksig" or insufficient proof of work).

So in the case of a change to a soft fork implementation being pushed to bitcoin-inquisition and it being an effective hard fork the above would mean transactions applying the updated rules of the soft fork implementation (impacted by the hard fork) would never be mined?

They won't be mined until the bug is fixed. Once that happens, if they can be included in blocks that are considered valid by core nodes, they will be.

I don't think there's any way to square the circle supporting effective hard forks in this repo whilst not changing the Core codebase.

I'm not sure what you're talking about here. No one's proposing to use this mechanism to test hard forks.

As far as updating a proposed soft fork to new rules that might be incompatible with previous versions of the same proposal, that is supported without it being a hard fork by the ability to disable enforcement of soft forks post-activation. Core doesn't enforce either set of rules in this case, so doesn't need any changes either way.

[ajtowns]

This has been working successfully for my node for a while (including while my inquisition node was crashing repeatedly), so going to merge this.