Make threshold encryption optional.

[afck]

Threshold encryption of batches adds a round of messages (although this could be partly ameliorated with #170) and is very CPU intensive (see amiller/HoneyBadgerBFT#9).

We could add a parameter to enable encryption only in a certain percentage of rounds, or "every n-th" round. In DHB we could even make that dynamic (#103), so that the network can adapt if censorship is suspected.

Implementation

As a first step, a new encryption_schedule configuration option should be added to [Dynamic]HoneyBadger[Builder]. I'd suggest an enum with variants Always, Never and EveryNthEpoch(n); or should it be a tuple instead that specifies how many encrypted epochs, followed by how many unencrypted ones?

HoneyBadger then needs to decide, based on the epoch number, whether to encrypt or not, and initialize EpochState accordingly. EpochState needs to be updated to be able to skip the encryption and decryption steps.

Let's also extend the simulation example to make the new parameter configurable there, and test whether it actually improves performance.

The second step would be to make it dynamic: We'd probably want a new Change variant EncryptionSchedule. It would allow validators to vote for encryption schedule changes in the same way they vote for adding or removing validators.

Once such a change has a majority, no key generation needs to be initialized. We'd just restart HoneyBadger with the new parameter.

[logannc]

I started looking at this today. So far I've mostly gotten the configuration sorted out, including dynamically changing the encryption schedule (I probably need to commit some analog to KeyGenMessage when we update, but I'm deferring that for the moment).

What I'm having a little trouble identifying is how to handle EpochState. I'm a little fuzzy on how everything ties together. Part of this is that I haven't yet read the paper so I'm a little weak on the primitives involved. I'll read up on that, but I was hoping I could get a bit of direction until then.

My current plan - which is not necessarily the best idea, it just might touch the least amount of stuff - is to just directly self.decryption.insert(our_id, DecryptionState::Complete(proposal)); in a method on EpochState where proposal is ser_prop from HoneyBadger::propose(...). My problem implementing this is that it seems like everything assumes I only have a ciphertext, so I don't see anything quickly adaptable to process an unencrypted proposal. That is, I gather I need to still do something with the proposal in the subset algorithm and, of course, send it to my peers but I don't see where/how to send an unencrypted message to my peers.

[afck]

Thank you for working on this issue!
I'm not sure whether @pacoferre, who is working on #103, has already touched on this one, too? Please make sure your changes will be compatible.

I probably need to commit some analog to KeyGenMessage when we update

I don't think so: Changing the encryption schedule doesn't need a new set of keys or any other renegotiation apart from the vote itself, so that part can be skipped.

just directly self.decryption.insert(our_id, DecryptionState::Complete(proposal));

That would only insert our proposal locally; we also need to send it to the other nodes. That happens automatically inside of Subset (which won't need to be changed).
So instead of inputting a (serialized) ciphertext into Subset, we'll need to input the serialized plaintext in the non-encryption rounds. (Or always input a serialized enum that has a ciphertext and a plaintext variant?)

I'll read up on that

The main part that's relevant to this issue is the "Algorithm HoneyBadgerBFT" box on page 6. Note that in the code, we call ACS Subset, and the part of the algorithm that deals with the queue (selecting B / N transactions etc.) is in QueueingHoneyBadger (not relevant for this issue either).
The code in the honey_badger module really just deals with counting epochs, and in each epoch:

• encrypting the proposals,
• passing them to ACS/Subset,
• decrypting the results.

There are some subtle differences between the code and the paper, so in the end it's most important to understand the honey_badger and dynamic_honey_badger modules themselves, which should be more or less self-contained and hopefully decently documented.
Please let us know where documentation and comments are lacking, and if you have any questions. It's definitely our goal to make the code as readable as possible.

[logannc]

I'll have to coordinate with him, particularly around Change.

Sorry, I wasn't clear: I know inserting into self.decryption won't send to others, but what I meant was that all of the machinery around Subset seems to be using the ciphertext. So wouldn't Subset need to be changed to have knowledge that it was plaintext? Or does Subset not actually operate on the ciphertexts, instead just forming a consensus around some subset of arbitrary values which it knows nothing about and we interpret the output of a round from Subset as either ciphertext or plaintext depending on the context?

Do values in a Subset not persist to the next round if they are not part of the consensus subset? (mostly rhetorical/a note for myself, if they don't I imagine it's because we reset the Subset object at the end of the epoch) If they persist across rounds, it could get tricky because we can only accept plaintext during non-encryption rounds (and probably shouldn't accept ciphertexts during that round), so I'll need to check if we can 'delay' including a value in a consensus subset... Hopefully thats not the case!

I'll try to take some notes on what I had trouble understanding and follow up later once I actually do understand it. :)

[afck]

Or does Subset not actually operate on the ciphertexts, instead just forming a consensus around some subset of arbitrary values

Exactly: Subset only reaches consensus on a subset of (more than ⅔ of) the nodes' proposals. It doesn't care about their content.

Unfortunately currently HoneyBadger itself does the encryption, EpochState inputs the ciphertext into Subset and later decrypts what Subset outputs. We should probably input the Contribution itself into EpochState instead, and encrypt it there, to have both in the same submodule.

Then, in non-encryption rounds, we should skip encryption and decryption and just serialize the Contribution itself.

Do values in a Subset not persist to the next round

No, they don't. They are just dropped if they're not included in the set.
(The transactions will still be in the nodes' queues, so they will eventually be part of another proposal.)

[pacoferre]

I'm sorry about lack of time :(

I made this commit on my fork pacoferre@cce8a35 and then an embarrasing fix :) pacoferre@26a0197

@logannc, don't know if you could use, perhaps you have done almost the same.

Few days ago I spent a little time re-reading documentation, but no more advance :(

[afck]

This has been implemented in #288.