Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
eventsource is required for the validator api. this implements the eventsource sink/server handler
the implementation is based off of this document: https://html.spec.whatwg.org/multipage/server-sent-events.html
note that this is a building block for the full eventsource server. there still needs to be work done
prysm has their own custom solution based off of protobuf/grpc: https://hackmd.io/@prysmaticlabs/eventstream-api using that would be not good
existing eventsource implementations for golang are not good for our situation. options are:
additionally, implemetations other than r3labs/sse are very incorrect because they do not split up the data field correctly when newlines are sent. (parsers by specification will fail to encode messages sent by most of these implementations that have newlines, as i understand it). the implementation by r3labs/sse is also incorrect because it does not respect \r
finally, all these implementations have very heavy implementation of the server, which we do not need since we will use fixed sequence ids. r3labs/sse for instance hijacks the entire handler and ties that to the server, losing a lot of flexiblity in how we implement our server
for the beacon api, we need to stream:
head, block, attestation, voluntary_exit, bls_to_execution_change, finalized_checkpoint, chain_reorg, contribution_and_proof, light_client_finality_update, light_client_optimistic_update, payload_attributes
some of these are rather big json payloads, and the ability to simultaneously stream them from io.Readers instead of making a full copy of the payload every time we wish to rebroadcast it will save a lot of heap size for both resource constrained environments and serving at scale.
the protocol itself is relatively simple, there are just a few gotchas