network optimization

[pshaughn]

Various observations, to begin with:

• for small messages, the number of separate messages sent is more important than the message size, since each message has network overhead to encapsulate it and causes events to fire on the server and client.
• infrequent messages are less important to optimize than frequent ones; in particular, messages that only happen at the start and end of a controller's lifecycle are less important than ones that repeat.
• asymptotically, as the number of clients goes up, the broadcasting of messages all-to-all across clients dominates network usage.
• not every f event actually needs to be seen by other clients: if it doesn't actually change the input state, then client predictions will be unchanged whether or not it's received.
• in theory the client doesn't necessarily need to send an f event to the server, if that event isn't changing the input state; however, if a NAKed event is a no-op for the server, then the client has no way of knowing instantaneously which f events it needs to send. if frame x is going to be NAKed, then frame x+1 needs to be sent even if it's the same input as frame x, even though the client hasn't seen the NAK yet to know this. Letting the server track NAKed events and delay them into the stream of input events seems possible, but hard to express cleanly, since the server might delay the event into a frame where the client is actually sending a different input.
• every f event that does change the input state, and every o event, does need to be seen by other clients, promptly. deferring them to send in batches would cause visible lag effects.
• an f event doesn't have to be echoed back to the client promptly if it isn't including a timing pong, and timing pongs don't need to be 30fps
• F events from the server do not need to be seen promptly; a client that isn't lagging doesn't do anything visible in response to one.

first optimization steps:

• for each controller, track the most recent input string that was broadcast for it. when broadcasting an f event, if it matches that, don't actually broadcast it, but do echo it back to its originator, and do add to the instance events so it'll be sent with an S event to catch up a newly joining client. This doesn't need a client-side change, (this is still sending 30fps timing pongs)
• change the F frame api so an F frame comes with a frame number and means "the past horizon has advanced to this frame number" rather than "the past horizon has advanced one frame". only send an F frame for frame numbers that are a multiple of a server-side constant. still actually advance the past horizon at 30fps, to distribute cpu workload smoothly over time.

[pshaughn]

first steps are done. next step would be to let the client not send an unchanged "f" event; for that to work, the client needs to be able to trust that any previous "f" event it sent will eventually be applied even if it was discarded as late for its specific frame number. not sure how to do this cleanly yet.