Skip to content
This repository has been archived by the owner on May 18, 2023. It is now read-only.

Describe how messages are delimited #13

Closed
nex3 opened this issue Apr 4, 2019 · 3 comments · Fixed by #16
Closed

Describe how messages are delimited #13

nex3 opened this issue Apr 4, 2019 · 3 comments · Fixed by #16
Labels
enhancement New feature or request

Comments

@nex3
Copy link
Contributor

nex3 commented Apr 4, 2019

Protocol buffers are not self-delimiting, so if we want to communicate over a stream that's not inherently packet-based (which we do, since we plan to use stdin/stdout), we'll need a scheme for indicating which bytes belong to which message. The "Techniques" page of the protobuf documentation suggests writing the message length before the message, which seems like as good a solution as any.

@nex3 nex3 added the enhancement New feature or request label Apr 4, 2019
@ghost
Copy link

ghost commented Apr 7, 2019

I don't get if it means sending two message (one "fixed length" and one the actual message) or parsing one message up to the "length" field and then parse the remaining part.
Either way, seems more complicated than a delimiter of some sort (not really part of the protocol, but could be negotiated like I was suggesting in #14

@nex3
Copy link
Contributor Author

nex3 commented Apr 7, 2019

I don't get if it means sending two message (one "fixed length" and one the actual message) or parsing one message up to the "length" field and then parse the remaining part.

I don't think the protocol buffer binary format has a way of sending fixed-length messages, so we'd probably want to do something out-of-band, like a 64-bit integer before each message.

Either way, seems more complicated than a delimiter of some sort

I'm not sure what we could use as a delimiter... as far as I know, any byte sequence could show up in a binary-encoded protocol buffer.

@ghost
Copy link

ghost commented Apr 9, 2019

Well, multipart/form-data uses the same idea and is considered solid, so I guess a sufficiently long/randomized string won't really be a problem. Besides the whole stack we're building this for stands atop of a long pile of "good enough randomness", lol.
Either way, this also isn't trivial. If you feel it's easier let's just go for the integer.

nex3 added a commit that referenced this issue Apr 11, 2019
@nex3 nex3 closed this as completed in #16 Apr 15, 2019
nex3 added a commit that referenced this issue Apr 15, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant