Binary data gets garbaged

[piranna]

When trying to send an image of just some KB, the content gets converted to a sequence of escaped Unicode glyphs ("u1234") getting a transfered size sometimes more than double bigger, and worst than that, message gets truncated because of the extra size. For text files (for example HTML) there's not this problem.

[piranna]

Enabling the UTF8 flag on BinaryPack constructor this seems to be fixed.

[michelle]

I actually cannot reproduce this. Are you converting your image to a string first?

[piranna]

Yes, I'm moving JSON strings.
El 03/05/2013 06:47, "Michelle Bu" [email protected] escribió:

I actually cannot reproduce this. Are you converting your image to a
string first?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-17378465
.

[michelle]

Why not just directly send JSON objects with Blobs? Binarypack takes care
of everything and will allow you to serialize binary. I'm guessing this is
your own thing.

Michelle

On Thu, May 2, 2013 at 10:58 PM, Jesús Leganés Combarro <
[email protected]> wrote:

Yes, I'm moving JSON strings.
El 03/05/2013 06:47, "Michelle Bu" [email protected] escribió:

I actually cannot reproduce this. Are you converting your image to a
string first?

—
Reply to this email directly or view it on GitHub<
https://github.com/michellebu/reliable/issues/11#issuecomment-17378465>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-17379628
.

[michelle]

It's probably good to just build in utf8 as an option for reliable, because
runtime really takes a hit in serializing utf8.

[piranna]

Maybe it's my fault :-) I'm using JSON on my project since begin 9 months
ago, mainly because it's available everywhere natively and don't need to
add any extra library to use it, but I have pending to check other
serialization schemes that could be smaller and faster. The fact of using
BinaryPack on reliable library it's just and implementation detail for me
and should be transparent, since I'm not using it directly.
El 03/05/2013 08:00, "Michelle Bu" [email protected] escribió:

Why not just directly send JSON objects with Blobs? Binarypack takes care
of everything and will allow you to serialize binary. I'm guessing this is
your own thing.

Michelle

On Thu, May 2, 2013 at 10:58 PM, Jesús Leganés Combarro <
[email protected]> wrote:

Yes, I'm moving JSON strings.
El 03/05/2013 06:47, "Michelle Bu" [email protected] escribió:

I actually cannot reproduce this. Are you converting your image to a
string first?

—
Reply to this email directly or view it on GitHub<
https://github.com/michellebu/reliable/issues/11#issuecomment-17378465>
.

—
Reply to this email directly or view it on GitHub<
https://github.com/michellebu/reliable/issues/11#issuecomment-17379628>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-17379674
.

[piranna]

Maybe it's the reason why it's so slow for me?
El 03/05/2013 08:01, "Michelle Bu" [email protected] escribió:

It's probably good to just build in utf8 as an option for reliable, because
runtime really takes a hit in serializing utf8.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-17379703
.

[piranna]

http://kentonv.github.io/capnproto/

At last found it :-P I didn't remember the name, but I was thinking about
using Cap'n Proto as serialization protocol, or something custom derived
from it. Main advantage is that's memory mapped using offsets, so I could
transfer the data using arrayBuffer as binaryType and on receiver just use
ArrayViewBuffer to access to its content without conversion. What do you
think?
El 03/05/2013 08:11, "[email protected]" [email protected] escribió:

Maybe it's the reason why it's so slow for me?
El 03/05/2013 08:01, "Michelle Bu" [email protected] escribió:

It's probably good to just build in utf8 as an option for reliable,
because
runtime really takes a hit in serializing utf8.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-17379703
.

[michelle]

Cap'n Proto was designed with static languages in mind and IMO has little place in Javascript if we're not involving any other lower level languages. The biggest issue I see with it is that it requires a schema--for something like PeerJS it would introduce unnecessary overhead. BinaryPack is actually a modified version of MessagePack, which is pretty widely adopted as a web serialization format.

[piranna]

MessagePack and its improved cousin BinaryPack seems to be better
candidates... :-D It only keep to see if there's any implementation using
ArrayBuffers :-)

2013/5/3 Michelle Bu [email protected]

Cap'n Proto was designed with static languages in mind and IMO has little
place in Javascript if we're not involving any other lower level languages.
The biggest issue I see with it is that it requires a schema--for something
like PeerJS it would introduce unnecessary overhead. BinaryPack is actually
a modified version of MessagePack, which is pretty widely adopted as a web
serialization format.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-17416094
.

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un monton
de sitios diferentes, simplemente escribe un sistema operativo Unix."
– Linus Tordvals, creador del sistema operativo Linux

[michelle]

Yeah basically there's no need to convert your file blob into a binary
string using binary serialization formats, but it's definitely worthwhile
to build in the utf8 options, especially since using utf8 on PeerJS with
reliable on Chrome will currently not actually serialize utf8.

On Fri, May 3, 2013 at 4:02 PM, Jesús Leganés Combarro <
[email protected]> wrote:

MessagePack and its improved cousin BinaryPack seems to be better
candidates... :-D It only keep to see if there's any implementation using
ArrayBuffers :-)

2013/5/3 Michelle Bu [email protected]

Cap'n Proto was designed with static languages in mind and IMO has
little
place in Javascript if we're not involving any other lower level
languages.
The biggest issue I see with it is that it requires a schema--for
something
like PeerJS it would introduce unnecessary overhead. BinaryPack is
actually
a modified version of MessagePack, which is pretty widely adopted as a
web
serialization format.

—
Reply to this email directly or view it on GitHub<
https://github.com/michellebu/reliable/issues/11#issuecomment-17416094>
.

"Si quieres viajar alrededor del mundo y ser invitado a hablar en un
monton
de sitios diferentes, simplemente escribe un sistema operativo Unix."
– Linus Tordvals, creador del sistema operativo Linux

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/11#issuecomment-17422823
.