Document and address WebAssembly constraints

[sffc]

@hagbard wrote a doc with some constraints that WebAssembly brings that make it not a viable option for ICU4X porting to the Web Platform. We should:

1. Add this document to GitHub
2. Raise these concerns to the WebAssembly standards group to make progress on addressing those problems in future WebAssembly releases

[hsivonen]

Raise these concerns to the WebAssembly standards group to make progress on addressing those problems in future WebAssembly releases

These numbers are relevant for crossing the WebIDL boundary. In that case, it was assumed that the wasm side needs to see UTF-8 anyway.

Since ICU4X will be able to operate on UTF-16, it would be worthwhile to consider a non-WebIDL method on JS String that takes a Uint16Array view to a SharedArrayBuffer and writes the string contents there.

TC39 may want to consider:

• A method on JS String that takes an enable-shared Uint8Array and returns in the most efficient way possible (two-item array?) the number of code units read and the number of code units written with the semantics copied from TextEncoder.encodeInto.
• A method on JS String that takes an enable-shared Uint16Array and writes as many code units as fit as UTF-16. (The number of code units written is implicitly the minimum of the string length and the buffer length.)
• A constructor that takes an enable-shared Uint16Array and constructs a string with the potentially-invalid UTF-16 contents of that buffer. (Probably preserving unpaired surrogates or having an option to replace them with the REPLACEMENT CHARACTER.)
• A constructor that takes an enable-shared Uint8Array and constructs a string with the potentially-invalid UTF-8 contents of the buffer replacing ill-formed sequences with the REPLACEMENT CHARACTER according to the Encoding Standard.

CC @annevk

[hsivonen]

writes as many code units as fit as UTF-16

Correction: Writes the code units as 16-bit units without ensuring UTF-16 validity.

[annevk]

As part of this it would be ideal if the feature request in whatwg/encoding#174 could also be considered. The most natural place for that seems to be some extension to JavaScript strings rather than a new feature on the Encoding API objects.

cc @alexcrichton @Pauan

[Pauan]

@hsivonen That just seems like it's reinventing TextEncoder and TextDecoder.

Since TextEncoder already accepts an encoding argument, why can't we just add new TextEncoder("utf16")? The API doesn't need to change (Uint8Array is fine), it just needs a new encoding added.

[hsivonen]

That just seems like it's reinventing TextEncoder and TextDecoder.

For UTF-8, yes, except without WebIDL. How big of a performance win not having WebIDL in there would be should be benchmarked before committing.

For UTF-16, no. TextEncoder writes to Uint8Array. This is about writing to a Uint16Array, which is guaranteed to be aligned right. I.e. this is about Unicode Encoding Forms, not about Unicode Encoding Schemes.

[Pauan]

This is about writing to a Uint16Array, which is guaranteed to be aligned right. I.e. this is about Unicode Encoding Forms, not about Unicode Encoding Schemes.

WebAssembly guarantees little endian (since no mainstream CPUs use big endian), so there's no problem with TextEncoder just writing into the Uint8Array with little endian order.

Or are you concerned about something else?

[hsivonen]

Or are you concerned about something else?

Yes: Alignment. A UTF-16LE encoder writing by byte to a buffer that is actually aligned to even addresses is either more complex (if it checks for alignment and has multiple paths) or less efficient (if it writes by byte) than one that gets to assume it can do aligned 16-bit writes.

[Pauan]

@hsivonen Okay, but I don't see why the encoder can't internally use Uint16Array (even if it returns Uint8Array). Since converting from Uint16Array to Uint8Array doesn't copy (since they can share the same underlying ArrayBuffer).

But even if there was some reason that wouldn't work, TextEncoder could just return Uint16Array if the encoding is "utf16". That sort of polymorphism is very common (even in web APIs).

So why do we need a new API that just does the same thing as TextEncoder/TextDecoder?

[sffc]

CC @littledan

[sffc]

See #207 for additional discussion on this subject.