Make parallelization customizable (for wasm)

[fursund]

Hi!

Trying to load an exr in WASM. It could look like it fails due to threadpool being used in the exr lib?

Perhaps I am just missing some setting in the toml?

Thanks!

[johannesvollmer]

The library is not yet tested on wasm, but it is a goal. Apart from switching parallelism off, what else is required for wasm? Maybe switching off functions with file paths?

Using concurrency is already optional in the library, we would only have to disable the rayon dependency and some type definitions, in order to use single-threaded exr on wasm

[fursund]

File IO I've already worked around with the buffered loading, which I think is fine for now. How would I switch of concurrency?

[fursund]

Perhaps using this could work: https://github.com/GoogleChromeLabs/wasm-bindgen-rayon

[fursund]

I see I could get it to work using "non_parallel()".... but it takes forever to load the EXR compared to the same code running natively.

[johannesvollmer]

File IO I've already worked around with the buffered loading

Excellent! So it's simply enough to not use the path-based functions? I thought it would be necessary to remove the functions when compiling for wasm. Nice!

I see I could get it to work using "non_parallel()".... but it takes forever to load the EXR compared to the same code running natively.

The library uses rayon only for testing purposes, the runtime code uses threadpool and flume.
Unless these work on wasm, performance will be the trade-off, unfortunately. Alternatively, we could try to make the whole parallelization/threadpool functionality customizable.

[johannesvollmer]

Here's an article about threading in wasm:
https://rustwasm.github.io/wasm-bindgen/examples/raytrace.html

Seems like it will not be simple. Even if the parallelization mechanics in exrs were customizable, it would be quite the hassle to make threads work at all

[fursund]

Not sure why, but it seems like my wasm was somehow broken. I managed to rebuild it again, and now it's almost as fast as the native binary. So actually not having threading at load is not a problem after all. Thanks your help!

[johannesvollmer]

Awesome!

The loading time depends on whether the file uses exr compression, and of course the image size. Uncompressed images should be unaffected by parallelism. RLE compression is fastest, PIZ is reasonably fast, and the other compression types use the zip algorithm, which takes quite some time

[johannesvollmer]

Leaving this issue open as a reminder to add parallelization customization :)