Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C++] Add/document configuration to build with WebAssembly #23221

Closed
asfimport opened this issue Oct 15, 2019 · 2 comments · Fixed by #37821
Closed

[C++] Add/document configuration to build with WebAssembly #23221

asfimport opened this issue Oct 15, 2019 · 2 comments · Fixed by #37821

Comments

@asfimport
Copy link
Collaborator

Not something I plan to work on, but given that https://github.com/finos/perspective created a custom wasm build of Arrow, it would seem a good idea to have a standard way for wasm projects to use the Arrow C++ library

Reporter: Wes McKinney / @wesm

Note: This issue was originally created as ARROW-6894. Please see the migration documentation for further details.

@joemarshall
Copy link
Contributor

It would definitely seem worth building for webassembly nowadays, or at least building pyarrow and the required dependencies given (a) webassembly is an officially supported platform in python 3.11 and (b) pandas supports arrow as a backend officially, which won't work on webassembly until there is a reliable arrow build.

The core of arrow builds fine in wasm, but several things make it a pain to build a working pyarrow. These are:

  1. Threads don't work in many webassembly environments (including pyodide), and there's no simple way of just removing threading in arrow build right now (I think?)
  2. Dependencies have to be built in arrow - this is okay for the set needed for pyarrow, but I never did manage to get them automatically downloading and building using the configuration in emscripten. I think some work is required on the download and build cmake calls to make sure they inherit all the emscripten flags etc. correctly.
  3. I couldn't make the setup.py work correctly for pyarrow, which is probably a similar problem to how it interacts with the pyodide package build process.

@joemarshall
Copy link
Contributor

This is pretty much the same as #34996 - in that it requires #35471 to work for it to be usable in most webassembly environments (i.e. browsers).

@kou kou closed this as completed in #37821 Apr 5, 2024
kou added a commit that referenced this issue Apr 5, 2024
Split from #37696 

This is just the cmake changes to enable building on emscripten.

Changes are:
1) Support for target system "emscripten"
2) Cmake preset for building libarrow ` ninja-release-python-emscripten` (same as `ninja-release-python`, but with emscripten support)
3) Override file for cmake on emscripten, to set various build parameters that need setting to make it build there.
4) Changes in pyarrow cmake so it works if you are building libarrow as shared library, and also an option to enable the cmake file there to just dump the current arrow configuration, which is useful for cross-compile builds.

* Closes: #23221

Lead-authored-by: Joe Marshall <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Co-authored-by: Joris Van den Bossche <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
@kou kou added this to the 16.0.0 milestone Apr 5, 2024
tolleybot pushed a commit to tmct/arrow that referenced this issue May 2, 2024
…e#37821)

Split from apache#37696 

This is just the cmake changes to enable building on emscripten.

Changes are:
1) Support for target system "emscripten"
2) Cmake preset for building libarrow ` ninja-release-python-emscripten` (same as `ninja-release-python`, but with emscripten support)
3) Override file for cmake on emscripten, to set various build parameters that need setting to make it build there.
4) Changes in pyarrow cmake so it works if you are building libarrow as shared library, and also an option to enable the cmake file there to just dump the current arrow configuration, which is useful for cross-compile builds.

* Closes: apache#23221

Lead-authored-by: Joe Marshall <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Co-authored-by: Joris Van den Bossche <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
tolleybot pushed a commit to tmct/arrow that referenced this issue May 4, 2024
…e#37821)

Split from apache#37696 

This is just the cmake changes to enable building on emscripten.

Changes are:
1) Support for target system "emscripten"
2) Cmake preset for building libarrow ` ninja-release-python-emscripten` (same as `ninja-release-python`, but with emscripten support)
3) Override file for cmake on emscripten, to set various build parameters that need setting to make it build there.
4) Changes in pyarrow cmake so it works if you are building libarrow as shared library, and also an option to enable the cmake file there to just dump the current arrow configuration, which is useful for cross-compile builds.

* Closes: apache#23221

Lead-authored-by: Joe Marshall <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Co-authored-by: Joris Van den Bossche <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
rok pushed a commit to tmct/arrow that referenced this issue May 8, 2024
…e#37821)

Split from apache#37696 

This is just the cmake changes to enable building on emscripten.

Changes are:
1) Support for target system "emscripten"
2) Cmake preset for building libarrow ` ninja-release-python-emscripten` (same as `ninja-release-python`, but with emscripten support)
3) Override file for cmake on emscripten, to set various build parameters that need setting to make it build there.
4) Changes in pyarrow cmake so it works if you are building libarrow as shared library, and also an option to enable the cmake file there to just dump the current arrow configuration, which is useful for cross-compile builds.

* Closes: apache#23221

Lead-authored-by: Joe Marshall <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Co-authored-by: Joris Van den Bossche <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
rok pushed a commit to tmct/arrow that referenced this issue May 8, 2024
…e#37821)

Split from apache#37696 

This is just the cmake changes to enable building on emscripten.

Changes are:
1) Support for target system "emscripten"
2) Cmake preset for building libarrow ` ninja-release-python-emscripten` (same as `ninja-release-python`, but with emscripten support)
3) Override file for cmake on emscripten, to set various build parameters that need setting to make it build there.
4) Changes in pyarrow cmake so it works if you are building libarrow as shared library, and also an option to enable the cmake file there to just dump the current arrow configuration, which is useful for cross-compile builds.

* Closes: apache#23221

Lead-authored-by: Joe Marshall <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Co-authored-by: Joris Van den Bossche <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
vibhatha pushed a commit to vibhatha/arrow that referenced this issue May 25, 2024
…e#37821)

Split from apache#37696 

This is just the cmake changes to enable building on emscripten.

Changes are:
1) Support for target system "emscripten"
2) Cmake preset for building libarrow ` ninja-release-python-emscripten` (same as `ninja-release-python`, but with emscripten support)
3) Override file for cmake on emscripten, to set various build parameters that need setting to make it build there.
4) Changes in pyarrow cmake so it works if you are building libarrow as shared library, and also an option to enable the cmake file there to just dump the current arrow configuration, which is useful for cross-compile builds.

* Closes: apache#23221

Lead-authored-by: Joe Marshall <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Co-authored-by: Joris Van den Bossche <[email protected]>
Co-authored-by: Sutou Kouhei <[email protected]>
Signed-off-by: Sutou Kouhei <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment