Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libarchive and some compression libs #337

Merged
merged 4 commits into from
Dec 21, 2023
Merged

Conversation

jeroen
Copy link
Contributor

@jeroen jeroen commented Dec 18, 2023

This is primarly to support the archive package from CRAN:

configure: sh -c "./configure.orig --build=x86_64-pc-linux-gnu --host=wasm32-unknown-emscripten"

Found pkg-config cflags and libs!
PKG_CONFIG_PATH=/opt/webr/wasm/lib/pkgconfig
PKG_CFLAGS=-I/opt/webr/wasm/include
PKG_LIBS=-L/opt/webr/wasm/lib -larchive -lexpat -llzma -lzstd -llz4 -lbz2 -lz
** libs
using C++ compiler: 'emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.47 (431685f05c67f0424c11473cc16798b9587bb536)'
em++ -DNDEBUG  -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cli/include' -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.3.0/build/include -I/opt/webr/R/build/R-4.3.0/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -I/opt/webr/wasm/include -fpic  -std=gnu++11 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c archive.cpp -o archive.o
em++ -DNDEBUG  -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cli/include' -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.3.0/build/include -I/opt/webr/R/build/R-4.3.0/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -I/opt/webr/wasm/include -fpic  -std=gnu++11 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c archive_extract.cpp -o archive_extract.o
em++ -DNDEBUG  -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cli/include' -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.3.0/build/include -I/opt/webr/R/build/R-4.3.0/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -I/opt/webr/wasm/include -fpic  -std=gnu++11 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c archive_read.cpp -o archive_read.o
em++ -DNDEBUG  -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cli/include' -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.3.0/build/include -I/opt/webr/R/build/R-4.3.0/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -I/opt/webr/wasm/include -fpic  -std=gnu++11 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c archive_write.cpp -o archive_write.o
em++ -DNDEBUG  -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cli/include' -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.3.0/build/include -I/opt/webr/R/build/R-4.3.0/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -I/opt/webr/wasm/include -fpic  -std=gnu++11 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c archive_write_direct.cpp -o archive_write_direct.o
em++ -DNDEBUG  -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cli/include' -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.3.0/build/include -I/opt/webr/R/build/R-4.3.0/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -I/opt/webr/wasm/include -fpic  -std=gnu++11 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c archive_write_files.cpp -o archive_write_files.o
em++ -DNDEBUG  -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cli/include' -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.3.0/build/include -I/opt/webr/R/build/R-4.3.0/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -I/opt/webr/wasm/include -fpic  -std=gnu++11 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c cpp11.cpp -o cpp11.o
em++ -DNDEBUG  -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cli/include' -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.3.0/build/include -I/opt/webr/R/build/R-4.3.0/src/include -s USE_BZIP2=1 -s USE_ZLIB=1 -I/opt/webr/wasm/include -fpic  -std=gnu++11 -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -DRCPP_DEMANGLER_ENABLED=0 -D__STRICT_ANSI__ -c r_archive.cpp -o r_archive.o
emcc -DNDEBUG  -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cli/include' -I'/root/R/x86_64-pc-linux-gnu-library/4.3/cpp11/include' -I/opt/webr/wasm/include -I/opt/webr/R/build/R-4.3.0/build/include -I/opt/webr/R/build/R-4.3.0/src/include -s USE_BZIP2=1 -s USE_ZLIB=1  -fpic  -Oz -fPIC -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -std=gnu11 -c connection/connection.c -o connection/connection.o
em++ -s SIDE_MODULE=1 -s WASM_BIGINT -s ASSERTIONS=1 -L/opt/webr/wasm/lib -L/opt/webr/wasm/R-4.3.0/lib/R/lib -s USE_BZIP2=1 -s USE_ZLIB=1 -fwasm-exceptions -s SUPPORT_LONGJMP=wasm -Oz -o archive.so archive.o archive_extract.o archive_read.o archive_write.o archive_write_direct.o archive_write_files.o cpp11.o r_archive.o -L/opt/webr/wasm/lib -larchive -lexpat -llzma -lzstd -llz4 -lbz2 -lz
installing to /tmp/RtmpIsO7Z9/file91f9dd25837/archive/libs
** R
** inst
** preparing package for lazy loading
** help
*** installing help indices
** building package indices
* creating tarball
packaged installation of 'archive' as 'archive_1.1.7_R_x86_64-pc-linux-gnu.tar.gz'
* DONE (archive)
Packaging: archive_1.1.7.data

Some other notes:

  • The opinionated selection of enabled libarchive feature libraries is taken from the homebrew formula. Typically if features are disabled in homebrew, not many people care about them.
  • I am not sure if we manually need to disable threading in libarchive or lz4. The easuest way to test is build the CRAN archive package and try to decompress archives in various formats.
  • Some of the other recipes (gdal?) might automatically pick up on zstd or lz4.

@jeroen
Copy link
Contributor Author

jeroen commented Dec 21, 2023

Test with r-universe:

install.packages('archive', repos = 
  c('https://r-lib.r-universe.dev', 'https://cran.r-universe.dev'))
library(archive)

example(archive_read)
example(file_read)
Screenshot 2023-12-21 at 15 23 33

@georgestagg
Copy link
Member

Tested build on macOS and confirmed working with example(archive_read), example(file_read), and extracting a sample .rar file

@georgestagg georgestagg merged commit 7eba2b8 into r-wasm:main Dec 21, 2023
0 of 2 checks passed
@jeroen jeroen deleted the compression branch December 21, 2023 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants