Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libR-sys bindings for Apple M1 #35

Closed
clauswilke opened this issue Dec 21, 2020 · 33 comments
Closed

libR-sys bindings for Apple M1 #35

clauswilke opened this issue Dec 21, 2020 · 33 comments

Comments

@clauswilke
Copy link
Member

Sooner rather than later, we will need bindings for Apple M1 chips. One strategy we could pursue is to cross-compile, as we're already doing for our Windows bindings. The only downside is that we can't test whether the bindings actually work.

Here's a relevant issue about Github Action runners on M1: actions/runner-images#2187

@psobolewskiPhD
Copy link

psobolewskiPhD commented Dec 27, 2020

(please be gentle, haven't done anything like this in almost 20 years, new to GitHub)
Based on this tweet/thread from @clauswilke
https://twitter.com/ClausWilke/status/1341127920379699201?s=20

Apple Silicon MBPro (M1), running Big Sur 11.1
I installed native Rust(up) based on rust-lang/rustup#2413
sh <(curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs) --default-toolchain beta
I installed native R via homebrew:
brew install r
https://formulae.brew.sh/formula/r#default
Likewise, installed llvm via homebrew
brew install llvm
https://formulae.brew.sh/formula/llvm#default

I installed bindgen:
cargo install bindgen

I used git to get libR-sys:
git clone https://github.com/extendr/libR-sys.git

Then I ran
cargo build --features use-bindgen
I got errors regarding llvm/libclang
I solved this by:
export PATH="$PATH:/opt/homebrew/opt/llvm/bin"
export LIBCLANG_PATH=/opt/homebrew/opt/llvm/lib

Next I got error:
fatal error: 'stdarg.h' file not found

Based on:
https://users.rust-lang.org/t/fatal-error-stdarg-h-file-not-found/45866/6
and
rust-lang/rust-bindgen#1834 (comment)
I looked in build.rs and found a reference to
LIBRSYS_LIBCLANG_INCLUDE_PATH
Based on this, i tried:
export LIBRSYS_LIBCLANG_INCLUDE_PATH=/opt/homebrew/Cellar/llvm/11.0.0/lib/clang/11.0.0/include

This resulted in:
Finished dev [unoptimized + debuginfo] target(s) in 0.01s
piotrsobolewski@Piotrs-MacBook-Pro-2 libR-sys % ls
Cargo.lock LICENSE bindings src wrapper.h
Cargo.toml README.md build.rs target

target dir seems new, not sure how to test what I got...

@clauswilke
Copy link
Member Author

You can test with:

cargo test -- --test-threads=1

Also, could you do the following? Rebuild with:

cargo clean
cargo build -vv --features use-bindgen

and then post the tail end of the output you get, starting from Compiling libR-sys ...

Finally, I think you had all these setup problems because you added the path to the homebrew llvm at the end of the existing path rather than at the beginning. Also, installing bindgen is not necessary.

@psobolewskiPhD
Copy link

psobolewskiPhD commented Dec 28, 2020

Running the test command:

cargo test -- --test-threads=1

error: failed to run custom build command for `libR-sys v0.2.0 (/Users/piotrsobolewski/Dev/libR-sys)`

Caused by:
  process didn't exit successfully: `/Users/piotrsobolewski/Dev/libR-sys/target/debug/build/libR-sys-10eb235f00b22f50/build-script-build` (exit code: 101)
  --- stdout
  cargo:rustc-env=R_HOME=/opt/homebrew/Cellar/r/4.0.3_1/lib/R
  cargo:r_home=/opt/homebrew/Cellar/r/4.0.3_1/lib/R
  cargo:rustc-link-search=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/lib
  cargo:rustc-link-lib=dylib=R
  cargo:rerun-if-changed=build.rs
  cargo:rerun-if-changed=wrapper.h

  --- stderr
  thread 'main' panicked at 'Cannot find libR-sys bindings file for R 4.0.3 on macos in bindings. Consider compiling with default features enabled.', build.rs:343:13
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Here's the output for:

cargo clean
cargo build -vv --features use-bindgen

Compiling libR-sys v0.2.0 (/Users/piotrsobolewski/Dev/libR-sys)
     Running `CARGO=/Users/piotrsobolewski/.rustup/toolchains/beta-aarch64-apple-darwin/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/Users/piotrsobolewski/Dev/libR-sys CARGO_PKG_AUTHORS='andy-thomason <[email protected]>:Claus O. Wilke <[email protected]>:Ilia Kosenkov <[email protected]>' CARGO_PKG_DESCRIPTION='Low level bindings to the R programming language.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE=MIT CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=libR-sys CARGO_PKG_REPOSITORY='' CARGO_PKG_VERSION=0.2.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 DYLD_FALLBACK_LIBRARY_PATH='/Users/piotrsobolewski/Dev/libR-sys/target/debug/deps:/Users/piotrsobolewski/.rustup/toolchains/beta-aarch64-apple-darwin/lib:/Users/piotrsobolewski/.rustup/toolchains/beta-aarch64-apple-darwin/lib:/Users/piotrsobolewski/lib:/usr/local/lib:/usr/lib' rustc --crate-name build_script_build --edition=2018 build.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --cfg 'feature="bindgen"' --cfg 'feature="regex"' --cfg 'feature="use-bindgen"' -C metadata=304fb32f69ac3925 -C extra-filename=-304fb32f69ac3925 --out-dir /Users/piotrsobolewski/Dev/libR-sys/target/debug/build/libR-sys-304fb32f69ac3925 -C incremental=/Users/piotrsobolewski/Dev/libR-sys/target/debug/incremental -L dependency=/Users/piotrsobolewski/Dev/libR-sys/target/debug/deps --extern bindgen=/Users/piotrsobolewski/Dev/libR-sys/target/debug/deps/libbindgen-c82dc57646cdc1cd.rlib --extern regex=/Users/piotrsobolewski/Dev/libR-sys/target/debug/deps/libregex-e5e59f67a3a2dc38.rlib -L native=/Users/piotrsobolewski/Dev/libR-sys/target/debug/build/libloading-33e732f6f6fee3f0/out`
     Running `/Users/piotrsobolewski/Dev/libR-sys/target/debug/build/libR-sys-304fb32f69ac3925/build-script-build`
[libR-sys 0.2.0] cargo:rustc-env=R_HOME=/opt/homebrew/Cellar/r/4.0.3_1/lib/R
[libR-sys 0.2.0] cargo:r_home=/opt/homebrew/Cellar/r/4.0.3_1/lib/R
[libR-sys 0.2.0] cargo:rustc-link-search=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/lib
[libR-sys 0.2.0] cargo:rustc-link-lib=dylib=R
[libR-sys 0.2.0] cargo:rerun-if-changed=build.rs
[libR-sys 0.2.0] cargo:rerun-if-changed=wrapper.h
[libR-sys 0.2.0] Generating bindings for target: aarch64-apple-darwin, os: macos, architecture: aarch64
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/llvm/11.0.0/lib/clang/11.0.0/include/stddef.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/llvm/11.0.0/lib/clang/11.0.0/include/__stddef_max_align_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/Rinternals.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_stdio.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_symbol_aliasing.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_posix_availability.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/Availability.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/AvailabilityVersions.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/AvailabilityInternal.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/_types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/_types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_pthread/_pthread_types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/_types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_va_list.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/_types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int8_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int16_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int32_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int64_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_u_int8_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_u_int16_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_u_int32_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_u_int64_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_intptr_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_uintptr_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_size_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_null.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/stdio.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/Availability.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_ctermid.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_off_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_ssize_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/secure/_stdio.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/secure/_common.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/llvm/11.0.0/lib/clang/11.0.0/include/limits.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/limits.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/machine/limits.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/limits.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/arm/_limits.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/syslimits.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/llvm/11.0.0/lib/clang/11.0.0/include/stddef.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Arith.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/libextern.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/math.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/Availability.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Boolean.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Complex.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Error.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Memory.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/llvm/11.0.0/lib/clang/11.0.0/include/stddef.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Utils.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Boolean.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Complex.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/llvm/11.0.0/lib/clang/11.0.0/include/stddef.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Print.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/llvm/11.0.0/lib/clang/11.0.0/include/stdarg.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Rdynload.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Boolean.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/libextern.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/Rconfig.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/Rinterface.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Boolean.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/llvm/11.0.0/lib/clang/11.0.0/include/stdint.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdint.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int8_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int16_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int32_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_int64_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_uint8_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_uint16_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_uint32_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_uint64_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_intptr_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/_types/_uintptr_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_intmax_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/_types/_uintmax_t.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/Rinternals.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/RStartup.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/llvm/11.0.0/lib/clang/11.0.0/include/stddef.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Boolean.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/Rembedded.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Boolean.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/Rversion.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Parse.h
[libR-sys 0.2.0] cargo:rerun-if-changed=/opt/homebrew/Cellar/r/4.0.3_1/lib/R/include/R_ext/Error.h
[libR-sys 0.2.0] cargo:r_version=262147
     Running `CARGO=/Users/piotrsobolewski/.rustup/toolchains/beta-aarch64-apple-darwin/bin/cargo CARGO_CRATE_NAME=libR_sys CARGO_MANIFEST_DIR=/Users/piotrsobolewski/Dev/libR-sys CARGO_PKG_AUTHORS='andy-thomason <[email protected]>:Claus O. Wilke <[email protected]>:Ilia Kosenkov <[email protected]>' CARGO_PKG_DESCRIPTION='Low level bindings to the R programming language.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE=MIT CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=libR-sys CARGO_PKG_REPOSITORY='' CARGO_PKG_VERSION=0.2.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 DYLD_FALLBACK_LIBRARY_PATH='/Users/piotrsobolewski/Dev/libR-sys/target/debug/deps:/Users/piotrsobolewski/.rustup/toolchains/beta-aarch64-apple-darwin/lib:/Users/piotrsobolewski/.rustup/toolchains/beta-aarch64-apple-darwin/lib:/Users/piotrsobolewski/lib:/usr/local/lib:/usr/lib' OUT_DIR=/Users/piotrsobolewski/Dev/libR-sys/target/debug/build/libR-sys-7bdda200febae5f1/out R_HOME=/opt/homebrew/Cellar/r/4.0.3_1/lib/R rustc --crate-name libR_sys --edition=2018 src/lib.rs --error-format=json --json=diagnostic-rendered-ansi --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debuginfo=2 --cfg 'feature="bindgen"' --cfg 'feature="regex"' --cfg 'feature="use-bindgen"' -C metadata=b960fc9c72c60669 -C extra-filename=-b960fc9c72c60669 --out-dir /Users/piotrsobolewski/Dev/libR-sys/target/debug/deps -C incremental=/Users/piotrsobolewski/Dev/libR-sys/target/debug/incremental -L dependency=/Users/piotrsobolewski/Dev/libR-sys/target/debug/deps -L /opt/homebrew/Cellar/r/4.0.3_1/lib/R/lib -l dylib=R`
    Finished dev [unoptimized + debuginfo] target(s) in 11.46s

@clauswilke
Copy link
Member Author

Sorry, my mistake:

cargo test --features use-bindgen -- --test-threads=1

@clauswilke
Copy link
Member Author

@Ilia-Kosenkov Do you think you could set up cross-compilation of bindings for Apple M1? The target tripel is aarch64-apple-darwin, as you can see from the build output @psobolewskiPhD posted.

@Ilia-Kosenkov
Copy link
Member

@clauswilke,
Let me try this. I have an idea how to do this taking into account we cannot run aarch tests on x86.

@Ilia-Kosenkov
Copy link
Member

@clauswilke,
We now have correct multi-target support, controlled through a spearate (and optional) config variable targets:

We are now able to compile, consume and test bindings on Windows using a single instance (so reduced number of virtual machines are required).

https://github.com/Ilia-Kosenkov/libR-sys/blob/9e557eba2bbbf9b9e193111ff0ab5c2690f0e003/.github/workflows/test.yml#L20-L38

However, I am unable to build for aarch64 using nightly version of rust. At some point it just fails with an avalanche of errors from some internal SDK, see
https://github.com/Ilia-Kosenkov/libR-sys/runs/1615571883?check_suite_focus=true#step:9:2205

It would be nice if someone can use an x86_64 mac to build for aarch64 and then report to me what configurations should be used to do so.
Currently I only set up libclang:
https://github.com/Ilia-Kosenkov/libR-sys/blob/9e557eba2bbbf9b9e193111ff0ab5c2690f0e003/.github/workflows/test.yml#L20-L38

@clauswilke
Copy link
Member Author

@Ilia-Kosenkov The problem is that the aarch64 target only works on macOS 11.0 (Big Sur), but the current version on GH Actions is 10.15 (Catalina).

Apparently it's possible to turn on Big Sur, though: actions/runner-images#1814

@Ilia-Kosenkov
Copy link
Member

@clauswilke,
I am experiencing enormous queue times for macOS-11.0, see status at
https://github.com/Ilia-Kosenkov/libR-sys/actions/runs/448457979

As of writing this message, my previous CI run is waiting for more than 30 minutes.

@Ilia-Kosenkov
Copy link
Member

OK, there is some progress, I will hopefully finish the script later today.

@psobolewskiPhD
Copy link

psobolewskiPhD commented Dec 28, 2020

Sorry, my mistake:

cargo test --features use-bindgen -- --test-threads=1

Here's the output:

 Compiling libR-sys v0.2.0 (/Users/piotrsobolewski/Dev/libR-sys)
    Finished test [unoptimized + debuginfo] target(s) in 0.84s
     Running target/debug/deps/libR_sys-728e38bcb317616f

running 23 tests
test bindgen_test_layout_R_CMethodDef ... ok
test bindgen_test_layout_R_CallMethodDef ... ok
test bindgen_test_layout_R_inpstream_st ... ok
test bindgen_test_layout_R_outpstream_st ... ok
test bindgen_test_layout_Rcomplex ... ok
test bindgen_test_layout___darwin_pthread_handler_rec ... ok
test bindgen_test_layout___double2 ... ok
test bindgen_test_layout___float2 ... ok
test bindgen_test_layout___mbstate_t ... ok
test bindgen_test_layout___sFILE ... ok
test bindgen_test_layout___sbuf ... ok
test bindgen_test_layout__opaque_pthread_attr_t ... ok
test bindgen_test_layout__opaque_pthread_cond_t ... ok
test bindgen_test_layout__opaque_pthread_condattr_t ... ok
test bindgen_test_layout__opaque_pthread_mutex_t ... ok
test bindgen_test_layout__opaque_pthread_mutexattr_t ... ok
test bindgen_test_layout__opaque_pthread_once_t ... ok
test bindgen_test_layout__opaque_pthread_rwlock_t ... ok
test bindgen_test_layout__opaque_pthread_rwlockattr_t ... ok
test bindgen_test_layout__opaque_pthread_t ... ok
test bindgen_test_layout_exception ... ok
test bindgen_test_layout_structRstart ... ok
test tests::test_eval ... [1] 1
ok

test result: ok. 23 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

   Doc-tests libR-sys

running 1 test
test src/lib.rs - (line 18) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

@psobolewskiPhD
Copy link

If you'd like me to test something else, another version or a binary perhaps, let me know.

@clauswilke
Copy link
Member Author

@psobolewskiPhD Thanks for your efforts so far. They show that the bindings can be generated and compiled on M1.

The next step will be to pre-compile bindings and add them to the repo. We normally do this mostly automated on GitHub Actions, and @Ilia-Kosenkov is looking into this right now. If he succeeds, it would be good if you could test the precomputed bindings as well. If he doesn't, we may ask you to send us the bindings your compilation generated.

@Ilia-Kosenkov
Copy link
Member

@clauswilke, @psobolewskiPhD
I get a strange error when running on MacOS-11.0
https://github.com/Ilia-Kosenkov/libR-sys/runs/1616442474?check_suite_focus=true#step:9:1146

It says it is unable to locate R, but this error is emitted for the first target, which is default (native) architecture, x86_64.
This should not happen at all.
The error seems to come from this block

libR-sys/build.rs

Lines 368 to 374 in 07dac17

let r_paths = match r_paths {
Ok(result) => result,
Err(error) => {
println!("Problem locating local R install: {:?}", error);
exit(1);
}
};

Is it possible R is somehow incorrectly installed on Big Sur? I am unable to test it because there is a huge lineup for MacOS-11.0 machines so I cannot prototype script and have no Mac at hand.

@clauswilke
Copy link
Member Author

It may require properly set R_HOME. It's possible that the paths are not set up properly. There's also the possibility that Big Sur won't allow R to start if the code isn't properly signed.

@Ilia-Kosenkov
Copy link
Member

Welp, there is an issue in r-lib/actiond, r-lib/actions#200, they claim they fixed it but I am not sure how to implement it into our own pipeline.

@clauswilke
Copy link
Member Author

They're just creating symlinks from the installed version of R to /usr/local/bin, this should work for us the same.

@Ilia-Kosenkov
Copy link
Member

@clauswilke,
Yes, I noticed it right now, I will do that for Mac builds, temporarily, but dunno how long it will take to actually test it.
Will let you know if I get something.

@Ilia-Kosenkov
Copy link
Member

@clauswilke
Copy link
Member Author

See:
#35 (comment)

@Ilia-Kosenkov
Copy link
Member

@clauswilke,
Sorry, missed that. I am not sure though what path would it be? Is it possible to obtain it automatically like we do
$(brew --prefix llvm)/lib".

@clauswilke
Copy link
Member Author

Probably. Apologies for the terse answers, on my phone.

@Ilia-Kosenkov
Copy link
Member

@clauswilke, its OK, I just have no experience with Macs and I think that path @psobolewskiPhD used is super-specific and we should not rely on hard-coded values.
I will try different solutions, but I believe I might need a day or two to fix this because of virtual machine allocation delays.

@clauswilke
Copy link
Member Author

I would try setting LLVM_CONFIG_PATH=$(brew --prefix llvm)/bin/llvm-config. If that still isn't enough, you can set LIBRSYS_LIBCLANG_INCLUDE_PATH=$($LLVM_CONFIG_PATH --includedir).

On my machine (which doesn't run Big Sur):

> export LLVM_CONFIG_PATH=$(brew --prefix llvm)/bin/llvm-config
> echo $LLVM_CONFIG_PATH
/usr/local/opt/llvm/bin/llvm-config
> export LIBRSYS_LIBCLANG_INCLUDE_PATH=$($LLVM_CONFIG_PATH --includedir)
> echo $LIBRSYS_LIBCLANG_INCLUDE_PATH
/usr/local/Cellar/llvm/11.0.0/include

@clauswilke
Copy link
Member Author

Sorry, $LLVM_CONFIG_PATH --includedir points to the wrong location. The folder @psobolewskiPhD used can be obtained from $($LLVM_CONFIG_PATH --libdir)/clang/$($LLVM_CONFIG_PATH --version)/include:

> export LIBRSYS_LIBCLANG_INCLUDE_PATH=$($LLVM_CONFIG_PATH --libdir)/clang/$($LLVM_CONFIG_PATH --version)/include
> ls $LIBRSYS_LIBCLANG_INCLUDE_PATH/stdarg.h
/usr/local/Cellar/llvm/11.0.0/lib/clang/11.0.0/include/stdarg.h

@Ilia-Kosenkov
Copy link
Member

Good news -- it works, we have bindngs.
https://github.com/Ilia-Kosenkov/libR-sys/actions/runs/449850341
There are two things to address/discuss:

  1. I will try to use llvm_config to get correct path at runtime (currently it is partially hardcoded)
  2. What versions of R/Rust should we use to get bindings? Currently I run release/nightly.
    I am afraid it will be troublesome to have more than one MacOS-11.0 machine in the job, as I currently wait for 0:40 -- 1:30 h to get one test, and we need 3 (R oldrel/release/devel).

As soon as we decide on this, I'll make a PR with new precomputed bindings which then can be tested by M1 owners.

@Ilia-Kosenkov
Copy link
Member

@psobolewskiPhD
Copy link

Let me know when/what you'd like tested, just probably need to spell it out like to a 2-year-old...

@Ilia-Kosenkov
Copy link
Member

Ilia-Kosenkov commented Dec 29, 2020

OK, here I infer llvm paths using llvm-config
https://github.com/Ilia-Kosenkov/libR-sys/actions/runs/450652869

https://github.com/Ilia-Kosenkov/libR-sys/blob/279b55acd25ef799a215fb812347344f021c0671/.github/workflows/test.yml#L111-L127

@clauswilke, we still have to decide on what Mac configs to use.

@psobolewskiPhD, will let you know as soon as we are ready.

@clauswilke
Copy link
Member Author

R release is good. I don't have a strong opinion on Rust stable or nightly, whichever you prefer.

@Ilia-Kosenkov
Copy link
Member

Ilia-Kosenkov commented Dec 29, 2020

@psobolewskiPhD, we have merged #45, so now there are bindings for aarch64,
https://github.com/extendr/libR-sys/blob/master/bindings/bindings-macos-aarch64-R4.0.rs
https://github.com/extendr/libR-sys/blob/master/bindings/bindings-macos-aarch64.rs

Could you please fetch/clone latest master branch and try

cargo test -vv -- --nocapture --test-threads=1

@psobolewskiPhD
Copy link

I think it worked?

 ~/Dev git clone https://github.com/extendr/libR-sys.git             
Cloning into 'libR-sys'...
remote: Enumerating objects: 32, done.
remote: Counting objects: 100% (32/32), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 12432 (delta 12), reused 12 (delta 4), pack-reused 12400
Receiving objects: 100% (12432/12432), 1.99 MiB | 1.44 MiB/s, done.
Resolving deltas: 100% (11995/11995), done.

~/Dev cd libR-sys/                                                  
 ~/Dev/libR-sys cargo test -- --nocapture --test-threads=1 
    Updating crates.io index
   Compiling libR-sys v0.2.0 (/Users/piotrsobolewski/Dev/libR-sys)
    Finished test [unoptimized + debuginfo] target(s) in 5.41s
     Running target/debug/deps/libR_sys-2480a01805fd53b0

running 21 tests
test bindgen_test_layout_R_CMethodDef ... ok
test bindgen_test_layout_R_CallMethodDef ... ok
test bindgen_test_layout_R_inpstream_st ... ok
test bindgen_test_layout_R_outpstream_st ... ok
test bindgen_test_layout_Rcomplex ... ok
test bindgen_test_layout___darwin_pthread_handler_rec ... ok
test bindgen_test_layout___mbstate_t ... ok
test bindgen_test_layout___sFILE ... ok
test bindgen_test_layout___sbuf ... ok
test bindgen_test_layout__opaque_pthread_attr_t ... ok
test bindgen_test_layout__opaque_pthread_cond_t ... ok
test bindgen_test_layout__opaque_pthread_condattr_t ... ok
test bindgen_test_layout__opaque_pthread_mutex_t ... ok
test bindgen_test_layout__opaque_pthread_mutexattr_t ... ok
test bindgen_test_layout__opaque_pthread_once_t ... ok
test bindgen_test_layout__opaque_pthread_rwlock_t ... ok
test bindgen_test_layout__opaque_pthread_rwlockattr_t ... ok
test bindgen_test_layout__opaque_pthread_t ... ok
test bindgen_test_layout_exception ... ok
test bindgen_test_layout_structRstart ... ok
test tests::test_eval ... [1] 1
ok

test result: ok. 21 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

   Doc-tests libR-sys

running 1 test
test src/lib.rs - (line 18) ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out

@clauswilke
Copy link
Member Author

Fantastic! This was all around easier than I expected. I'm closing this issue. Let's open a new one if any problems crop up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants