Running cargo test fails out-of-the-box on an M1 Mac

[data-sync-user]

Some notes from a discussion earlier today with @skhamis; please add your own notes here as appropriate.

On a checkout of uniffi-rs on an M1 Mac, cargo test fails when running the Kotlin bindings tests. IIUC the issue we ran into was:

• On the M1, cargo builds native libraries as arm64 by default.
• The JVM installed on the M1 was an x86_64 binary, meaning that Java programs would run in virtualized mode.
• Under such circumstances, JNA (rightfully!) wants to load x64_64 dylibs, and is not able to load arm64 dylibs. (ref Platform.RESOURCE_PREFIX on Darwin x64 and M1 java-native-access/jna#1313 for a bit of discussion about this in JNA).
• So when we try to run the UniFFI kotlin bindings tests we:
  • cargo build the target crate for the native host, making an arm64 dylib.
  • Run java to execute the Kotlin tests, which launches as a virtualized executable.
  • The tests try to load the dylib and error out, complaining that it is the wrong architecture.
    • (Which is misleading from a debugging point of view, because it's java that is the "wrong" architecture)

Two things we could try to solve this:

• Figure out how to get an M1 native Java environment, and document it in our README.
• Figure out how to make cargo build x86_64 binaries for use in these tests.

┆Issue is synchronized with this Jira Task

[data-sync-user]

➤ Sammy Khamis commented:

This is an excellent write-up! I think you covered everything here. The only thing I'll add is the actual error log for verbosity:

java.lang.UnsatisfiedLinkError: dlopen(/Users/skhamis/moz/uniffi-rs/target/debug/libuniffi_coverall.dylib, 9): no suitable image found. Did find:
/Users/skhamis/moz/uniffi-rs/target/debug/libuniffi_coverall.dylib: mach-o, but wrong architecture
/Users/skhamis/moz/uniffi-rs/target/debug/libuniffi_coverall.dylib: mach-o, but wrong architecture
at com.sun.jna.Native.open(Native Method)
at com.sun.jna.NativeLibrary.loadLibrary(NativeLibrary.java:277)
at com.sun.jna.NativeLibrary.getInstance(NativeLibrary.java:461)
at com.sun.jna.Library$Handler.(Library.java:192)
at com.sun.jna.Native.load(Native.java:596)
at com.sun.jna.Native.load(Native.java:570)
at uniffi.coverall._UniFFILib$Companion$INSTANCE$2.invoke(coverall.kt:1293)
at uniffi.coverall._UniFFILib$Companion$INSTANCE$2.invoke(coverall.kt:750)
at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:74)
at uniffi.coverall._UniFFILib$Companion.getINSTANCE$main(coverall.kt:750)
at uniffi.coverall.CoverallKt.createSomeDict(coverall.kt:1094)
at Test_coverall.(test_coverall.kts:12)
test uniffi_foreign_language_testcase_test_coverall_kts ... FAILED

failures:

    * uniffi_foreign_language_testcase_test_coverall_kts stdout ----

Error: running kotlinc failed
thread 'uniffi_foreign_language_testcase_test_coverall_kts' panicked at 'assertion failed: (left == right)
left: 1,
right: 0: the test returned a termination value with a non-zero status code (1) which indicates a failure', /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/test/src/lib.rs:193:5
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

failures:
uniffi_foreign_language_testcase_test_coverall_kts

test result: FAILED. 2 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 32.53sand running a file on the lib that is erroring out nets:
/Users/skhamis/moz/uniffi-rs/target/debug/libuniffi_coverall.dylib: Mach-O 64-bit dynamically linked shared library arm64

[data-sync-user]

➤ Sammy Khamis commented:

So wanted to update on this as I had time to finally fix it! The real issue was indeed having incompatible Java and JNA versions which is "fixed" by installing an arm64 java.

1. I ended up settling on using JEnv ( https://www.jenv.be/ ) which is a really nice way to manage multiple JVMs
2. I installed Azul JDK (offers native arm64 JVM) in addition to my existing OpenJDK (the original x86_64)
3. I specifically switched to use JVM arm64 for the uniffi-rs project
   All tests pass!

While this definitely over-complicates things for M1 users. I actually believe this is the way to go for the near future as it allows flexibility for arm64 and x86_64 when one or the other is needed for simple compiling. I do think this was in part contributing to "operator error" as I had x86_64 JVM but JNA auto-detects and "forces" arm64 since it detects that platform.

Don't necessarily know what the "right" way to fix this would be other than possibly detecting what JVM they have and generating the libs based on that but this seems really specific to Apple silicon. Possibly a doc that FAQ with what is happening? Definitely open to any ideas!