Zig it

[lptr]

Some experiments with cross-compilation using Zig's C/C++ cross-compilation support. My efforts are focused on getting file-events to cross-compile, though they should be relatively simple to extend to the rest of native-platform.

I used Zig 0.13, the currently released version. I installed it with brew install zig. The recommendation is to use the nightly, though, as Zig is moving fast; perhaps that would take us further in some ways.

Attempt 1: rewire Gradle

At first I tried getting Gradle to use Zig's commands to build by adding this under model {}:

model {
    toolChains {
        clang(Clang) {
            eachPlatform {
                cCompiler.executable = "zcc"
                cppCompiler.executable = "z++"
            }
        }
    }
    // ...
}

...and adding a couple bash scripts to invoke zig cc and zig c++ for those names. This got me to compile the code for the local platform on macOS, and I could run the tests and all. 🎉

However, I couldn't get multi-platform working this way, as I could get Zig's linker to step in.

Attempt 2: use Zig build

I then went with another direction, and tried to re-phrase the native part of the build in Zig.

Cross-compiling on macOS to Linux

This worked all the way, and I can now cross-compile on macOS like this:

$ uname -a
Darwin N2W52L96LD 23.6.0 Darwin Kernel Version 23.6.0: Mon Jul 29 21:14:30 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6000 arm64
$ zig build build -Dtarget=x86_64-linux-gnu
$ file zig-out/lib/libfile-events.so 
zig-out/lib/libfile-events.so: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, with debug_info, not stripped

🎉

I haven't tried the library yet from Java, but I think it's highly likely that it would work.

Cross-compilation to aarch64 works, too:

$ zig build build -Dtarget=aarch64-linux-gnu  
$ file zig-out/lib/libfile-events.so 
zig-out/lib/libfile-events.so: ELF 64-bit LSB shared object, ARM aarch64, version 1 (SYSV), dynamically linked, with debug_info, not stripped

Compiling locally on macOS

Compilation works well, and I got object files. However, the linker fails because it can't find the CoreFoundation library. I'm not completely sure how to fix this.

Compilation and linking works when not cross-compiling.

It also looks like if we wanted to cross-compile macOS on Linux, we'd need something like https://github.com/tpoechtrager/osxcrossd.

Notes on cross-compiling on Linux: ziglang/zig#1349. We essentially need the macOS SDKs packaged up and available, as detailed in https://github.com/tpoechtrager/osxcross#packaging-the-sdk.

Another option is to do cross-compilation for all platforms on a Mac where the SDK is available.

Cross-compiling to Windows

This worked the worst. IIUC, MSVC C++ support is just not there yet in Zig (ziglang/zig#5312). We could use the GNU toolchain, which would require some slight adjustments to the code. I did not try that yet.

Got this working with the GNU toolchain with a few small modifications to the code:

$ zig build build -Dtarget=x86_64-windows-gnu 
$ file zig-out/bin/file-events.dll 
zig-out/bin/file-events.dll: PE32+ executable (DLL) (GUI) x86-64, for MS Windows

I could also cross-compile to aarch64:

$ zig build build -Dtarget=aarch64-windows-gnu 
$ file zig-out/bin/file-events.dll                   
zig-out/bin/file-events.dll: PE32+ executable (DLL) (GUI) Aarch64, for MS Windows

Shrinking

This is as simple as passing -Doptimize=ReleaseSmall on the command-line. Each platform/arch builds to a ~500 kB library that compresses down to around 140 kB with gzip (the size we need to ship). This is somewhat larger for macOS where we currently ship ~80 kB gzip'd, but much smaller for Windows and Linux where we ship 3-400 kB for each platform/arch. Overall we'd save space, even after adding more platforms like Windows and Linux ARM.

TODO

• Try Zig nightly
• Integrate Zig build into the Gradle build
• Use Zig also for native-platform (not just file-events)
• Make this work on CI (we can build on Zig's Docker image)
• Shrink the size of the produced libraries (we can strip symbols etc.)
• Figure out how to build for macOS
• Get cross-arch compilation working
• Rework Windows code to work with GNU toolchain

[lptr]

Managed to get Zig running via Docker like this:

$ docker run -it -v $(pwd):/app -w /app -v $JAVA_HOME:/java:ro -e JAVA_HOME=/java chainguard/zig:latest build build

[lptr]

This has been superseded by https://github.com/lptr/file-events.