Can't build libz-ng-sys on i686-pc-windows-msvc

[EliahKagan]

Building fails with an error getting the VCTargetsPath

On a 32-bit x86 Windows 10 system (build 19041.vb_release.191206-1406), where Rust is using the Visual C++ Build tools (rustup toolchain stable-i686-pc-windows-msvc), attempting commands such as cargo build or cargo test in any projects that use libz-ng-sys fails with the error:

  CMake Error at CMakeLists.txt:35 (project):
    Failed to run MSBuild command:

      C:/Program Files/Microsoft Visual Studio/2022/BuildTools/MSBuild/Current/Bin/amd64/MSBuild.exe

    to get the value of VCTargetsPath:

      unknown error

The amd64 part looks wrong, since this is not cross-compilation and this is a 32-bit system. That directory does exist, but it seems odd that it would be used here and perhaps that is relevant.

Discovery and relationship to gitoxide

I originally had this problem when attempting to build or install gitoxide. It happens both with cargo install gitoxide and with commands like cargo build invoked in the cloned gitoxide repository directory. The problem appears specific to libz-ng-sys, though I am unsure if the problem is in this Rust create or in the underlying native library.

• Running cargo install gitoxide --no-default-features --features max-pure allows the build to succeed by not depending on libz-ng-sys.
• Running cargo build --release --no-default-features --features max-control,gix-features/zlib-stock,gitoxide-core-blocking-client,http-client-curl, which is documented to have the same effect as the default max feature except to use "the stock zlib library instead of zlib-ng," likewise succeeds.

Why this is not specific to gitoxide

I don't think the bug--assuming this is a bug, rather than a configuration problem that I may not properly understand--is in gitoxide, since this always happens with libz-ng-sys. Here's a simpler example, in a project that doesn't have anything else going on besides its libz-ng-sys dependency:

PS C:\Users\ek\repos> cargo new try-libz-ng-sys
    Creating binary (application) `try-libz-ng-sys` package
note: see more `Cargo.toml` keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
PS C:\Users\ek\repos> cd .\try-libz-ng-sys\
PS C:\Users\ek\repos\try-libz-ng-sys> cargo add libz-ng-sys
    Updating crates.io index
      Adding libz-ng-sys v1.1.15 to dependencies
    Updating crates.io index
     Locking 5 packages to latest compatible versions
PS C:\Users\ek\repos\try-libz-ng-sys> cargo build
  Downloaded cmake v0.1.50
  Downloaded cc v1.0.102
  Downloaded libc v0.2.155
  Downloaded libz-ng-sys v1.1.15
  Downloaded 4 crates (3.2 MB) in 3.16s (largest was `libz-ng-sys` at 2.4 MB)
   Compiling cc v1.0.102
   Compiling libc v0.2.155
   Compiling cmake v0.1.50
   Compiling libz-ng-sys v1.1.15
error: failed to run custom build command for `libz-ng-sys v1.1.15`

Caused by:
  process didn't exit successfully: `C:\Users\ek\repos\try-libz-ng-sys\target\debug\build\libz-ng-sys-80a5fec7b53b81c4\build-script-build_zng` (exit code: 101)
  --- stdout
  CMAKE_TOOLCHAIN_FILE_i686-pc-windows-msvc = None
  CMAKE_TOOLCHAIN_FILE_i686_pc_windows_msvc = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_i686-pc-windows-msvc = None
  CMAKE_GENERATOR_i686_pc_windows_msvc = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_i686-pc-windows-msvc = None
  CMAKE_PREFIX_PATH_i686_pc_windows_msvc = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_i686-pc-windows-msvc = None
  CMAKE_i686_pc_windows_msvc = None
  HOST_CMAKE = None
  CMAKE = None
  running: "cmake" "C:\\Users\\ek\\.cargo\\registry\\src\\index.crates.io-1cd66030c949c28d\\libz-ng-sys-1.1.15\\src/zlib-ng" "-G" "Visual Studio 17 2022" "-Thost=x86" "-AWin32" "-DBUILD_SHARED_LIBS=OFF" "-DZLIB_COMPAT=OFF" "-DZLIB_ENABLE_TESTS=OFF" "-DWITH_GZFILEOP=ON" "-DCMAKE_GENERATOR_PLATFORM=Win32" "-DCMAKE_INSTALL_PREFIX=C:\\Users\\ek\\repos\\try-libz-ng-sys\\target\\debug\\build\\libz-ng-sys-f444bf96aef349e8\\out" "-DCMAKE_C_FLAGS= -nologo -MD -Brepro" "-DCMAKE_C_FLAGS_DEBUG= -nologo -MD -Brepro" "-DCMAKE_CXX_FLAGS= -nologo -MD -Brepro" "-DCMAKE_CXX_FLAGS_DEBUG= -nologo -MD -Brepro" "-DCMAKE_ASM_FLAGS= -nologo -MD -Brepro" "-DCMAKE_ASM_FLAGS_DEBUG= -nologo -MD -Brepro" "-DCMAKE_BUILD_TYPE=Debug"
  -- Using CMake version 3.29.6
  -- ZLIB_HEADER_VERSION: 1.3.0
  -- ZLIBNG_HEADER_VERSION: 2.1.6
  -- Selecting Windows SDK version 10.0.20348.0 to target Windows 10.0.19045.
  -- Configuring incomplete, errors occurred!

  --- stderr
  CMake Error at CMakeLists.txt:35 (project):
    Failed to run MSBuild command:

      C:/Program Files/Microsoft Visual Studio/2022/BuildTools/MSBuild/Current/Bin/amd64/MSBuild.exe

    to get the value of VCTargetsPath:

      unknown error



  thread 'main' panicked at C:\Users\ek\.cargo\registry\src\index.crates.io-1cd66030c949c28d\cmake-0.1.50\src\lib.rs:1098:5:

  command did not execute successfully, got: exit code: 1

  build script failed, must exit now
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Why this is probably not related to the cmake crate code

I don't believe the problem is in cmake-rs. Setting environment variables to get a readable backtrace and repeating the build gives:

PS C:\Users\ek\repos\try-libz-ng-sys> $env:RUST_BACKTRACE = '1'
PS C:\Users\ek\repos\try-libz-ng-sys> $env:CARGO_PROFILE_DEV_BUILD_OVERRIDE_DEBUG = 'true'
PS C:\Users\ek\repos\try-libz-ng-sys> cargo build
   Compiling cc v1.0.102
   Compiling libc v0.2.155
   Compiling cmake v0.1.50
   Compiling libz-ng-sys v1.1.15
error: failed to run custom build command for `libz-ng-sys v1.1.15`

Caused by:
  process didn't exit successfully: `C:\Users\ek\repos\try-libz-ng-sys\target\debug\build\libz-ng-sys-c8c2e254c3f755b6\build-script-build_zng` (exit code: 101)
  --- stdout
  CMAKE_TOOLCHAIN_FILE_i686-pc-windows-msvc = None
  CMAKE_TOOLCHAIN_FILE_i686_pc_windows_msvc = None
  HOST_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_i686-pc-windows-msvc = None
  CMAKE_GENERATOR_i686_pc_windows_msvc = None
  HOST_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_i686-pc-windows-msvc = None
  CMAKE_PREFIX_PATH_i686_pc_windows_msvc = None
  HOST_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_i686-pc-windows-msvc = None
  CMAKE_i686_pc_windows_msvc = None
  HOST_CMAKE = None
  CMAKE = None
  running: "cmake" "C:\\Users\\ek\\.cargo\\registry\\src\\index.crates.io-1cd66030c949c28d\\libz-ng-sys-1.1.15\\src/zlib-ng" "-G" "Visual Studio 17 2022" "-Thost=x86" "-AWin32" "-DBUILD_SHARED_LIBS=OFF" "-DZLIB_COMPAT=OFF" "-DZLIB_ENABLE_TESTS=OFF" "-DWITH_GZFILEOP=ON" "-DCMAKE_GENERATOR_PLATFORM=Win32" "-DCMAKE_INSTALL_PREFIX=C:\\Users\\ek\\repos\\try-libz-ng-sys\\target\\debug\\build\\libz-ng-sys-4f8f5509e9e4f77a\\out" "-DCMAKE_C_FLAGS= -nologo -MD -Brepro" "-DCMAKE_C_FLAGS_DEBUG= -nologo -MD -Brepro" "-DCMAKE_CXX_FLAGS= -nologo -MD -Brepro" "-DCMAKE_CXX_FLAGS_DEBUG= -nologo -MD -Brepro" "-DCMAKE_ASM_FLAGS= -nologo -MD -Brepro" "-DCMAKE_ASM_FLAGS_DEBUG= -nologo -MD -Brepro" "-DCMAKE_BUILD_TYPE=Debug"
  -- Using CMake version 3.29.6
  -- ZLIB_HEADER_VERSION: 1.3.0
  -- ZLIBNG_HEADER_VERSION: 2.1.6
  -- Selecting Windows SDK version 10.0.20348.0 to target Windows 10.0.19045.
  -- Configuring incomplete, errors occurred!

  --- stderr
  CMake Error at CMakeLists.txt:35 (project):
    Failed to run MSBuild command:

      C:/Program Files/Microsoft Visual Studio/2022/BuildTools/MSBuild/Current/Bin/amd64/MSBuild.exe

    to get the value of VCTargetsPath:

      unknown error



  thread 'main' panicked at C:\Users\ek\.cargo\registry\src\index.crates.io-1cd66030c949c28d\cmake-0.1.50\src\lib.rs:1098:5:

  command did not execute successfully, got: exit code: 1

  build script failed, must exit now
  stack backtrace:
     0: std::panicking::begin_panic_handler
               at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library\std\src\panicking.rs:652
     1: core::panicking::panic_fmt
               at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library\core\src\panicking.rs:72
     2: cmake::fail
               at C:\Users\ek\.cargo\registry\src\index.crates.io-1cd66030c949c28d\cmake-0.1.50\src\lib.rs:1098
     3: alloc::alloc::realloc
               at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library\alloc\src\alloc.rs:138
     4: alloc::alloc::Global::grow_impl
               at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library\alloc\src\alloc.rs:215
     5: alloc::alloc::impl$1::grow
               at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library\alloc\src\alloc.rs:268
     6: alloc::raw_vec::finish_grow<alloc::alloc::Global>
               at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library\alloc\src\raw_vec.rs:570
     7: cmake::Config::build
               at C:\Users\ek\.cargo\registry\src\index.crates.io-1cd66030c949c28d\cmake-0.1.50\src\lib.rs:813
     8: build_script_build_zng::build_zlib_ng
               at .\build_zng.rs:21
     9: build_script_build_zng::main
               at .\build_zng.rs:59
    10: core::ops::function::FnOnce::call_once<void (*)(),tuple$<> >
               at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081\library\core\src\ops\function.rs:250
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

The reference to cmake-rs code, other than its fail function, is to this invocation of the cmake command:

run(cmd.env("CMAKE_PREFIX_PATH", cmake_prefix_path), "cmake");

So that's just the cmake external command failing.

The system has a working cmake command

I don't think the problem is with cmake itself. It is installed and otherwise working:

PS C:\Users\ek\repos\try-libz-ng-sys> cmake --version
cmake version 3.29.6

CMake suite maintained and supported by Kitware (kitware.com/cmake).

Toolchain and environment variations tested

Because Visual Studio 2022 does not support 32-bit systems, I obtained the needed MSVC toolchain by installing the Visual Studio C++ build tools, which is also an approach listed in the Getting started instructions for Rust on Windows. Specifically, I installed the Visual Studio Build Tools 2022, version 17.10.3, selecting "Desktop development with C++."

• Originally I went with the default selections, which included "Windows 11 SDK (10.0.22621.0)" and no other Windows SDKs. The build attempted to use this SDK. But it is a Windows 11 SDK, on a Windows 10 system. Just in case that was the problem, I uninstalled all Rust tools with rustup, verified that my .cargo directory was deleted, uninstalled that Windows 11 SDK, and installed all four of the Windows 10 SDKs instead (10.0.20348.0, 10.0.19041.0, and 10.0.18362.0) in the Visual Studio Installer, and then bootstrapped rustup again and started over. The above output is all from after doing that.

• Although I would not expect cargo commands to require binaries provided by those build tools to be in the PATH (and other builds, including e.g. those that use curl-sys, are working), I have repeated the above from the Developer PowerShell for VS 2022, which has them. This gave the same result. I did this both times, with the default build tools installation that had a Windows 11 SDK, and after modifying that installation to have no Windows 11 SDKs and all available Windows 10 SDKs.

[Byron]

Thanks so much for the detailed report, and for trying to get to the bottom of this!

It looks like this is a known zlib-ng incompatibility, and 32bit support for Win MSVC might not be their priority or is entirely unsupported, hence CMake ends up failing.

There is two points I can share:

• there is the zlib-ng-no-cmake-experimental-community-maintained feature toggle which avoids using CMake. If this would work for gitoxide, maybe it should switch to it.
• The flate2 crate which gitoxide is using also has a zlib-ng-rs feature, maybe that will build on 32 bit platforms. If so, maybe it could be used on certain targets by default.

[EliahKagan]

• there is the zlib-ng-no-cmake-experimental-community-maintained feature toggle which avoids using CMake. If this would work for gitoxide, maybe it should switch to it.

Would this be while using the same crate as a dependency? The libz-ng-sys crate, which gitoxide depends on and which produces the error for it, doesn't seem to have a zlib-ng-no-cmake-experimental-community-maintained feature.

The libz-sys crate has zlib-ng and zlib-ng-no-cmake-experimental-community-maintained features, and when installing libz-sys an error like the one shown above occurs with the zlib-ng feature but is averted with the zlib-ng-no-cmake-experimental-community-maintained feature. But the error reported above is while installing the zlib-ng-sys crate.

[Byron]

Would this be while using the same crate as a dependency? The libz-ng-sys crate, which gitoxide depends on and which produces the error for it, doesn't seem to have a zlib-ng-no-cmake-experimental-community-maintained feature.

That's true, and it's because gitoxide uses the flate2 crate, which doesn't expose this feature toggle. It should nonetheless be possible to turn it on, by switching flate2 to using the libz-sys crate, and then by adding libz-sys as direct dependency to the respective gitoxide crate to be able to turn on its zlib-ng-no-cmake-experimental-community-maintained feature. At least in theory that could work, maybe along with target.cfg(…).dependencies conditionals.

The libz-sys crate has zlib-ng and zlib-ng-no-cmake-experimental-community-maintained features, and when installing libz-sys an error like the one shown above occurs with the zlib-ng feature but is averted with the zlib-ng-no-cmake-experimental-community-maintained feature. But the error reported above is while installing the zlib-ng-sys crate.

Great, this means that maybe using the zlib-ng-no-cmake-experimental-community-maintained on this target would be a fix. For users of flate2, this fix would have to be implemented there. And by depending on it, one would basically have to lift it from community-maintained to being officially maintained here, I presume.

Otherwise, if it doesn't work there anyway, then the worst that could happen is that it wouldn't work due to other reasons, so it seems like the right thing to do. This would probably also be the case for a fix in this case, which would play well with flate2 for example which then would only have to assure that on this platform, zlib-ng isn't actually used to avoid pulling in the libz-ng-sys crate.

When reading the above it all sounds a little convoluted, so maybe one could instead also add the zlib-ng-no-cmake-experimental-community-maintained feature toggle to the libz-ng-sys crate and maybe automatically use it in the
stable-i686-pc-windows-msvc target? That way, all would be fine for downstream crates automatically.