Building from local sources causes two macosx wheels to be produced

[rgommers]

I am getting a curious failure when building from a local source path, that I'm not getting when building from an sdist from PyPI with the exact same content. The error happens when building numpy on macOS 14 and looks like this:

% rattler-build build
...
 │ │ 4/1044] /Users/rgommers/code/tmp/trypixi/output/bld/rattler-build_numpy_1722879674/work/numpy/_core/tests/examples/limited_api/setup.py
 │ │ Successfully built numpy-2.0.1-cp312-cp312-macosx_11_0_arm64.whl
 │ │ + $PREFIX/bin/python -m pip install dist/numpy-2.0.1-cp312-cp312-macosx_11_0_arm64.whl dist/numpy-2.1.0.dev0-cp312-cp312-macosx_14_0_arm64.whl
 │ │ Processing ./dist/numpy-2.0.1-cp312-cp312-macosx_11_0_arm64.whl
 │ │ Processing ./dist/numpy-2.1.0.dev0-cp312-cp312-macosx_14_0_arm64.whl

What is happening is that when I build numpy 2.0.1 from PyPI, things works as expected and a conda package rolls out with a macosx_11_0 wheel as the intermediate binary wheel; with a local git repo checkout of the v2.0.1 tag the build chokes because two wheels appear (a macosx_11_0 one which is expected, and a macosx_14_0 one which is not).

Here is the recipe.yaml (modified from the example in the docs). It works when uncommenting the source: - url: lines and commenting out the source: path: lines:

context:
  version: 2.0.1

package:
  name: numpy
  version: ${{ version }}

# source:
#  - url: https://github.com/numpy/numpy/releases/download/v${{ version }}/numpy-${{ version }}.tar.gz
#    sha256: 485b87235796410c3519a699cfe1faab097e509e90ebb05dcd098db2ae87e7b3

source:
  path: ./numpy
  use_gitignore: false 

build:
  python:
    entry_points:
      - f2py = numpy.f2py.f2py2e:main  # [win]
      - numpy-config = 'numpy._configtool:main'

requirements:
  build:
    - ${{ compiler('c') }}
    - ${{ compiler('cxx') }}
    - ninja
    - cython
    - python-build
    - meson-python
  host:
    # note: variant is injected here!
    - python
    - pkg-config
    - pip
    - libblas
    - libcblas
    - liblapack
  run:
    - python
  run_exports:
    - ${{ pin_subpackage("numpy") }}

tests:
  - python:
      imports:
        - numpy
        - numpy.fft
        - numpy.linalg
        - numpy.ctypeslib

  - script:
    - f2py -h

about:
  homepage: http://numpy.org/
  license: BSD-3-Clause
  license_file: LICENSE.txt
  summary: The fundamental package for scientific computing with Python.
  documentation: https://numpy.org/doc/stable/
  repository: https://github.com/numpy/numpy

And the build.sh:

mkdir builddir

# Note: changed `$PYTHON` to `python` here
python -m build -w -n -x \
    -Cbuilddir=builddir \
    -Csetup-args=-Dblas=blas \
    -Csetup-args=-Dlapack=lapack

$PYTHON -m pip install dist/numpy*.whl

Looking at the timestamps of the wheels, the 14_0 old is old:

% ls -l output/bld/rattler-build_numpy_1722879674/work/dist/
total 24400
-rw-r--r--  1 rgommers  staff  6231769 Aug  5 19:42 numpy-2.0.1-cp312-cp312-macosx_11_0_arm64.whl
-rw-r--r--  1 rgommers  staff  6258321 Aug  3 16:46 numpy-2.1.0.dev0-cp312-cp312-macosx_14_0_arm64.whl

Debugging this is slow, so I thought I'd open a bug report at this point. Questions I have:

• Is it expected that there is any difference between building from PyPI and from local sources (e.g., a MACOSX_DEPLOYMENT_TARGET environment variable is or isn't set on purpose)?
• Is it possible there's a caching bug that re-inserts an older wheel for local builds somehow, because path: ./numpy doesn't have a distinguishable name that changes when the version changes?

[wolfv]

Thanks for testing and the bug report! That is indeed pretty curious.

Is your local folder relatively clean (ie. what does git clean -n say? I wonder if there is some weird interaction going on.

Also do things change if you enable the use of .gitignore?

[rgommers]

Is your local folder relatively clean (ie. what does git clean -n say?

Ah I should have added: since I cannot easily (at least for now) upstream this into the main numpy repo, I was experimenting by putting all metadata files in a directory one level up from the git repo itself (so not under version control).

The folder was quite clean, nothing but rattler-build and pixi metdata files in it, plus a clone of the numpy repo. I did rm -rf output to ensure I don't get a clash there with previous build outputs.

Creating a git repo in this folder like so doesn't make a difference:

git init
git add pixi.lock pixi.toml recipe.yaml variants.yaml .gitignore .gitattributes build.sh
git commit

and then adding numpy to .gitignore in a second commit still doesn't make a difference.

I'll start over to see if I can come up with a clean reproducer.

---

Perhaps unrelated hiccup found during debugging: I hit Ctrl-C in one run after the wheel was created (because the huge amount of lines that are printed for info/recipe/output/bld/... make my terminal swallow any build output higher up EDIT: the large output was because of gh-981), and after that rerunning the build makes rattler-build panic:

rattler-build build

 ╭─ Finding outputs from recipe
 │ Found 1 variants
 │ Build variant: numpy-2.1.0.dev0-h60d57d3_0
 │
 │ ╭─────────────────┬───────────╮
 │ │ Variant         ┆ Version   │
 │ ╞═════════════════╪═══════════╡
 │ │ target_platform ┆ osx-arm64 │
 │ ╰─────────────────┴───────────╯
 │
 ╰─────────────────── (took 0 seconds)

 ╭─ Checking existing builds
 │
 ╰─────────────────── (took 0 seconds)

 ╭─ Running build for recipe: numpy-2.1.0.dev0-h60d57d3_0
thread 'main' panicked at /Users/runner/miniforge3/conda-bld/rattler-build_1722536715281/_build_env/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rattler_index-0.19.21/src/lib.rs:82:55:
Could not open conda file: ZipError(InvalidArchive("No valid central directory found"))
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 │
 ╰─────────────────── (took 9 seconds)

With RUST_BACKTRACE=1:

RUST_BACKTRACE=1 rattler-build build

 ╭─ Finding outputs from recipe
 │ Found 1 variants
 │ Build variant: numpy-2.1.0.dev0-h60d57d3_0
 │
 │ ╭─────────────────┬───────────╮
 │ │ Variant         ┆ Version   │
 │ ╞═════════════════╪═══════════╡
 │ │ target_platform ┆ osx-arm64 │
 │ ╰─────────────────┴───────────╯
 │
 ╰─────────────────── (took 0 seconds)

 ╭─ Checking existing builds
 │
 ╰─────────────────── (took 0 seconds)

 ╭─ Running build for recipe: numpy-2.1.0.dev0-h60d57d3_0
thread 'main' panicked at /Users/runner/miniforge3/conda-bld/rattler-build_1722536715281/_build_env/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rattler_index-0.19.21/src/lib.rs:82:55:
Could not open conda file: ZipError(InvalidArchive("No valid central directory found"))
stack backtrace:
   0: _rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: rattler_index::package_record_from_conda
   4: rattler_index::index
   5: rattler_build::build::run_build::{{closure}}
   6: rattler_build::main::{{closure}}
   7: tokio::runtime::park::CachedParkThread::block_on
   8: tokio::runtime::runtime::Runtime::block_on
   9: rattler_build::main
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
 │
 ╰─────────────────── (took 9 seconds)

With RUST_BACKTRACE=full:

RUST_BACKTRACE=full rattler-build build

 ╭─ Finding outputs from recipe
 │ Found 1 variants
 │ Build variant: numpy-2.1.0.dev0-h60d57d3_0
 │
 │ ╭─────────────────┬───────────╮
 │ │ Variant         ┆ Version   │
 │ ╞═════════════════╪═══════════╡
 │ │ target_platform ┆ osx-arm64 │
 │ ╰─────────────────┴───────────╯
 │
 ╰─────────────────── (took 0 seconds)

 ╭─ Checking existing builds
 │
 ╰─────────────────── (took 0 seconds)

 ╭─ Running build for recipe: numpy-2.1.0.dev0-h60d57d3_0
thread 'main' panicked at /Users/runner/miniforge3/conda-bld/rattler-build_1722536715281/_build_env/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rattler_index-0.19.21/src/lib.rs:82:55:
Could not open conda file: ZipError(InvalidArchive("No valid central directory found"))
stack backtrace:
   0:        0x101064624 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h1f3776e0b5c7517d
   1:        0x1010888f4 - core::fmt::write::heedef092c8c0962e
   2:        0x10105ff7c - std::io::Write::write_fmt::h7178e8e2ea928914
   3:        0x10106447c - std::sys_common::backtrace::print::h417292deb95532ed
   4:        0x1010657fc - std::panicking::default_hook::{{closure}}::h0cb68f1228c4613a
   5:        0x1010654f0 - std::panicking::default_hook::h24535936bc1f51de
   6:        0x1010660b4 - std::panicking::rust_panic_with_hook::h5db4d2345b297bed
   7:        0x101065ae4 - std::panicking::begin_panic_handler::{{closure}}::h3fd558f09a0d5492
   8:        0x101064aac - std::sys_common::backtrace::__rust_end_short_backtrace::hfc76eebe1ce501b2
   9:        0x101065854 - _rust_begin_unwind
  10:        0x101103104 - core::panicking::panic_fmt::hc2b459a5bd3dce66
  11:        0x101103478 - core::result::unwrap_failed::h88d7eb352f20b747
  12:        0x100c45fc8 - rattler_index::package_record_from_conda::h89fcce1ce1baf83e
  13:        0x100c46a50 - rattler_index::index::h164e6f46bc12bc3a
  14:        0x1005da5d8 - rattler_build::build::run_build::{{closure}}::hef92c0cd6b69af98
  15:        0x1003ca310 - rattler_build::main::{{closure}}::hd2fc96a409506382
  16:        0x1003bf860 - tokio::runtime::park::CachedParkThread::block_on::he1907b58a0e17e97
  17:        0x10049d804 - tokio::runtime::runtime::Runtime::block_on::hf0c0982d00f56672
  18:        0x10044cca0 - rattler_build::main::h10c015f819cf6e00
  19:        0x100559b28 - std::sys_common::backtrace::__rust_begin_short_backtrace::h830500427654ee7b
  20:        0x10053c8b4 - std::rt::lang_start::{{closure}}::h4badbcfbb7092d66
  21:        0x101057a30 - std::rt::lang_start_internal::hecc68fef83c8f44d
  22:        0x10044ceb0 - _main
 │
 ╰─────────────────── (took 9 seconds)

[rgommers]

Okay, I figured it out - what happened is that I ran python -m build -wnx inside the numpy clone and then afterwards tried building a conda package. So the source dir already had a dist folder with a wheel in it. Building multiple wheels inside the same repo normally works, but having a wheel present is problematic for rattler-build because the recipe contains a pip install dist/*.whl step which only works if a single wheel is present.

This will be a minor annoyance when setting up dev tasks with a local source path, but it's not a caching bug as I initially suspected. I think this issue can be closed @wolfv (unless the panic requires follow-up).

[wolfv]

OK, that's good to hear :) Maybe you can run rm dist/* or add it to your .gitignore so that it's not copied into the build folder at all?

Regarding the invalid conda file – I'll open a new issue. We should not produce invalid conda files. We should instead create the package in the tmpdir first, and then move it over once done (so that we don't end up with broken packages in the output folder).

[rgommers]

Thanks! Yes, both of those solutions should work. dist/ was already in the .gitignore of the numpy repo, but that didn't help because the recipe wasn't part of the same repo.