Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrator bot opens PRs ignoring number of children? #2961

Open
h-vetinari opened this issue Aug 29, 2024 · 17 comments
Open

Migrator bot opens PRs ignoring number of children? #2961

h-vetinari opened this issue Aug 29, 2024 · 17 comments

Comments

@h-vetinari
Copy link
Contributor

h-vetinari commented Aug 29, 2024

There are open PRs in the 3.13 migration with 0(!) children, while feedstocks with >2000 children are awaiting PR. This seems quite suboptimal as a strategy, if we don't tackle the biggest bottlenecks the earliest.

Bottom of the table filtered to "In PR" currently looks like this:

image

@h-vetinari h-vetinari changed the title Migrator bot is opening PRs ignoring number of children? Migrator bot opens PRs ignoring number of children? Aug 29, 2024
@h-vetinari
Copy link
Contributor Author

Actually, it seems to disregard dependencies at least in some cases? It opened a PR for numpy without ever opening a PR for cython, even though numpy depends on that explicitly.

CC @beckermr

@jakirkham
Copy link
Contributor

It's worth noting we needed to pause the migrator1 due to an absence of cross-python2

1: conda-forge/conda-forge-pinning-feedstock#6361
2: conda-forge/cross-python-feedstock#79

@h-vetinari
Copy link
Contributor Author

I know, but that has no bearing on this issue.

@beckermr
Copy link
Contributor

Yeah I have no idea here. Someone will have to load the thing and look at the graph. I won't get to it for a while, but feel free to go ahead.

@beckermr
Copy link
Contributor

So numpy is marked as part of a cycle in the graph for python. See this file and this entry

 "cycles": {
   "__set__": true,
   "elements": [
    "fenics-basix",
    "fenics-basix-meta",
    "gnuradio",
    "gnuradio-soapy",
    "heavydb-ext",
    "intel-compiler-repack",
    "numpy",
    "omniscidb",
    "pyopencl"
   ]
  },

Feedstocks in cycles always go first no matter what. The surprising thing is that the solver checks passed.

@beckermr
Copy link
Contributor

The bot also skips solver checks for nodes in cycles. So this PR is expected.

@beckermr
Copy link
Contributor

The final bit here is that the bot only opens PRs for things it can solve. So even if it tried a feedstock with more downstream packages, if that one does not solve, it will not make the PR. This explains the rest of the things. I am going to close this issue. Feel free to reopen!

@h-vetinari
Copy link
Contributor Author

So numpy is marked as part of a cycle in the graph for python.

That seems spurious, numpy shouldn't depend on any of these (I guess there's a longshot that it relies on intel repack through lapack and MKL).

Also, omniscidb is archived, so presumably shouldn't be part of any cycle calculations...

@beckermr beckermr reopened this Aug 29, 2024
@h-vetinari
Copy link
Contributor Author

I mixed up the intel feedstocks; the intel-compiler-repack at least has a spurious python dependency, which at least explains why the bot wants to build it for various pythons. We just need to add a * there IMO.

@beckermr
Copy link
Contributor

beckermr commented Aug 29, 2024

numpy in a loop is real. here is what the graph says:

intel-compiler-repack 
 pred: ['pyopencl'] 
 succ: ['numpy']
numpy 
 pred: ['intel-compiler-repack'] 
 succ: ['pyopencl']
pyopencl 
 pred: ['numpy'] 
 succ: ['intel-compiler-repack']

(edit: printed the cycle from the wrong graph)

The intel-compiler-repack feedstock has a test that depends on pyopencl:

https://github.com/conda-forge/intel-compiler-repack-feedstock/blob/ef8dbff1a48f66ef87660d9059d003f4e00fe68e/recipe/meta.yaml#L303-L314

@h-vetinari
Copy link
Contributor Author

numpy in a loop is real.

According to the current metadata perhaps, but it's all spurious IMO. With conda-forge/intel-compiler-repack-feedstock#47, we remove intel-compiler-repack from the migration completely, and the loop disappears.

@h-vetinari
Copy link
Contributor Author

h-vetinari commented Sep 3, 2024

So the loop is gone, but cython isn't being marked as finished, despite the bot saying cython: archived or done

Pulling out the logs from the most recent bot run:

MIGRATIONYAML-python313 IS MIGRATING cython
  2024-09-03 01:58:31,118 DEBUG    conda_forge_tick.migrators.migration_yaml || filter cython: need to wait for []
  2024-09-03 01:58:31,118 DEBUG    conda_forge_tick.migrators.core || cython: bad attr - {'exception': 'Error running \'conda-forge-feedstock-ops-container rerender --log-level info --timeout 900\' in container - error RuntimeError raised:\n\'Failed to rerender.\noutput: Adding in variants from internal_defaults\nAdding in variants from /tmp/tmpd4kqdxou/conda-smithy/conda_build_config.yaml\nINFO:conda_smithy.configure_feedstock:Downloading conda-forge-pinning-2024.09.02.15.16.22\nINFO:conda_smithy.configure_feedstock:Extracting conda-forge-pinning to /tmp/tmpd4kqdxou/conda-smithy\nINFO:conda_smithy.configure_feedstock:__pycache__ rendering is skipped\nINFO:conda_smithy.configure_feedstock:README rendering is skipped\nINFO:conda_smithy.configure_feedstock:pypy38.yaml is closed now. Removing\nWARNING: Setting build platform. This is only useful when pretending to be on another platform, such as for rendering necessary dependencies on a non-native platform. I trust that you know what you\'re doing.\nWARNING: Setting build arch. This is only useful when pretending to be on another arch, such as for rendering necessary dependencies on a non-native arch. I trust that you know what you\'re doing.\nWARNING: No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.23\nINFO:conda_smithy.configure_feedstock:Applying migrations: /tmp/tmpge8rwfr7/cython-feedstock/.ci_support/migrations/python312.yaml\nTraceback (most recent call last):\n  File "/opt/conda/envs/cf-scripts/bin/conda-smithy", line 10, in <module>\n    sys.exit(main())\n             ^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 758, in main\n    args.subcommand_func(args)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 601, in __call__\n    self._call(args, args.temporary_directory)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 604, in _call\n    configure_feedstock.main(\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 2785, in main\n    render_azure(env, config, forge_dir, return_metadata=True)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1871, in render_azure\n    return _render_ci_provider(\n           ^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1090, in _render_ci_provider\n    metas = _conda_build_api_render_for_smithy(\n            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 890, in _conda_build_api_render_for_smithy\n    metadata_tuples = render_recipe(\n                      ^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/render.py", line 990, in render_recipe\n    variants = get_package_variants(m, variants=variants)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 697, in get_package_variants\n    combined_spec, specs = get_package_combined_spec(\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 661, in get_package_combined_spec\n    validate_spec(f, spec)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 200, in validate_spec\n    raise ValueError(\nValueError: Variant configuration errors in argument_variants:\n  zip fields in zip_key group frozenset({\'numpy\', \'channel_sources\', \'python\', \'python_impl\'}) are not all the same length\n\n\'', 'traceback': ['Traceback (most recent call last):', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/auto_tick.py", line 581, in _run_migrator_on_feedstock_branch', '    migrator_uid, pr_json = run_with_tmpdir(', '                            ^^^^^^^^^^^^^^^^', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/auto_tick.py", line 161, in run_with_tmpdir', '    return run(', '           ^^^^', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/auto_tick.py", line 274, in run', '    rerender_msg = rerender_feedstock(feedstock_dir, timeout=900)', '                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/rerender_feedstock.py", line 24, in rerender_feedstock', '    return _rerender(feedstock_dir, timeout=timeout, use_container=use_container)', '           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/micromamba/envs/cf-scripts/lib/python3.11/site-packages/conda_forge_feedstock_ops/rerender.py", line 48, in rerender', '    return rerender_containerized(', '           ^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/micromamba/envs/cf-scripts/lib/python3.11/site-packages/conda_forge_feedstock_ops/rerender.py", line 110, in rerender_containerized', '    data = run_container_operation(', '           ^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/micromamba/envs/cf-scripts/lib/python3.11/site-packages/conda_forge_feedstock_ops/container_utils.py", line 183, in run_container_operation', '    raise ContainerRuntimeError(', "conda_forge_feedstock_ops.container_utils.ContainerRuntimeError: Error running 'conda-forge-feedstock-ops-container rerender --log-level info --timeout 900' in container - error RuntimeError raised:", "'Failed to rerender.", 'output: Adding in variants from internal_defaults', 'Adding in variants from /tmp/tmpd4kqdxou/conda-smithy/conda_build_config.yaml', 'INFO:conda_smithy.configure_feedstock:Downloading conda-forge-pinning-2024.09.02.15.16.22', 'INFO:conda_smithy.configure_feedstock:Extracting conda-forge-pinning to /tmp/tmpd4kqdxou/conda-smithy', 'INFO:conda_smithy.configure_feedstock:__pycache__ rendering is skipped', 'INFO:conda_smithy.configure_feedstock:README rendering is skipped', 'INFO:conda_smithy.configure_feedstock:pypy38.yaml is closed now. Removing', "WARNING: Setting build platform. This is only useful when pretending to be on another platform, such as for rendering necessary dependencies on a non-native platform. I trust that you know what you're doing.", "WARNING: Setting build arch. This is only useful when pretending to be on another arch, such as for rendering necessary dependencies on a non-native arch. I trust that you know what you're doing.", 'WARNING: No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.23', 'INFO:conda_smithy.configure_feedstock:Applying migrations: /tmp/tmpge8rwfr7/cython-feedstock/.ci_support/migrations/python312.yaml', 'Traceback (most recent call last):', '  File "/opt/conda/envs/cf-scripts/bin/conda-smithy", line 10, in <module>', '    sys.exit(main())', '             ^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 758, in main', '    args.subcommand_func(args)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 601, in __call__', '    self._call(args, args.temporary_directory)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 604, in _call', '    configure_feedstock.main(', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 2785, in main', '    render_azure(env, config, forge_dir, return_metadata=True)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1871, in render_azure', '    return _render_ci_provider(', '           ^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1090, in _render_ci_provider', '    metas = _conda_build_api_render_for_smithy(', '            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 890, in _conda_build_api_render_for_smithy', '    metadata_tuples = render_recipe(', '                      ^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/render.py", line 990, in render_recipe', '    variants = get_package_variants(m, variants=variants)', '               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 697, in get_package_variants', '    combined_spec, specs = get_package_combined_spec(', '                           ^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 661, in get_package_combined_spec', '    validate_spec(f, spec)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 200, in validate_spec', '    raise ValueError(', 'ValueError: Variant configuration errors in argument_variants:', "  zip fields in zip_key group frozenset({'numpy', 'channel_sources', 'python', 'python_impl'}) are not all the same length", '', "'", '']}
  2024-09-03 01:58:31,118 DEBUG    conda_forge_tick.migrators.core || filter cython: archived or done
  2024-09-03 01:58:31,118 INFO     conda_forge_tick.auto_tick || skipping node cython w/ branch main
  2024-09-03 01:58:31,118 DEBUG    conda_forge_tick.migrators.migration_yaml || filter cython: need to wait for []
  2024-09-03 01:58:31,118 DEBUG    conda_forge_tick.migrators.core || cython: bad attr - {'exception': 'Error running \'conda-forge-feedstock-ops-container rerender --log-level info --timeout 900\' in container - error RuntimeError raised:\n\'Failed to rerender.\noutput: Adding in variants from internal_defaults\nAdding in variants from /tmp/tmpd4kqdxou/conda-smithy/conda_build_config.yaml\nINFO:conda_smithy.configure_feedstock:Downloading conda-forge-pinning-2024.09.02.15.16.22\nINFO:conda_smithy.configure_feedstock:Extracting conda-forge-pinning to /tmp/tmpd4kqdxou/conda-smithy\nINFO:conda_smithy.configure_feedstock:__pycache__ rendering is skipped\nINFO:conda_smithy.configure_feedstock:README rendering is skipped\nINFO:conda_smithy.configure_feedstock:pypy38.yaml is closed now. Removing\nWARNING: Setting build platform. This is only useful when pretending to be on another platform, such as for rendering necessary dependencies on a non-native platform. I trust that you know what you\'re doing.\nWARNING: Setting build arch. This is only useful when pretending to be on another arch, such as for rendering necessary dependencies on a non-native arch. I trust that you know what you\'re doing.\nWARNING: No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.23\nINFO:conda_smithy.configure_feedstock:Applying migrations: /tmp/tmpge8rwfr7/cython-feedstock/.ci_support/migrations/python312.yaml\nTraceback (most recent call last):\n  File "/opt/conda/envs/cf-scripts/bin/conda-smithy", line 10, in <module>\n    sys.exit(main())\n             ^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 758, in main\n    args.subcommand_func(args)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 601, in __call__\n    self._call(args, args.temporary_directory)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 604, in _call\n    configure_feedstock.main(\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 2785, in main\n    render_azure(env, config, forge_dir, return_metadata=True)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1871, in render_azure\n    return _render_ci_provider(\n           ^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1090, in _render_ci_provider\n    metas = _conda_build_api_render_for_smithy(\n            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 890, in _conda_build_api_render_for_smithy\n    metadata_tuples = render_recipe(\n                      ^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/render.py", line 990, in render_recipe\n    variants = get_package_variants(m, variants=variants)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 697, in get_package_variants\n    combined_spec, specs = get_package_combined_spec(\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 661, in get_package_combined_spec\n    validate_spec(f, spec)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 200, in validate_spec\n    raise ValueError(\nValueError: Variant configuration errors in argument_variants:\n  zip fields in zip_key group frozenset({\'numpy\', \'channel_sources\', \'python\', \'python_impl\'}) are not all the same length\n\n\'', 'traceback': ['Traceback (most recent call last):', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/auto_tick.py", line 581, in _run_migrator_on_feedstock_branch', '    migrator_uid, pr_json = run_with_tmpdir(', '                            ^^^^^^^^^^^^^^^^', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/auto_tick.py", line 161, in run_with_tmpdir', '    return run(', '           ^^^^', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/auto_tick.py", line 274, in run', '    rerender_msg = rerender_feedstock(feedstock_dir, timeout=900)', '                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/rerender_feedstock.py", line 24, in rerender_feedstock', '    return _rerender(feedstock_dir, timeout=timeout, use_container=use_container)', '           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/micromamba/envs/cf-scripts/lib/python3.11/site-packages/conda_forge_feedstock_ops/rerender.py", line 48, in rerender', '    return rerender_containerized(', '           ^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/micromamba/envs/cf-scripts/lib/python3.11/site-packages/conda_forge_feedstock_ops/rerender.py", line 110, in rerender_containerized', '    data = run_container_operation(', '           ^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/micromamba/envs/cf-scripts/lib/python3.11/site-packages/conda_forge_feedstock_ops/container_utils.py", line 183, in run_container_operation', '    raise ContainerRuntimeError(', "conda_forge_feedstock_ops.container_utils.ContainerRuntimeError: Error running 'conda-forge-feedstock-ops-container rerender --log-level info --timeout 900' in container - error RuntimeError raised:", "'Failed to rerender.", 'output: Adding in variants from internal_defaults', 'Adding in variants from /tmp/tmpd4kqdxou/conda-smithy/conda_build_config.yaml', 'INFO:conda_smithy.configure_feedstock:Downloading conda-forge-pinning-2024.09.02.15.16.22', 'INFO:conda_smithy.configure_feedstock:Extracting conda-forge-pinning to /tmp/tmpd4kqdxou/conda-smithy', 'INFO:conda_smithy.configure_feedstock:__pycache__ rendering is skipped', 'INFO:conda_smithy.configure_feedstock:README rendering is skipped', 'INFO:conda_smithy.configure_feedstock:pypy38.yaml is closed now. Removing', "WARNING: Setting build platform. This is only useful when pretending to be on another platform, such as for rendering necessary dependencies on a non-native platform. I trust that you know what you're doing.", "WARNING: Setting build arch. This is only useful when pretending to be on another arch, such as for rendering necessary dependencies on a non-native arch. I trust that you know what you're doing.", 'WARNING: No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.23', 'INFO:conda_smithy.configure_feedstock:Applying migrations: /tmp/tmpge8rwfr7/cython-feedstock/.ci_support/migrations/python312.yaml', 'Traceback (most recent call last):', '  File "/opt/conda/envs/cf-scripts/bin/conda-smithy", line 10, in <module>', '    sys.exit(main())', '             ^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 758, in main', '    args.subcommand_func(args)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 601, in __call__', '    self._call(args, args.temporary_directory)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 604, in _call', '    configure_feedstock.main(', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 2785, in main', '    render_azure(env, config, forge_dir, return_metadata=True)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1871, in render_azure', '    return _render_ci_provider(', '           ^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1090, in _render_ci_provider', '    metas = _conda_build_api_render_for_smithy(', '            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 890, in _conda_build_api_render_for_smithy', '    metadata_tuples = render_recipe(', '                      ^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/render.py", line 990, in render_recipe', '    variants = get_package_variants(m, variants=variants)', '               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 697, in get_package_variants', '    combined_spec, specs = get_package_combined_spec(', '                           ^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 661, in get_package_combined_spec', '    validate_spec(f, spec)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 200, in validate_spec', '    raise ValueError(', 'ValueError: Variant configuration errors in argument_variants:', "  zip fields in zip_key group frozenset({'numpy', 'channel_sources', 'python', 'python_impl'}) are not all the same length", '', "'", '']}
  2024-09-03 01:58:31,118 DEBUG    conda_forge_tick.migrators.core || filter cython: archived or done
  2024-09-03 01:58:31,118 INFO     conda_forge_tick.auto_tick || skipping node cython w/ branch dev
  2024-09-03 01:58:31,118 DEBUG    conda_forge_tick.migrators.migration_yaml || filter cython: need to wait for []
  2024-09-03 01:58:31,118 DEBUG    conda_forge_tick.migrators.core || cython: bad attr - {'exception': 'Error running \'conda-forge-feedstock-ops-container rerender --log-level info --timeout 900\' in container - error RuntimeError raised:\n\'Failed to rerender.\noutput: Adding in variants from internal_defaults\nAdding in variants from /tmp/tmpd4kqdxou/conda-smithy/conda_build_config.yaml\nINFO:conda_smithy.configure_feedstock:Downloading conda-forge-pinning-2024.09.02.15.16.22\nINFO:conda_smithy.configure_feedstock:Extracting conda-forge-pinning to /tmp/tmpd4kqdxou/conda-smithy\nINFO:conda_smithy.configure_feedstock:__pycache__ rendering is skipped\nINFO:conda_smithy.configure_feedstock:README rendering is skipped\nINFO:conda_smithy.configure_feedstock:pypy38.yaml is closed now. Removing\nWARNING: Setting build platform. This is only useful when pretending to be on another platform, such as for rendering necessary dependencies on a non-native platform. I trust that you know what you\'re doing.\nWARNING: Setting build arch. This is only useful when pretending to be on another arch, such as for rendering necessary dependencies on a non-native arch. I trust that you know what you\'re doing.\nWARNING: No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.23\nINFO:conda_smithy.configure_feedstock:Applying migrations: /tmp/tmpge8rwfr7/cython-feedstock/.ci_support/migrations/python312.yaml\nTraceback (most recent call last):\n  File "/opt/conda/envs/cf-scripts/bin/conda-smithy", line 10, in <module>\n    sys.exit(main())\n             ^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 758, in main\n    args.subcommand_func(args)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 601, in __call__\n    self._call(args, args.temporary_directory)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 604, in _call\n    configure_feedstock.main(\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 2785, in main\n    render_azure(env, config, forge_dir, return_metadata=True)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1871, in render_azure\n    return _render_ci_provider(\n           ^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1090, in _render_ci_provider\n    metas = _conda_build_api_render_for_smithy(\n            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 890, in _conda_build_api_render_for_smithy\n    metadata_tuples = render_recipe(\n                      ^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/render.py", line 990, in render_recipe\n    variants = get_package_variants(m, variants=variants)\n               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 697, in get_package_variants\n    combined_spec, specs = get_package_combined_spec(\n                           ^^^^^^^^^^^^^^^^^^^^^^^^^^\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 661, in get_package_combined_spec\n    validate_spec(f, spec)\n  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 200, in validate_spec\n    raise ValueError(\nValueError: Variant configuration errors in argument_variants:\n  zip fields in zip_key group frozenset({\'numpy\', \'channel_sources\', \'python\', \'python_impl\'}) are not all the same length\n\n\'', 'traceback': ['Traceback (most recent call last):', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/auto_tick.py", line 581, in _run_migrator_on_feedstock_branch', '    migrator_uid, pr_json = run_with_tmpdir(', '                            ^^^^^^^^^^^^^^^^', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/auto_tick.py", line 161, in run_with_tmpdir', '    return run(', '           ^^^^', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/auto_tick.py", line 274, in run', '    rerender_msg = rerender_feedstock(feedstock_dir, timeout=900)', '                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/work/cf-scripts/cf-scripts/cf-scripts/conda_forge_tick/rerender_feedstock.py", line 24, in rerender_feedstock', '    return _rerender(feedstock_dir, timeout=timeout, use_container=use_container)', '           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/micromamba/envs/cf-scripts/lib/python3.11/site-packages/conda_forge_feedstock_ops/rerender.py", line 48, in rerender', '    return rerender_containerized(', '           ^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/micromamba/envs/cf-scripts/lib/python3.11/site-packages/conda_forge_feedstock_ops/rerender.py", line 110, in rerender_containerized', '    data = run_container_operation(', '           ^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/home/runner/micromamba/envs/cf-scripts/lib/python3.11/site-packages/conda_forge_feedstock_ops/container_utils.py", line 183, in run_container_operation', '    raise ContainerRuntimeError(', "conda_forge_feedstock_ops.container_utils.ContainerRuntimeError: Error running 'conda-forge-feedstock-ops-container rerender --log-level info --timeout 900' in container - error RuntimeError raised:", "'Failed to rerender.", 'output: Adding in variants from internal_defaults', 'Adding in variants from /tmp/tmpd4kqdxou/conda-smithy/conda_build_config.yaml', 'INFO:conda_smithy.configure_feedstock:Downloading conda-forge-pinning-2024.09.02.15.16.22', 'INFO:conda_smithy.configure_feedstock:Extracting conda-forge-pinning to /tmp/tmpd4kqdxou/conda-smithy', 'INFO:conda_smithy.configure_feedstock:__pycache__ rendering is skipped', 'INFO:conda_smithy.configure_feedstock:README rendering is skipped', 'INFO:conda_smithy.configure_feedstock:pypy38.yaml is closed now. Removing', "WARNING: Setting build platform. This is only useful when pretending to be on another platform, such as for rendering necessary dependencies on a non-native platform. I trust that you know what you're doing.", "WARNING: Setting build arch. This is only useful when pretending to be on another arch, such as for rendering necessary dependencies on a non-native arch. I trust that you know what you're doing.", 'WARNING: No numpy version specified in conda_build_config.yaml.  Falling back to default numpy value of 1.23', 'INFO:conda_smithy.configure_feedstock:Applying migrations: /tmp/tmpge8rwfr7/cython-feedstock/.ci_support/migrations/python312.yaml', 'Traceback (most recent call last):', '  File "/opt/conda/envs/cf-scripts/bin/conda-smithy", line 10, in <module>', '    sys.exit(main())', '             ^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 758, in main', '    args.subcommand_func(args)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 601, in __call__', '    self._call(args, args.temporary_directory)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/cli.py", line 604, in _call', '    configure_feedstock.main(', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 2785, in main', '    render_azure(env, config, forge_dir, return_metadata=True)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1871, in render_azure', '    return _render_ci_provider(', '           ^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 1090, in _render_ci_provider', '    metas = _conda_build_api_render_for_smithy(', '            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_smithy/configure_feedstock.py", line 890, in _conda_build_api_render_for_smithy', '    metadata_tuples = render_recipe(', '                      ^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/render.py", line 990, in render_recipe', '    variants = get_package_variants(m, variants=variants)', '               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 697, in get_package_variants', '    combined_spec, specs = get_package_combined_spec(', '                           ^^^^^^^^^^^^^^^^^^^^^^^^^^', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 661, in get_package_combined_spec', '    validate_spec(f, spec)', '  File "/opt/conda/envs/cf-scripts/lib/python3.11/site-packages/conda_build/variants.py", line 200, in validate_spec', '    raise ValueError(', 'ValueError: Variant configuration errors in argument_variants:', "  zip fields in zip_key group frozenset({'numpy', 'channel_sources', 'python', 'python_impl'}) are not all the same length", '', "'", '']}
  2024-09-03 01:58:31,119 DEBUG    conda_forge_tick.migrators.core || filter cython: archived or done
  2024-09-03 01:58:31,119 INFO     conda_forge_tick.auto_tick || skipping node cython w/ branch 0.29.x

@beckermr
Copy link
Contributor

beckermr commented Sep 3, 2024

The "archived or done" log message is incorrect. That message gets triggered by bad node attributes as well. I have fixed it in a PR. Let me check on the real issue.

@beckermr
Copy link
Contributor

beckermr commented Sep 3, 2024

The real issue was the bot marking nodes as bad for rerender errors when it should not have been. Better testing would have caught this, but is hard.

@h-vetinari
Copy link
Contributor Author

Thanks a lot for investigating and fixing this! 🙏

@beckermr
Copy link
Contributor

beckermr commented Sep 3, 2024

closing again since things are moving now. reopen if you see more issues!

@beckermr beckermr closed this as completed Sep 3, 2024
@h-vetinari
Copy link
Contributor Author

I'm reopening not because it's critical, more because I find it curious that some remnants of that loop still seem to be hanging around:

image

scipy should not depend on pynacl in any way

@h-vetinari h-vetinari reopened this Sep 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants