Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python311Packages.cvxopt: use LP64 blas on darwin; fix build #270016

Merged
merged 2 commits into from Dec 1, 2023
Merged

python311Packages.cvxopt: use LP64 blas on darwin; fix build #270016

merged 2 commits into from Dec 1, 2023

Conversation

ghost
Copy link

@ghost ghost commented Nov 26, 2023

darwin builds were using the ILP64 openblas library which causes the build to segufault in the unittests, which might be why the unit tests were disabled on darwin.

ILP64 is unsupported so it was a bug for darwin to use it in the first place.

this change aligns darwin to the linux build to use LP64 blas / lapack libraries. this also aligns with the assert at the top of the file which asserts blas and lapack are LP64. nixpkgs blas & lapack point to openblasCompat which is the openblas LP64, the opeblas package is ILP64. numpy and most (all) other math python packages require LP64.

the darwin unit test is enabled and now passes after this change as do the consumers of this package which were segfaulting in their unit tests.

Fixes: #271105
ZHF: #265948
https://hydra.nixos.org/build/241904734
note: the hydra failure is for python311Packages.cvxpy but that is just because it's unit tests run and were segfaulting in cvxopt

Result of nixpkgs-review pr 270016 run on x86_64-darwin 1

33 packages marked as broken and skipped:
  • labplot
  • python310Packages.pot
  • python310Packages.pot.dist
  • python310Packages.qiskit
  • python310Packages.qiskit-aer
  • python310Packages.qiskit-aer.dist
  • python310Packages.qiskit-finance
  • python310Packages.qiskit-finance.dist
  • python310Packages.qiskit-ibmq-provider
  • python310Packages.qiskit-ibmq-provider.dist
  • python310Packages.qiskit-ignis
  • python310Packages.qiskit-ignis.dist
  • python310Packages.qiskit-nature
  • python310Packages.qiskit-nature.dist
  • python310Packages.qiskit-optimization
  • python310Packages.qiskit-optimization.dist
  • python310Packages.qiskit.dist
  • python311Packages.pot
  • python311Packages.pot.dist
  • python311Packages.qiskit
  • python311Packages.qiskit-aer
  • python311Packages.qiskit-aer.dist
  • python311Packages.qiskit-finance
  • python311Packages.qiskit-finance.dist
  • python311Packages.qiskit-ibmq-provider
  • python311Packages.qiskit-ibmq-provider.dist
  • python311Packages.qiskit-ignis
  • python311Packages.qiskit-ignis.dist
  • python311Packages.qiskit-nature
  • python311Packages.qiskit-nature.dist
  • python311Packages.qiskit-optimization
  • python311Packages.qiskit-optimization.dist
  • python311Packages.qiskit.dist
28 packages built:
  • python310Packages.cvxopt
  • python310Packages.cvxopt.dist
  • python310Packages.cvxpy
  • python310Packages.cvxpy.dist
  • python310Packages.osqp
  • python310Packages.osqp.dist
  • python310Packages.picos
  • python310Packages.picos.dist
  • python310Packages.qpsolvers
  • python310Packages.qpsolvers.dist
  • python310Packages.qutip
  • python310Packages.qutip.dist
  • python310Packages.scikit-survival
  • python310Packages.scikit-survival.dist
  • python311Packages.cvxopt
  • python311Packages.cvxopt.dist
  • python311Packages.cvxpy
  • python311Packages.cvxpy.dist
  • python311Packages.osqp
  • python311Packages.osqp.dist
  • python311Packages.picos
  • python311Packages.picos.dist
  • python311Packages.qpsolvers
  • python311Packages.qpsolvers.dist
  • python311Packages.qutip
  • python311Packages.qutip.dist
  • python311Packages.scikit-survival
  • python311Packages.scikit-survival.dist

Result of nixpkgs-review pr 270016 run on x86_64-linux 1

28 packages marked as broken and skipped:
  • python310Packages.qiskit
  • python310Packages.qiskit-aer
  • python310Packages.qiskit-aer.dist
  • python310Packages.qiskit-finance
  • python310Packages.qiskit-finance.dist
  • python310Packages.qiskit-ibmq-provider
  • python310Packages.qiskit-ibmq-provider.dist
  • python310Packages.qiskit-ignis
  • python310Packages.qiskit-ignis.dist
  • python310Packages.qiskit-nature
  • python310Packages.qiskit-nature.dist
  • python310Packages.qiskit-optimization
  • python310Packages.qiskit-optimization.dist
  • python310Packages.qiskit.dist
  • python311Packages.qiskit
  • python311Packages.qiskit-aer
  • python311Packages.qiskit-aer.dist
  • python311Packages.qiskit-finance
  • python311Packages.qiskit-finance.dist
  • python311Packages.qiskit-ibmq-provider
  • python311Packages.qiskit-ibmq-provider.dist
  • python311Packages.qiskit-ignis
  • python311Packages.qiskit-ignis.dist
  • python311Packages.qiskit-nature
  • python311Packages.qiskit-nature.dist
  • python311Packages.qiskit-optimization
  • python311Packages.qiskit-optimization.dist
  • python311Packages.qiskit.dist
48 packages built:
  • cantor (libsForQt5.cantor ,plasma5Packages.cantor)
  • labplot
  • python310Packages.bambi
  • python310Packages.bambi.dist
  • python310Packages.blackjax
  • python310Packages.blackjax.dist
  • python310Packages.cvxopt
  • python310Packages.cvxopt.dist
  • python310Packages.cvxpy
  • python310Packages.cvxpy.dist
  • python310Packages.jaxopt
  • python310Packages.jaxopt.dist
  • python310Packages.osqp
  • python310Packages.osqp.dist
  • python310Packages.picos
  • python310Packages.picos.dist
  • python310Packages.pot
  • python310Packages.pot.dist
  • python310Packages.qpsolvers
  • python310Packages.qpsolvers.dist
  • python310Packages.qutip
  • python310Packages.qutip.dist
  • python310Packages.scikit-survival
  • python310Packages.scikit-survival.dist
  • python311Packages.bambi
  • python311Packages.bambi.dist
  • python311Packages.blackjax
  • python311Packages.blackjax.dist
  • python311Packages.cvxopt
  • python311Packages.cvxopt.dist
  • python311Packages.cvxpy
  • python311Packages.cvxpy.dist
  • python311Packages.jaxopt
  • python311Packages.jaxopt.dist
  • python311Packages.osqp
  • python311Packages.osqp.dist
  • python311Packages.picos
  • python311Packages.picos.dist
  • python311Packages.pot
  • python311Packages.pot.dist
  • python311Packages.qpsolvers
  • python311Packages.qpsolvers.dist
  • python311Packages.qutip
  • python311Packages.qutip.dist
  • python311Packages.scikit-survival
  • python311Packages.scikit-survival.dist
  • sage
  • sageWithDoc

Description of changes

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 23.11 Release Notes (or backporting 23.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Priorities

Add a 👍 reaction to pull requests you find important.

@ghost ghost added the 0.kind: ZHF Fixes Fixes during the Zero Hydra Failures (ZHF) campaign label Nov 26, 2023
@ghost ghost requested review from edwtjo and winterqt November 26, 2023 00:31
darwin builds were using the ILP64 openblas library which causes the
build to segufault in the unittests, which might be why the unit tests
were disabled on darwin...

this change aligns darwin to the linux build to use LP64 blas / lapack
libraries. this also aligns with the assert at the top of the file which
asserts blas and lapack are LP64.

the darwin unit test is re-enabled and passes.
@ofborg ofborg bot added 6.topic: darwin Running or building packages on Darwin 10.rebuild-darwin: 11-100 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Nov 26, 2023
@ofborg ofborg bot added 10.rebuild-linux: 11-100 and removed 10.rebuild-linux: 0 This PR does not cause any packages to rebuild on Linux labels Nov 27, 2023
@ghost ghost mentioned this pull request Nov 28, 2023
13 tasks
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-ready-for-review/3032/3003

Copy link
Member

@pbsds pbsds left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Diff LGTM, don't have a mac to test on

@ghost
Copy link
Author

ghost commented Dec 1, 2023

@emilytrau if you have a chance to look at this, i'm trying to get this package in as it silently (and not so silently) breaks a bunch of consumers on darwin. thx!

@emilytrau
Copy link
Member

Result of nixpkgs-review pr 270016 run on aarch64-darwin 1

32 packages marked as broken and skipped:
  • python310Packages.pot
  • python310Packages.pot.dist
  • python310Packages.qiskit
  • python310Packages.qiskit-aer
  • python310Packages.qiskit-aer.dist
  • python310Packages.qiskit-finance
  • python310Packages.qiskit-finance.dist
  • python310Packages.qiskit-ibmq-provider
  • python310Packages.qiskit-ibmq-provider.dist
  • python310Packages.qiskit-ignis
  • python310Packages.qiskit-ignis.dist
  • python310Packages.qiskit-nature
  • python310Packages.qiskit-nature.dist
  • python310Packages.qiskit-optimization
  • python310Packages.qiskit-optimization.dist
  • python310Packages.qiskit.dist
  • python311Packages.pot
  • python311Packages.pot.dist
  • python311Packages.qiskit
  • python311Packages.qiskit-aer
  • python311Packages.qiskit-aer.dist
  • python311Packages.qiskit-finance
  • python311Packages.qiskit-finance.dist
  • python311Packages.qiskit-ibmq-provider
  • python311Packages.qiskit-ibmq-provider.dist
  • python311Packages.qiskit-ignis
  • python311Packages.qiskit-ignis.dist
  • python311Packages.qiskit-nature
  • python311Packages.qiskit-nature.dist
  • python311Packages.qiskit-optimization
  • python311Packages.qiskit-optimization.dist
  • python311Packages.qiskit.dist
4 packages failed to build:
  • python310Packages.qutip
  • python310Packages.qutip.dist
  • python311Packages.qutip
  • python311Packages.qutip.dist
24 packages built:
  • python310Packages.cvxopt
  • python310Packages.cvxopt.dist
  • python310Packages.cvxpy
  • python310Packages.cvxpy.dist
  • python310Packages.osqp
  • python310Packages.osqp.dist
  • python310Packages.picos
  • python310Packages.picos.dist
  • python310Packages.qpsolvers
  • python310Packages.qpsolvers.dist
  • python310Packages.scikit-survival
  • python310Packages.scikit-survival.dist
  • python311Packages.cvxopt
  • python311Packages.cvxopt.dist
  • python311Packages.cvxpy
  • python311Packages.cvxpy.dist
  • python311Packages.osqp
  • python311Packages.osqp.dist
  • python311Packages.picos
  • python311Packages.picos.dist
  • python311Packages.qpsolvers
  • python311Packages.qpsolvers.dist
  • python311Packages.scikit-survival
  • python311Packages.scikit-survival.dist

Copy link
Member

@emilytrau emilytrau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!

@emilytrau emilytrau merged commit c27bf65 into NixOS:master Dec 1, 2023
28 checks passed
Copy link
Contributor

github-actions bot commented Dec 1, 2023

Successfully created backport PR for release-23.11:

@ghost
Copy link
Author

ghost commented Dec 1, 2023

woot! thanks!

@ghost ghost deleted the cvxopt-LP64-darwin branch December 1, 2023 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
0.kind: ZHF Fixes Fixes during the Zero Hydra Failures (ZHF) campaign 6.topic: darwin Running or building packages on Darwin 6.topic: python 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100
Projects
None yet
Development

Successfully merging this pull request may close these issues.

python3Packages.cvxopt crashes on darwin
3 participants