Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update gcc/gfortran to 12.x + Apple Silicon support, update openblas to 0.3.21, gsl to 2.7.1 #33816

Closed
mkoeppe opened this issue May 6, 2022 · 83 comments
Closed

Update gcc/gfortran to 12.x + Apple Silicon support, update openblas to 0.3.21, gsl to 2.7.1 #33816

mkoeppe opened this issue May 6, 2022 · 83 comments
Labels
c: packages: standard p: major / 3 t: enhancement
Milestone
sage-9.8

Comments

@mkoeppe
Copy link
Contributor

mkoeppe commented May 6, 2022

We would use 12.2.0 plus the feature patch for Apple Silicon support that the homebrew folks use:
https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/gcc.rb#L12

Depends on #34266
Depends on #34228

CC: @culler @jhpalmieri @kliem

Component: packages: standard

Author: Matthias Koeppe

Branch/Commit: e89ba55

Reviewer: John Palmieri

Issue created by migration from https://trac.sagemath.org/ticket/33816
@mkoeppe mkoeppe added this to the sage-9.7 milestone May 6, 2022
@mkoeppe
Copy link
Contributor Author

mkoeppe commented May 20, 2022

comment:1

Looks like homebrew is able to use upstream gcc 12 on M1 - https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/[email protected]

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Jun 30, 2022

comment:2

... with one big patch: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/[email protected]#L43

@mkoeppe mkoeppe modified the milestones: sage-9.7, sage-9.8 Aug 31, 2022
@mkoeppe

This comment has been minimized.

Sign in to view
@mkoeppe mkoeppe changed the title Update gcc/gfortran to 12.1 Update gcc/gfortran to 12.x + Apple Silicon support Sep 20, 2022
@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 20, 2022

Branch: u/mkoeppe/update_gcc_gfortran_to_12_x___apple_silicon_support

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 20, 2022

Commit: 9ae559b

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 20, 2022

New commits:

1dfeb14build/pkgs/gcc: Update to 12.2.0
9ae559bbuild/pkgs/gcc/patches/gcc-12.2.0-arm.patch: Import from https://raw.githubusercontent.com/Homebrew/formula-patches/1d184289/gcc/gcc-12.2.0-arm.diff

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 20, 2022

Author: Matthias Koeppe

@jhpalmieri
Copy link
Member

comment:7

I don't know if this is ready yet but I tried it anyway on my new M2 laptop. It failed; the end of the log file reads

0  0x1010cc1a0  __assert_rtn + 140
1  0x100f53a8c  mach_o::relocatable::Parser<arm64>::parse(mach_o::relocatable::ParserOptions const&) + 4536
2  0x100f25d38  mach_o::relocatable::Parser<arm64>::parse(unsigned char const*, unsigned long long, char const*, long, ld::File::Ordinal, mach_o::relocatable::ParserOptions const&) + 148
3  0x100f8e4ac  ld::tool::InputFiles::makeFile(Options::FileInfo const&, bool) + 1468
4  0x100f91360  ___ZN2ld4tool10InputFilesC2ER7Options_block_invoke + 56
5  0x188a041f4  _dispatch_client_callout2 + 20
6  0x188a17954  _dispatch_apply_invoke + 224
7  0x188a041b4  _dispatch_client_callout + 20
8  0x188a15a04  _dispatch_root_queue_drain + 680
9  0x188a16104  _dispatch_worker_thread2 + 164
10  0x188bc4324  _pthread_wqthread + 228
A linker snapshot was created at:
	/tmp/libstdc++.6.dylib-2022-09-20-133305.ld-snapshot
ld: Assertion failed: (_file->_atomsArrayCount == computedAtomCount && "more atoms allocated than expected"), function parse, file macho_relocatable_file.cpp, line 2061.

I can post the full log file if you want.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 20, 2022

comment:8

Thanks for testing. Also on Intel I ran into a build error. This will need more work.

@jhpalmieri
Copy link
Member

comment:9

I'm happy to test. It's just a MacBook Air, but it builds Sage faster than my iMac Pro, and I like giving it tasks.

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 21, 2022

Branch pushed to git repo; I updated commit sha1. New commits:

49c407bbuild/pkgs/gcc/spkg-build.in: Remove workarounds for ancient OS X

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 21, 2022

Changed commit from 9ae559b to 49c407b

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 21, 2022

Branch pushed to git repo; I updated commit sha1. New commits:

ba6de83build/pkgs/gcc/build-gcc: Unset AS, LD

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 21, 2022

Changed commit from 49c407b to ba6de83

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 21, 2022

comment:12

On Intel, this still fails tox -e local-macos-nohomebrew -- gfortran, but it passes with the full bootstrapped build tox -e local-macos-nohomebrew -- gcc.

The failure when trying to build gfortran using Apple's gcc (clang) is introduced by the patch - passing the argument -nodefaultrpaths to the compiler.

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 21, 2022

Changed commit from ba6de83 to 5d9563b

@sagetrac-git
Copy link
Mannequin

sagetrac-git mannequin commented Sep 21, 2022

Branch pushed to git repo; I updated commit sha1. New commits:

5d9563bbuild/pkgs/gfortran/spkg-build.in: Work around build failure with Apple gcc (clang)

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 21, 2022

comment:14

This builds for me now on Intel. Haven't tested anything else yet

@jhpalmieri
Copy link
Member

comment:15

Still fails for me on m2.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 21, 2022

comment:16

full log please

@jhpalmieri
Copy link
Member

comment:17

Hold on, I tried make gcc. I'm going to try make gfortran and see how that goes.

@jhpalmieri
Copy link
Member

comment:18

gfortran built successfully.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 21, 2022

comment:19

Great! I think we can ignore the failure to build gcc. We haven't been testing those in quite a while

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 22, 2022

comment:55

Replying to Matthias Köppe:

Unfortunately, the 2nd build stage ("minimal") [...] has not started to run yet (I think because of general GH Actions flakiness).

Actually, a typo was the cause. Fixed now

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 22, 2022

Changed dependencies from #34266 to #34266, #34228

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 22, 2022

comment:57

2nd stage running now at https://github.com/mkoeppe/sage/actions/runs/3108109396/jobs/5036946542

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 22, 2022

Changed reviewer from https://github.com/mkoeppe/sage/actions/runs/3094982876 to https://github.com/mkoeppe/sage/actions/runs/3094982876, https://github.com/mkoeppe/sage/actions/runs/3108109396/jobs/5036946542

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 22, 2022

comment:58

Replying to Matthias Köppe:

The relevant tests among the many at https://github.com/mkoeppe/sage/actions/runs/3094982876 are the ones named "minimal-pre", which builds all non-Python packages. They have succeeded for all supported platforms.

(although 4 timed out)

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 22, 2022

comment:59

There are some failures, I'll have to investigate what's going on there.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 23, 2022

comment:60

Looks like the failures are only due to a flaw in my new 2-stage workflow, related to the use of -arch=native. For example, looking into the config.log of the failing fflas_ffpack package on fedora-35-minimal https://github.com/mkoeppe/sage/actions/runs/3108109396/jobs/5036948018, one sees:

configure:18251: checking for use of MKL
configure:18268: result: no 
configure:18288: checking for BLAS
configure:18304: g++ -std=gnu++11 -o conftest -g -O2   -I. -I.. -I. -I./fflas-ffpack -fabi-version=6   -Wl,-rpath-link,/sage/local/lib -L/sage/local/lib -Wl,-rpath,/sage/local/lib -Wl,-rpath-link,/sage/local/lib -L/sage/local/lib -Wl,-rpath,/sage/local/lib  conftest.cpp  -lopenblas  >&5
configure:18304: $? = 0
configure:18304: ./conftest
./configure: line 1735: 100961 Illegal instruction     (core dumped) ./conftest$ac_exeext

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 23, 2022

comment:61

I've opened #34572 to improve this.

@jhpalmieri
Copy link
Member

comment:62

Do we need to test on OS X pre-Monterey? Otherwise, I'm ready to give this a positive review, at least from the OS X point of view.

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 23, 2022

comment:63

A test on Catalina (10.15) ran at https://github.com/mkoeppe/sage/actions/runs/3094982845/jobs/5008934048, looks OK

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 23, 2022

comment:64

I've manually tested tox -e docker-fedora-35-minimal (also ran make ptest), also fine.

I've also tested EXTRA_CONFIGURE_ARGS=--without-system-gfortran tox -e docker-ubuntu-trusty-toolchain-gcc_9-minimal, a bit of an usual configuration, which has a failure in src/sage/misc/inline_fortran.py, but I don't think this is important enough to investigate more

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 23, 2022

comment:65

So I think this is ready to go

@jhpalmieri
Copy link
Member

comment:66

Okay, let's merge it early in this release cycle, not that too many people are building our gfortran anyway...

@jhpalmieri
Copy link
Member

Changed reviewer from https://github.com/mkoeppe/sage/actions/runs/3094982876, https://github.com/mkoeppe/sage/actions/runs/3108109396/jobs/5036946542 to John Palmieri

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 23, 2022

comment:67

Thanks!

@vbraun
Copy link
Member

vbraun commented Sep 26, 2022

comment:68

Full run: http://build.sagemath.org/#/builders/15/builds/5

Flint testsuite dies on the kucalc buildbot (Ubuntu 16.04.5 LTS)

gcc -g -O2 -I/var/lib/buildbot/slave/sage_git/build/local/var/tmp/sage/build/flint-2.8.4/src -I/var/lib/buildbot/slave/sage_git/build/local/var/tmp/sage/build/flint-2.8.4/src/build -I/var/lib/buildbot/slave/sage_git/build/local/include -I/var/lib/buildbot/slave/sage_git/build/local/include -I/var/lib/buildbot/slave/sage_git/build/local/include test_helpers.o test/t-sdiv_qrnnd.c -o build/test/t-sdiv_qrnnd -L/var/lib/buildbot/slave/sage_git/build/local/var/tmp/sage/build/flint-2.8.4/src -L/var/lib/buildbot/slave/sage_git/build/local/lib -L/var/lib/buildbot/slave/sage_git/build/local/lib -L/var/lib/buildbot/slave/sage_git/build/local/lib -lflint -lmpfr -lgmp -lm -lntl -lpthread  -Wl,-rpath-link,/var/lib/buildbot/slave/sage_git/build/local/lib -L/var/lib/buildbot/slave/sage_git/build/local/lib -Wl,-rpath,/var/lib/buildbot/slave/sage_git/build/local/lib -Wl,-rpath-link,/var/lib/buildbot/slave/sage_git/build/local/lib -L/var/lib/buildbot/slave/sage_git/build/local/lib -Wl,-rpath,/var/lib/buildbot/slave/sage_git/build/local/lib  -Wl,-rpath,/var/lib/buildbot/slave/sage_git/build/local/lib -Wl,-rpath,/var/lib/buildbot/slave/sage_git/build/local/lib -Wl,-rpath,/var/lib/buildbot/slave/sage_git/build/local/var/tmp/sage/build/flint-2.8.4/src
(WANTDEPS=1; export WANTDEPS; BUILD_DIR=build; export BUILD_DIR; make -f Makefile.subdirs -C . check || exit $?;)
sub_dddmmmsss....invert_limb....udiv_qrnnd....add_ssssaaaaaaaa....byte_swap....umul_ppmm....add_ssaaaa....add_sssaaaaaa....udiv_qrnnd_preinv....PASS
smul_ppmm....count_leading_zeros....count_trailing_zeros....PASS
sub_ddmmss....PASS
sdiv_qrnnd....PASS
PASS
PASS
Makefile.subdirs:107: recipe for target 'build/test/t-sdiv_qrnnd_RUN' failed
make[6]: *** [build/test/t-sdiv_qrnnd_RUN] Floating point exception (core dumped)
PASS
PASS
PASS
PASS
PASS
PASS
PASS
make[6]: Target 'check' not remade because of errors.
Makefile:219: recipe for target 'check' failed
make[5]: *** [check] Error 2

@jhpalmieri
Copy link
Member

comment:69

Our web page https://trac.sagemath.org/wiki/ReleaseTours/sage-9.8#Configurationchanges says "Users of older Linux distributions (in particular, ubuntu-xenial or older, debian-stretch or older, linuxmint-18 or older) should upgrade their systems before attempting to install Sage from source." So does a test failure on Ubuntu 16.04 matter?

@mkoeppe
Copy link
Contributor Author

mkoeppe commented Sep 27, 2022

comment:70

Yes, I think this buildbot should be updated to a supported configuration, for example using https://askubuntu.com/questions/1140183/install-gcc-9-on-ubuntu-18-04/1149383#1149383

@jhpalmieri
Copy link
Member

comment:71

So do we just set this back to positive, or does someone have to do something with the buildbot first?

@vbraun
Copy link
Member

vbraun commented Oct 17, 2022

Changed branch from u/mkoeppe/update_gcc_gfortran_to_12_x___apple_silicon_support to e89ba55

@vbraun vbraun closed this as completed in 25d8bdc Oct 17, 2022
vbraun pushed a commit to vbraun/sage that referenced this issue Dec 10, 2023
sagemathgh-36828: README.md: Update section on macOS arm64
88ffb72 
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

Since Sage 9.8, it is no longer required to use homebrew when building
on M1/M2 because of:
- sagemath#33816

Here we correct the README.

- [x] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36828
Reported by: Matthias Köppe
Reviewer(s): John H. Palmieri
vbraun pushed a commit to vbraun/sage that referenced this issue Dec 13, 2023
sagemathgh-36828: README.md: Update section on macOS arm64
31541b6 
    
<!-- ^^^^^
Please provide a concise, informative and self-explanatory title.
Don't put issue numbers in there, do this in the PR body below.
For example, instead of "Fixes sagemath#1234" use "Introduce new method to
calculate 1+1"
-->
<!-- Describe your changes here in detail -->

<!-- Why is this change required? What problem does it solve? -->
<!-- If this PR resolves an open issue, please link to it here. For
example "Fixes sagemath#12345". -->
<!-- If your change requires a documentation PR, please link it
appropriately. -->

### 📝 Checklist

<!-- Put an `x` in all the boxes that apply. -->
<!-- If your change requires a documentation PR, please link it
appropriately -->
<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
<!-- Feel free to remove irrelevant items. -->

Since Sage 9.8, it is no longer required to use homebrew when building
on M1/M2 because of:
- sagemath#33816

Here we correct the README.

- [x] The title is concise, informative, and self-explanatory.
- [ ] The description explains in detail what this PR is about.
- [ ] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.

### ⌛ Dependencies

<!-- List all open PRs that this PR logically depends on
- sagemath#12345: short description why this is a dependency
- sagemath#34567: ...
-->

<!-- If you're unsure about any of these, don't hesitate to ask. We're
here to help! -->
    
URL: sagemath#36828
Reported by: Matthias Köppe
Reviewer(s): John H. Palmieri
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
c: packages: standard p: major / 3 t: enhancement
Projects
None yet
Milestone
sage-9.8
Development

No branches or pull requests
3 participants
@vbraun @jhpalmieri @mkoeppe