Meta-ticket: Port to Apple silicon

[mkoeppe]

... on top of homebrew, which provides gfortran 11 and many usable packages.

Tickets:

• Upgrade cmake to 3.19.3 #31258 Upgrade cmake to 3.19.3
• gfortran: Accept system gfortran 11 #31838 gfortran: Accept system gfortran 11
• Fix suitesparse / cvxopt path configuration for Apple Silicon, Alpine Linux #31905 cvxopt: Correct link directory
• update GMP to version 6.2.1 #32796 gmp: update to 6.2.1
• Update numpy to 1.21.4, networkx to 2.6.3 #32815 Update numpy to 1.21.3, networkx to 2.6.3
• tachyon, primecountpy, libhomfly, m4ri: Fix build errors on Apple Silicon with homebrew #33191 Fix build errors on Apple Silicon with homebrew
• iml: Refresh build scripts with current autotools #33195 iml: Refresh build scripts with current autotools
• openblas: Update to 0.3.20 #33194 openblas: Update to 0.3.19
• Update gcc/gfortran to 12.x + Apple Silicon support, update openblas to 0.3.21, gsl to 2.7.1 #33816 Update gcc/gfortran to 12.x + Apple Silicon support, update openblas to 0.3.21, gsl to 2.7.1

Discussions:

• https://groups.google.com/g/sage-devel/c/P-RGHeng5m8/m/vChq3b9yBwAJ
• https://groups.google.com/g/sage-devel/c/_vYkTlLRWWY
• https://groups.google.com/g/sage-devel/c/6LldEeJu3ZA

See also:

• Meta-ticket: Support Xcode 12 #30494: Meta-ticket: Support Xcode 12
• Meta-ticket: support macOS 11 Big Sur #30651: Meta-ticket: support macOS 11 Big Sur

CC: @dimpase @isuruf @kcrisman @kiwifb @williamstein

Component: porting

Reviewer: Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/30592

[slel]

comment:1

Does GitHub Actions provide that kind of machines for CI?

Somewhat related:

• LFortran issue 188: Investigate how to enable macOS for a MR also

[johnalanwoods]

comment:5

gFortran is now working natively on Apple Silicon (ARM64) GNU Fortran (Homebrew GCC 10.2.0_2) 10.2.1 20201220

as part of GCC.

gcc: stable 10.2.0 (bottled), HEAD GNU compiler collection https://gcc.gnu.org/

[mkoeppe]

comment:8

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.

[kcrisman]

comment:9

If I wanted to help test this, what could I do? I tried upgrading my Sage yesterday (finally!) but ran into problems with pari and some other packages; then I did make distclean and now it won't even install gmp, wonderful lines

[gmp-6.2.0] configure: error: Oops, mp_limb_t is 64 bits, but the assembler code
[gmp-6.2.0] in this configuration expects 32 bits.

that look scary. Anyway, I'd be quite interested in testing anything directly from source that does not use Homebrew, which after all the warnings many years ago about Macports and Fink conflicts I'm skeptical of.

[dimpase]

comment:10

Homebrew is the most popular "missing package manager" for macOS, and I really do not understand why you are sceptical about. Sage on x86_64 Homebrew mostly works, with many, many packages used rather than built from source. (Besides, do you have any idea how much effort went into this, supporting Sage using Homebrew packages? No, well, you many be "sceptical" about it, but it does work.)

Macports doesn't work with Sage, and Fink is mostly dead, IMHO

Homebrew surely does have ready Apple M1 GMP, cf.
https://formulae.brew.sh/formula/gmp#default

Maintaining the Sage status quo - porting patches from Hombrew back to Sage, etc, it's such a waste of time if you ask me.

[kcrisman]

comment:11

Homebrew is the most popular "missing package manager" for macOS, and I really do not understand why you are sceptical about. Sage on x86_64 Homebrew mostly works, with many, many packages used rather than built from source. (Besides, do you have any idea how much effort went into this, supporting Sage using Homebrew packages? No, well, you many be "sceptical" about it, but it does work.)

I guess bad experiences in the past with the other "managers" we will not name again - and, separately, liking to maintain ability for Mac users to download source and just type make. But I will give it a try.

---

However, as to the main topic of this ticket, is this simply "use Homebrew/Rosetta" or is there presumably some other testing I can do to help?

[williamstein]

comment:12

Replying to @dimpase:

(Besides, do you have any idea how much effort went into this, supporting Sage using Homebrew packages? No, well, you many be "sceptical" about it, but it does work.)

Just curious -- is there work underway to make sagemath itself a homebrew package, so one could do "brew install sagemath"? That would be pretty awesome.

[DaveWitteMorris]

comment:13

Making sage into a homebrew package is on the wishlist. (I don't think anyone is doing anything about it.) I think further discussion of this should go to #29395.

[dimpase]

comment:14

Replying to @williamstein:

Replying to @dimpase:

(Besides, do you have any idea how much effort went into this, supporting Sage using Homebrew packages? No, well, you many be "sceptical" about it, but it does work.)

Just curious -- is there work underway to make sagemath itself a homebrew package, so one could do "brew install sagemath"? That would be pretty awesome.

I played with creating Homebrew formulae a while ago, see https://github.com/sagemath/homebrew-science/commits/master

It's not that hard - and in fact e.g. arb and flint made it into mainline Homebrew since I touched this last time. It's more annoying that we sometimes need a variation of what's in Homebrew, e.g. it ships single-threaded Pari, but we need multi-threaded (as that's what giac requires).

[dimpase]

comment:15

Replying to @kcrisman:

However, as to the main topic of this ticket, is this simply "use Homebrew/Rosetta" or is there presumably some other testing I can do to help?

no, no Rosetta, just pure, unfiltered, Homebrew.
(Rosetta is an x86_64 emulator, as you might recall)

Give it a try please, I'd like to know how far we are there.

[dimpase]

comment:16

by the way, this ticket says "need gfortran first" (for M1), but it's already there.
https://formulae.brew.sh/formula/gcc (it's gcc/gfortran 11)

[kcrisman]

comment:18

(Rosetta is an x86_64 emulator, as you might recall)

Exactly; so homebrew will be using ARM stuff natively in this case?

Give it a try please, I'd like to know how far we are there.

Wish I hadn't destroyed my working install :( but I think I have a few old binaries I can use if under duress. That's why I waited for the semester to finish!

---

I'll post on devel if I achieve anything interesting. One note in prereqs: With respect to BLAS (mentioned in one of the threads) I get

==> Pouring openblas--0.3.15_1.big_sur.bottle.tar.gz
==> Caveats
openblas is keg-only, which means it was not symlinked into /usr/local,
because macOS provides BLAS in Accelerate.framework.

For compilers to find openblas you may need to set:
  export LDFLAGS="-L/usr/local/opt/openblas/lib"
  export CPPFLAGS="-I/usr/local/opt/openblas/include"

For pkg-config to find openblas you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/openblas/lib/pkgconfig"

==> Summary
🍺  /usr/local/Cellar/openblas/0.3.15_1: 23 files, 120MB

so hopefully accelerate will continue to work here for Sage as an option.

[dimpase]

comment:19

Replying to @kcrisman:

(Rosetta is an x86_64 emulator, as you might recall)

Exactly; so homebrew will be using ARM stuff natively in this case?

yes. They have a setup for this now (IIRC they use /opt/local for native M1, and /usr/local for Rosetta)

Give it a try please, I'd like to know how far we are there.

Wish I hadn't destroyed my working install :( but I think I have a few old binaries I can use if under duress. That's why I waited for the semester to finish!

---

I'll post on devel if I achieve anything interesting. One note in prereqs: With respect to BLAS (mentioned in one of the threads) I get

==> Pouring openblas--0.3.15_1.big_sur.bottle.tar.gz
==> Caveats
openblas is keg-only, which means it was not symlinked into /usr/local,
because macOS provides BLAS in Accelerate.framework.

For compilers to find openblas you may need to set:
  export LDFLAGS="-L/usr/local/opt/openblas/lib"
  export CPPFLAGS="-I/usr/local/opt/openblas/include"

For pkg-config to find openblas you may need to set:
  export PKG_CONFIG_PATH="/usr/local/opt/openblas/lib/pkgconfig"

==> Summary
🍺  /usr/local/Cellar/openblas/0.3.15_1: 23 files, 120MB

so hopefully accelerate will continue to work here for Sage as an option.

it's not an option if you want to use more external Homebrew packages, which depend on (open)blas.
Anyhow, sourcing .homebrew-build-env should take care of setting everything, including Homebrew's OpenBLAS,
to be used by Sage.

[kcrisman]

comment:21

We may also need to add openjpeg to the list here.

[dimpase]

comment:22

Replying to @kcrisman:

We may also need to add openjpeg to the list here.

I see no need for this. openjpeg is an optional dep of Pillow, and thus a problem may arise from a broken install of openjpeg - or no problem arises if openjpeg is not installed. But it's not Sage's job to do quality control of every package on a user box...

[culler]

comment:23

While we have had a bit of trouble getting complete information, it appears that the current version of the SageMath app (based on rc0) for macOS works fine in the Intel emulation mode on M1 macs. This was not always the case, but we are now taking care to avoid AVX, AVX2 and BMI instructions unless they are enabled at runtime by the "fat" version of gmp. This appears to eliminate the crashes that were seen before in emulation mode.

[mkoeppe]

comment:66

Attachment: openblas-0.3.19.log

This looks similar to https://trac.macports.org/ticket/61700; it sounds like we would have to configure openblas differently. We can probably get this from homebrew's build script. This won't make it into Sage 9.5.

Thanks again for testing!

[GMS103]

comment:67

And make ptestlong (after make with #33191 and #33195 but wihtout #33194) gives

----------------------------------------------------------------------
sage -t --long --random-seed=207327918669224047721672426685648861531 src/sage/functions/error.py  # 1 doctest failed
sage -t --long --random-seed=207327918669224047721672426685648861531 src/sage/matrix/matrix_gf2e_dense.pyx  # 1 doctest failed
sage -t --long --random-seed=207327918669224047721672426685648861531 src/sage/rings/complex_arb.pyx  # 2 doctests failed
sage -t --long --random-seed=207327918669224047721672426685648861531 src/sage/rings/real_arb.pyx  # 5 doctests failed
----------------------------------------------------------------------
Total time for all tests: 1294.6 seconds
    cpu time: 7872.1 seconds
    cumulative wall time: 9610.5 seconds
Features detected for doctesting: dvipng,ffmpeg,graphviz,imagemagick,pandoc,pdf2svg,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.plot,sage.rings.number_field,sage.rings.real_double,sage.symbolic,sagemath_doc_html,sphinx
Pytest is not installed, skip checking tests that rely on it.
make: *** [ptestlong] Error 1

[GMS103]

Attachment: config.3.log

[GMS103]

Attachment: install.3.log

Attachment: ptestlong.log

[GMS103]

comment:68

Just in case, things I think can be taken into account (besides #33191 and #33195):

1. Install everything ./configure (or ./configure --enable-download-from-upstream-url) recommends with brew install [...].

   • Be aware of the 2 versions of homebrew: the new arm64 should be used.
   • In particular, to install polymake do

   cd /usr/local
   sudo ln -s /opt/homebrew/lib lib
   brew install apaffenholz/polymake/polymake
   

   (see Apple M1: homebrew "brew install apaffenholz/polymake/polymake" fails (/usr/local instead of /opt/homebrew) apaffenholz/homebrew-polymake#3)

   • Do not forget source .homebrew-build-env afterwards.

2. Before make, do:

   eval "$(/opt/homebrew/bin/brew shellenv)"
   export PATH="/opt/homebrew/opt/openssl@3/bin:$PATH"
   export LDFLAGS="-L/opt/homebrew/opt/openssl@3/lib -L/opt/homebrew/opt/openblas/lib"
   export CPPFLAGS="-I/opt/homebrew/opt/openssl@3/include -I/opt/homebrew/opt/openblas/include"
   export PKG_CONFIG_PATH="/opt/homebrew/opt/openssl@3/lib/pkgconfig:/opt/homebrew/opt/openblas/lib/pkgconfig"
   

   (recommended by brew, but can be overlooked; perhaps to be put in .homebrew-build-env?)

3. export MAKE="make -j16" works fine on Apple M1 (all 8 cores at 100% most of the time).

[mkoeppe]

comment:69

The PKG_CONFIG_PATH addition for openssl is already taken care of by .homebrew-build-env. I don't think we need the changes to the other variables

[mkoeppe]

comment:70

Thanks for writing this summary - it will be helpful for those who find their way here from https://wiki.sagemath.org/ReleaseTours/sage-9.5#Sources

[mkoeppe]

comment:71

New report of build failures with 9.5.rc3 on M1 with macOS 12.1: https://groups.google.com/g/sage-release/c/1YV0dNtEZz4/m/2kQ-fKfaAQAJ

From the singular log:

libtool: compile:  g++ -std=gnu++11 -DHAVE_CONFIG_H -I. -I../.. -I../.. -I../.. -I../../libpolys -I../../libpolys -I/Users/admin/SageMath/Git/sage/local/var/tmp/sage/build/singular-4.2.1p3/src -I/Users/admin/SageMath/Git/sage/local/var/tmp/sage/build/singular-4.2.1p3/src/factory -I/Users/admin/SageMath/Git/sage/local/var/tmp/sage/build/singular-4.2.1p3/src/factory/include -I/Users/admin/SageMath/Git/sage/local/var/tmp/sage/build/singular-4.2.1p3/src -I/Users/admin/SageMath/Git/sage/local/var/tmp/sage/build/singular-4.2.1p3/src -I/Users/admin/SageMath/Git/sage/local/include -I/Users/admin/SageMath/Git/sage/local/include/ -I/Users/admin/SageMath/Git/sage/local/include -g -O2 -pipe -fno-common -g0 -O3 -Wno-unused-function -Wno-trigraphs -Wno-unused-parameter -Wunknown-pragmas -Wno-unused-variable -fomit-frame-pointer -fwrapv -fvisibility=default -finline-functions -fno-exceptions -fno-threadsafe-statics -funroll-loops -fno-rtti -Qunused-arguments -c mpr_inout.cc  -fno-common -DPIC -o .libs/mpr_inout.o
In file included from mpr_inout.cc:11:
In file included from ../../kernel/mod2.h:13:
../../libpolys/misc/auxiliary.h:70:2: error: "Unexpected SIZEOF_LONG"
#error "Unexpected SIZEOF_LONG"

(resolved by removing stuff in /usr/local)

[fchapoton]

comment:72

bump to 9.6

[jhpalmieri]

comment:76

Maybe we can close this.

[dimpase]

comment:77

how about #31905 ?

[jhpalmieri]

comment:78

Replying to Dima Pasechnik:

how about #31905 ?

I think we now agree that's not an issue. I don't know what to do with #30494, but maybe we don't need both this ticket and that one. So I propose closing this one and thinking about what to do with #30494 separately.

[dimpase]

Reviewer: Dima Pasechnik

[dimpase]

comment:80

close as done