Updates to linbox-1.7.0, givaro-4.2.0, fflas-ffpack-2.5.0

[tornaria]

📚 Description

Draft PR taken from #32959. I rebased on top of 9.8 and squashed together so there's one commit per update, plus the two commits that adapt sage to changes in linbox api. I left out sagemath/sagetrac-mirror@4006e10 since I understand it's not needed with singular 4.3+.

I'm testing on void, but note I needed to include a few patches for this to work on i686:

• givaro: PR Fix dom_power issue on 32bit machine linbox-team/givaro#197
• fflas-ffack: PR Fix Simd on i686: SimdChooser, #ifdef guard linbox-team/fflas-ffpack#357
• linbox: PR Only register uint128_t as a TypeName when it's available. linbox-team/linbox#292 and Only register uint128_t as a TypeName when it's available linbox-team/linbox#294

I'm making this draft PR to get the ball rolling...

@ClementPernet

Resolves #32959

📝 Checklist

• I have made sure that the title is self-explanatory and the description concisely explains the PR.
• I have linked an issue or discussion.

⌛ Dependencies

• Support linbox 1.7.0 and 1.6.3 at the same time #35612 for supporting linbox 1.7.0. This PR builds on top of that one and actually updates linbox et al in sagemath.
• build/pkgs/{givaro,fflas_ffpack,linbox}: Accept matching set of new versions #36997

[mkoeppe]

I left out sagemath/sagetrac-mirror@4006e10 since I understand it's not needed with singular 4.3+.

In this case, we'll have to tighten the Singular requirements in build/pkgs/singular/spkg-configure.m4

[tornaria]

The problem here is that configure decided to use givaro and fflas-ffpack from system, and rebuilding linbox to 1.7.0 failed. I guess this is related to #34359. I don't know what is the way forward.

Also, there seems to be other issues:
a. looking at the patch for sage, it seems the API for linbox has changed: I don't know if it is straightforward to support 1.6.3 at the same time as 1.7.0 (I didn't try).
b. the ABI has also changed but the soname didn't, this is a problem, e.g. I get a lot of crashes when running sage built with the old versions after upgrading linbox et al.

[mkoeppe]

What versions of packages do we want to accept from the system? #34359 is relevant

[tornaria]

What versions of packages do we want to accept from the system? #34359 is relevant

Even debian (testing) has already moved to linbox 1.7.0. Maybe we can just require givaro 4.2.0, fflas 2.5.0, and linbox 1.7.0?

The important bit is that linbox 1.7.0 cannot be built with older givaro & fflas so either you force everything to the new versions or you have to add logic in givaro and fflas to accept old versions only if system linbox is available.

[mkoeppe]

Maybe we can just require givaro 4.2.0, fflas 2.5.0, and linbox 1.7.0?

Sounds reasonable.

[tornaria]

I don't see an easy way to support both 1.6.3 and 1.7.0. Three lines have to be changed:

$ git diff
diff --git a/src/sage/libs/linbox/linbox.pxd b/src/sage/libs/linbox/linbox.pxd
index dcc482960cc..6db69735abc 100644
--- a/src/sage/libs/linbox/linbox.pxd
+++ b/src/sage/libs/linbox/linbox.pxd
@@ -32,7 +32,7 @@ cdef extern from "linbox/matrix/dense-matrix.h":
         ctypedef Modular_double Field
         ctypedef double Element
         DenseMatrix_Modular_double(Field F, size_t m, size_t n)
-        DenseMatrix_Modular_double(Field F, size_t m, size_t n, Element*)
+        DenseMatrix_Modular_double(Field F, Element*, size_t m, size_t n)
         void setEntry(size_t i, size_t j, Element& a)
         Element &getEntry(size_t i, size_t j)
 
@@ -42,7 +42,7 @@ cdef extern from "linbox/matrix/dense-matrix.h":
         ctypedef Modular_float Field
         ctypedef float Element
         DenseMatrix_Modular_float(Field F, size_t m, size_t n)
-        DenseMatrix_Modular_float(Field F, size_t m, size_t n, Element*)
+        DenseMatrix_Modular_float(Field F, Element*, size_t m, size_t n)
         void setEntry(size_t i, size_t j, Element& a)
         Element &getEntry(size_t i, size_t j)
 
diff --git a/src/sage/matrix/matrix_modn_dense_template.pxi b/src/sage/matrix/matrix_modn_dense_template.pxi
index e45e05f3641..abf29badce6 100644
--- a/src/sage/matrix/matrix_modn_dense_template.pxi
+++ b/src/sage/matrix/matrix_modn_dense_template.pxi
@@ -221,7 +221,7 @@ cdef inline linbox_echelonize_efd(celement modulus, celement* entries, Py_ssize_
         return 0,[]
 
     cdef ModField *F = new ModField(<long>modulus)
-    cdef DenseMatrix *A = new DenseMatrix(F[0], <Py_ssize_t>nrows, <Py_ssize_t>ncols, <ModField.Element*>entries)
+    cdef DenseMatrix *A = new DenseMatrix(F[0], <ModField.Element*>entries,<Py_ssize_t>nrows, <Py_ssize_t>ncols)
     cdef Py_ssize_t r = reducedRowEchelonize(A[0])
     cdef Py_ssize_t i,j
     for i in range(nrows):

The - lines support 1.7.0, the - lines support 1.6.3. I don't know how conditional compile one or the other. Better yet woudl be to have cython "change" the calls to the constructor when 1.6.3 is in use.

Of course, the easiest way is to just migrate to 1.7.0 altogether.

[tornaria]

In #35612 I removed the use of the changed api, that was only in a single place and easy to replace. Now sagelib uses only part of the API that is unchanged between 1.6.3 and 1.7.0.

I rebased this PR on top of #35612. If possible, that one should be merged quickly for 10.0, and this one can be worked out together with #34359.

I would say sagemath should update linbox et al soon after 10.0, and in terms of system support: accept any version of linbox starting with 1.6.3. As for givaro and fflas-ffpack, accept them from system only if either linbox is installed, or if they are at least 4.2.0 and 2.5.0. It should not be allowed to use older givaro and/or fflas-ffpack when linbox will be built by sage.

[tornaria]

Rebased to 10.0.rc2 without any change (removes spurious "fix the linter commits")

[tornaria]

I've just rebased this PR to 10.2.rc0. We've been shipping these updates for a while in void linux without any issue that I know of. There are some patches we use that I added here (IIRC they are all from upstream), and this is regularly tested on x86_64 (glibc and musl) and on i686 (glibc only). I think it's been tested on aarch64 as well.

I'm no longer interested in this PR since I'm unable to test. If someone else wants to take over, feel free...

[tornaria]

Rebased to 10.2.rc1

[mkoeppe]

On debian-bullseye-i386-standard (https://github.com/sagemath/sage/actions/runs/6836592787/job/18591708478?pr=35148#step:14:397):

 ../../linbox/algorithms/polynomial-matrix/fft-integral.inl: In member function 'void LinBox::FFT_base<Field, Simd, typename std::enable_if<(Field::is_elt_integral_v && typename Simd::is_same_element<Field>::value)>::type>::DIF_core_laststeps(LinBox::FFT_base<Field, Simd, typename std::enable_if<(Field::is_elt_integral_v && typename Simd::is_same_element<Field>::value)>::type>::Element*, size_t, size_t, const Element*, const Element*, const simd_vect_t&, const simd_vect_t&, LinBox::FFTSimdHelper<4>) const':
  ../../linbox/algorithms/polynomial-matrix/fft-integral.inl:703:49: error: incomplete type 'Simd128<long long unsigned int>' {aka 'Simd128_impl<true, true, false, 8>'} used in nested name specifier
    703 |                         V7 = Simd128<uint64_t>::mulx (V4, Wp);
        |                                                 ^~~~

[tornaria]

On debian-bullseye-i386-standard (https://github.com/sagemath/sage/actions/runs/6836592787/job/18591708478?pr=35148#step:14:397):

 ../../linbox/algorithms/polynomial-matrix/fft-integral.inl: In member function 'void LinBox::FFT_base<Field, Simd, typename std::enable_if<(Field::is_elt_integral_v && typename Simd::is_same_element<Field>::value)>::type>::DIF_core_laststeps(LinBox::FFT_base<Field, Simd, typename std::enable_if<(Field::is_elt_integral_v && typename Simd::is_same_element<Field>::value)>::type>::Element*, size_t, size_t, const Element*, const Element*, const simd_vect_t&, const simd_vect_t&, LinBox::FFTSimdHelper<4>) const':
  ../../linbox/algorithms/polynomial-matrix/fft-integral.inl:703:49: error: incomplete type 'Simd128<long long unsigned int>' {aka 'Simd128_impl<true, true, false, 8>'} used in nested name specifier
    703 |                         V7 = Simd128<uint64_t>::mulx (V4, Wp);
        |                                                 ^~~~

No idea, I'm building on i686 without trouble (needs 292.patch and 294.patch which I added here). May be because it's being built without sse2. Do we really need to support pentium 3?

Does it work now?

[tornaria]

It seems all tests passed here (I only looked in debian-bullseye-i386-standard, but the failure seems to be something else).

[mkoeppe]

Same "incomplete type" errors are still present

[tornaria]

Same "incomplete type" errors are still present

I give up. How should I know this is a 32 bit problem if ALL builders fail, etc.

There's something very weird going on here. The lines that use Simd128<uint64_t> are all guarded by

#ifdef __FFLASFFPACK_HAVE_SSE4_1_INSTRUCTIONS

so it makes sense that they shouldn't be compiled, but for some reason are.

[tornaria]

Also in fact

configure: WARNING: unrecognized options: --disable-maintainer-mode, --disable-openmp, --disable-sse3, --disable-ssse3, --disable-sse41, --disable-sse42, --disable-fma, --disable-fma4, --disable-avx, --disable-avx2

so all these flags (even the two that I removed) are actually unused.

I'm looking back at my templates and I see I'm using --without-archnative in configure flags because otherwise they will use -march=native to build.

[tornaria]

Summary: removing all these unused flags and adding --without-archnative to configure (to the three packages: givaro, fflas_ffpack, linbox) might work.

But I don't have time to follow up on this, so I'll just orphan this PR (is there such a thing?)

[mkoeppe]

How should I know this is a 32 bit problem if ALL builders fail

By looking at the Summary page of the "CI Linux incremental" workflow - https://github.com/sagemath/sage/actions/runs/6841299882?pr=35148
and searching for "linbox".

[github-actions]

Documentation preview for this PR (built with commit 66e9d15; changes) is ready! 🎉

[mkoeppe]

i'll try with https://sources.debian.org/patches/linbox/1.7.0-3/

[mkoeppe]

It looks like more work with upstream is needed.

• ci-sage: Update linbox-team/fflas-ffpack#386

(that's a very good, on-topic PR number;)

[vbraun]

Also needs this patch linbox-team/givaro#226 (comment) to build givaro on Fedora 40

[vbraun]

And part of the commit referenced in linbox-team/linbox#310 ...

[vbraun]

I've rebased the branch at https://github.com/vbraun/sage/tree/linbox-flas-givaro-4.2.0 and added the two additional upstream patches, now at least the linbox-flas-givaro builds on gcc-14

[vbraun]

I've created a new PR at #37938