Support minimal system Python version 3.6 (for venv)

[embray]

With #27824 it is possible to create a venv over an existing Python 3.7 instead of building the python3 spkg.

This is designed to work with Python 3.7 (and < 3.8) since that's what's currently used by sage-the-distribution and is best-tested.

However, it turns out an existing Python 3.6 can work just fine as the minimum Python version with a few small fixes (sage-the-distribution used Python 3.6 until last year, so the degree to which we diverged from supporting it was minimal).

This would be a nice-to-have, as it would speed up the build on various distributions that we still support:

• centos-8 (3.6.8),
• fedora-26 (3.6.5, but has 3.7.0 too), https://github.com/mkoeppe/sage/runs/743789829 - clean doctests, but exceeds 6h
• fedora-27 (3.6.6, but has 3.7.0 too), https://github.com/mkoeppe/sage/runs/743789883 - clean except for sympow, exceeds 6h
• fedora-28 (3.6.5, but has 3.7.3 too), https://github.com/mkoeppe/sage/runs/743789896 - with gcc_spkg, build errors for gmpy2, cython, pynac
• opensuse-leap-15.0 (3.6.5), -15.1 (3.6.9), -15.2 (3.6.10),
  but not (because python3 is too old):
• debian-stretch (oldstable) (3.5.3),
• ubuntu-xenial (3.5.2)
  and not (because a dependency of python3 is too old):
• centos-7 (3.6.8): sqlite is too old (https://github.com/mkoeppe/sage/runs/743789990)

(All this is according to https://repology.org/project/python/versions as of 2020-06-05, not verified.)

Maybe since Sage is less tested with 3.6, there could be a warning against using it.

See also: #27754 (Upgrade: Python 3.8.x)

CC: @jhpalmieri @dimpase @kiwifb @orlitzky

Component: python3

Author: Erik Bray, Matthias Koeppe

Branch: abb5607

Reviewer: Matthias Koeppe, Michael Orlitzky

Issue created by migration from https://trac.sagemath.org/ticket/29033

[mkoeppe]

comment:2

Related to the question of what Python versions are good is #26457 - "Do not depend on a patched Python"

[mkoeppe]

comment:3

Batch modifying tickets that will likely not be ready for 9.1, based on a review of the ticket title, branch/review status, and last modification date.

[mkoeppe]

Changed dependencies from #29032 to none

[mkoeppe]

comment:4

This would need rebasing

[mkoeppe]

comment:5

Edited description to clarify that this is about the venv python3, not sage-system-python.

[mkoeppe]

Changed branch from u/embray/build/configure/python3.6 to u/mkoeppe/build/configure/python3.6

[mkoeppe]

comment:9

Tests at https://github.com/mkoeppe/sage/pull/37/checks

---

New commits:

abb5607

Allow Sage to work with a system Python 3.6.

[mkoeppe]

Changed commit from b4a777e to abb5607

[mkoeppe]

comment:14

TODO: Check fedora-28 and centos-8 in https://github.com/mkoeppe/sage/actions/runs/126407840

[orlitzky]

comment:15

LGTM if those tests pass.

[mkoeppe]

comment:16

Tests now at https://github.com/mkoeppe/sage/actions/runs/127369767

[orlitzky]

Reviewer: Michael Orlitzky

[mkoeppe]

comment:18

Thank you!

[mkoeppe]

Changed author from Erik Bray to Erik Bray, Matthias Koeppe

[mkoeppe]

Changed reviewer from Michael Orlitzky to Matthias Koeppe, Michael Orlitzky

[vbraun]

Changed branch from u/mkoeppe/build/configure/python3.6 to abb5607

[antonio-rojas]

Changed commit from abb5607 to none

[antonio-rojas]

comment:21

C.UTF-8 locale is a Debian-specific thing (and maybe Fedora). Please don't assume that it is available. On Arch, this causes lots (as in, one every other line in the compilation output) of warnings

/bin/sh: warning: setlocale: LC_ALL: cannot change locale (C.UTF-8)

[dimpase]

comment:22

on another ticket, fixing docbuild on non-EN locales, someone using Arch complained that LANG=C causes a lot of warnings.

[orlitzky]

comment:23

Replying to @antonio-rojas:

C.UTF-8 locale is a Debian-specific thing (and maybe Fedora). Please don't assume that it is available.

Sorry about that, I didn't realize that Gentoo's glibc was patched (the patches are in an unusual place). It looks like discussion is still ongoing for glibc:

https://sourceware.org/pipermail/libc-alpha/2020-June/115224.html

Musl, on the other hand, already supports it I think?

https://wiki.musl-libc.org/functional-differences-from-glibc.html

In any case, we should probably revert to C in the meantime.

[dimpase]

comment:24

the locale issue is being dealt with on #30053

[dimpase]

comment:25

This has introduced an unfortunate bug related to a weird search order in AC_PATH_PROGS_FEATURE_CHECK(), which has the outer loop over PATH entries.
Thus, a python on the list [python.foo python.baz ....] of pythons that comes first in PATH and has all the features will be picked, not the first python on the list.

As modifying PATH to put the best python 1st might be error-prone (think about Conda, Homebrew, etc), this should be fixed.

This is of course an autoconf bug, too, as AC_PATH_PROGS() has a sane order, ie. its outer loop is over the list entries.

[dimpase]

comment:26

Replying to @dimpase:

This has introduced an unfortunate bug related to a weird search order in AC_PATH_PROGS_FEATURE_CHECK(), which has the outer loop over PATH entries.
Thus, a python on the list [python.foo python.baz ....] of pythons that comes first in PATH and has all the features will be picked, not the first python on the list.

As modifying PATH to put the best python 1st might be error-prone (think about Conda, Homebrew, etc), this should be fixed.

This is of course an autoconf bug, too, as AC_PATH_PROGS() has a sane order, ie. its outer loop is over the list entries.

see #30546 for a fix

[mkoeppe]

Follow up regarding centos-7:

(because a dependency of python3 is too old):

• centos-7 (3.6.8): sqlite is too old (https://github.com/mkoeppe/sage/runs/743789990)

After #30559, sqlite is no longer the problem, but python3 is still not accepted on this system because of zlib -> libpng:

Checking whether SageMath should install SPKG zlib...
checking for inflateEnd in -lz... yes
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking for LIBPNG... no
checking for inflateValidate in -lz... no
configure: no suitable system package found for SPKG zlib
-----------------------------------------------------------------------------
Checking whether SageMath should install SPKG libpng...
checking installing zlib? ... yes; install libpng as well
configure: no suitable system package found for SPKG libpng

[mkoeppe]

comment:28

#31009 is taking care of this.