Drop support for Python 3.7

[nsoranzo]

Describe the bug
Python 3.7 reached end of life on 2023-06-27. A number of our dependencies have already dropped support for 3.7 (e.g. numpy since April 2022; black) and more will do in the upcoming months.
With Python 3.12 coming out in October, supporting 5 major Python releases in Galaxy 23.2 seems enough to me, and this will also allow some code cleanup.

xref. https://devguide.python.org/versions/

[jmchilton]

We don't use any of those numpy features I assume.

Is there room to compromise on this - dropping support in the app but maintaining it in the library code - at least below the webapps level?

[nsoranzo]

It's not about any specific dependency feature.

When the ecosystem starts moving, you need to pin package versions (e.g. #16390 ) or introduce hacks (e.g. commit 4ca163b ). And Poetry will find even more difficult to resolve our over 120 dependencies.

[jmchilton]

For a decade we valued keeping things stable for deployers over improving the experience of developers - this is unfortunate and a misalignment of priorities IMO. If there isn't going to be compromise on this for the library packages than I'm against dropping Python 3.7 in the main app.

[mvdbeek]

Deployers are not supposed to run python projects beyond their EOL, it's just that python 2s EOL was extremely (and I think unplanned-ly) long. Also the ecosystem has made giant leaps in terms of how easy it is to update the python version (conda, brew, source are all valid options even on the most outdated of systems). If you use our playbooks -> that's managed. If you use run.sh we use conda to install a compatible python version. If you use docker/kubernetes ... you don't care. If we care about the ease of install in such an extraordinary way (which I think supporting beyond EOL is) we (or the library consumers that create apps with similar long-term policies) should start bundling the python interpreter.

[natefoo]

3.6 was EOL’d and dropped before folks were even switching to EL8, where it was the default python. I think this is unfortunately just the way of things. People who need an easy experience will be using Ubuntu, which provides a new enough python. People doing fancy stuff on EL know how to get a newer one. I’m ok with this.

[jmchilton]

The lack of compromise here is obviously disheartening.

[mvdbeek]

I'm sad to hear this. How would you feel about dropping 3.7 in 24.0 as a compromise ? I do think that supporting EOL python is going to be an increasingly difficult effort, and split support level across packages (do you have util and tool-util in mind, or more ?) would also require non-trivial investment in configuring our tooling.

[nsoranzo]

Our linters/checkers are dropping support for Python 3.7:

• flake8 >=6.0.0 (using flake8 5.x means we need workarounds like https://github.com/galaxyproject/galaxy/pull/16639/files#diff-889aaf1157662e23f5e73182feadd8364b2da64f576a82b7efb6378f833a7bba )
• mypy >=1.5, see https://mypy-lang.blogspot.com/2023/08/mypy-15-released.html

[nsoranzo]

Discussed this at the 2023-09-05 Backend WG meeting, the plan is to add a new CI workflow which is a subset of test_galaxy_packages.yaml running unit tests for the Galaxy packages needed by pulsar (currently galaxy-job-metrics, galaxy-objectstore, galaxy-tool-util and galaxy-util) under Python 3.7.

[nsoranzo]

Added deprecation notice of Python 3.7, with plan to drop it in v24.0, in commit 6d35812.