Drop Python 3.6

[DiddiLeija]

Closes #525.

Any suggestions to remove 3.6-only code?

[henryiii]

I'd rather hope my recent changes going in (like the nox file listing) would happen before nox drops 3.6. GHA/Azure's ubuntu-18.04 still runs Python 3.6 if you use pipx, CentOS 7 & 8 both have python3 -> 3.6, etc. So don't see a huge need to push this out immediately.

I'd use pyupgrade to remove 3.6 code, see https://scikit-hep.org/developer/style. I'd also inject from __future__ import annotations using isort then run it, which will clean up annotations.

[DiddiLeija]

I'd rather hope my recent changes going in (like the nox file listing) would happen before nox drops 3.6. GHA/Azure's ubuntu-18.04 still runs Python 3.6 if you use pipx, CentOS 7 & 8 both have python3 -> 3.6, etc. So don't see a huge need to push this out immediately.

I'd use pyupgrade to remove 3.6 code, see https://scikit-hep.org/developer/style. I'd also inject from __future__ import annotations using isort then run it, which will clean up annotations.

Since this seems to be covered by #530, I'll wait for that PR before merging this.

[DiddiLeija]

Ah, another weird CI failure.

[DiddiLeija]

I think our current pre-commit stuff has cleaned the code for dropping Python 3.6, right?

[Tolker-KU]

I think our current pre-commit stuff has cleaned the code for dropping Python 3.6, right?

I think the arg to pyupgrade should be bumped here: https://github.com/theacodes/nox/blob/main/.pre-commit-config.yaml#L29

[cjolowicz]

As per #526 (comment), we should wait a little before dropping Python 3.6, because it will break leave out people using pipx run nox on Ubuntu 18.04 runners, as well as people installing Nox with the system Python on the current CentOS release. I understand @henryiii 's comment as requesting to cut a Nox release with recently added features before we drop Python 3.6.

As per that same comment, a good way to upgrade the source code to use Python 3.7 features is to add the __future__ annotations import using isort, and to replace --py36-plus with --py37-plus in the pyupgrade hook configuration. Could also happen in a follow-up PR to this one.

[henryiii]

If you add me to your fork, I'm happy to help. You can use the following to inject the import:

git ls-files '*.py' | xargs pipx run isort -a "from __future__ import annotations"

I also always git grep for "version_info", "Python 3", and a few others things. I've most recently done this with tomerfiliba/plumbum#573.

Dropping 2.7 & 3.5 is huge and really helps with maintenance. I don't think dropping 3.6 will be nearly as impactful, though the annotations are nice.

PS: it will not "break" people using pipx run nox on Ubuntu 18.04, it just will mean they don't get the new features we have been adding, as they will no longer get any nox upgrades once the drop happens. That's why it's nice to make sure there's an up-to-date release.

[theacodes]

Is there any compelling reason to drop 3.6 right now? Are there any specific things in 3.6 that are holding us back?

[DiddiLeija]

I don't know about strong reasons to do this. The only reasons I know could be issues with installing dependencies for tests, and that Python 3.6 is EOL'ed.

[henryiii]

There were lots of benefits to going 3.6+, but there's really not as much going to 3.7+ beyond from __future__ import annotations (which is really nice, but if you've already written Python 3.6 annotations, it's not exactly pressing). Usually core libraries start dropping support ~6 months after EOL (if I remember correctly). Given Requires-Python, this likely won't cause a problem for a while longer, as well - since old versions keep working, so it's only when you want to start requiring new versions of 3.7+ only libraries that you'll likely hit a problem.

3.8 was a bit bigger from an API standpoint, as it lets you write positional only arguments, so designing a new API (like NumPy's new API) benefits from 3.8+. But as far as maintaining an existing library, I don't think there's much of an issue with 3.6+ for a while.

I can help update the PR then you can see if you think it's worth dropping 3.6. :)

[henryiii]

@DiddiLeija if you want to give me access to your repo I can set pyupgrade & isort to clean up the 3.6 code, or you can modify the pre-commit file as follows:

# isort hook must be before the pyupgrade one, and add the following entry:
    args: ["-a", "from __future__ import annotations"]

# pyupgrade hook should be updated from --py36-plus like this:
    args: ["--py37-plus"]

This should automatically clean up what it can, then it's easy to compare 3.6+ with 3.7+.

[DiddiLeija]

Hi @henryiii, sorry for being offline all this time (I was busy with offline responsibilities). I will try to do it by myself, but if I need help, I will add you to my fork ;)

[DiddiLeija]

Ok... I tried what suggested in #526 (comment). Any comments about the changes?

[henryiii]

Why did CHANGELOG.md and CONTRIBUTING.md change? Is there an issue with endlines or something on your git setup? That's odd.

[DiddiLeija]

Hmm... I surely made something wrong 😕.

@henryiii, I added you to DiddiLeija/nox.

[henryiii]

Ahh, those are (now broken) symlinks. Are you on Windows?

[DiddiLeija]

Thanks a lot @henryiii! Well, I suppose this is now ready to merge.

[henryiii]

I think the point was to compare and see if it's worth dropping now, or waiting a bit. This was you can see the trade offs. A few small workarounds removed, and a little bit nicer syntax, but nothing too major (not like going 3.6+). It's up to the maintainers.

[henryiii]

(I can rebase this and squash a bit so the automated stuff is operate from the few hand changes if it needs to be kept around for a while)

[DiddiLeija]

(I can rebase this and squash a bit so the automated stuff is operate from the few hand changes if it needs to be kept around for a while)

You can do it if you want, but as far as I know, I will have to squash the commits for the merge ("merge commits" are not allowed here).

[henryiii]

The idea would be to make it easy to apply in the future if it's not something you want to do right now. By separating the automated parts, you can rerun the automated parts - there weren't a huge number of hand written changes.

(So I'd only do it if you want to wait)

[DiddiLeija]

I get it. From my side, we can merge at any time, but I also would like to hear other maintainers. BTW, a new Nox release is in the air: https://github.com/theacodes/nox/releases/tag/2022.1.7. I don't know where, but we said that we could drop Python 3.6 after that release?

[FollowTheProcess]

From my side I think it's okay to drop 3.6 in the next release. As @henryiii said it's not as big a win as 3.5 -> 3.6 but as 3.6 is now EOL I think it's fair game for us to drop, we've already cut a release with all the new recent additions (2022.1.7) so 3.6 users won't be missing out on that (which was a good point for not dropping it until after that release)

[DiddiLeija]

Yeah, it's fair (and safe) to merge this now, unless we want to do #561 (comment) first.

[DiddiLeija]

If nobody says "Stop! Don't do this!" before next week, I'll just go ahead and merge this, since seems like we now want to do this.

[henryiii]

A few more bits of information:

Python stats from https://mayeut.github.io/manylinux-timeline/ Jan 10 (would be better averaged, but still interesting):

Version	Usage
2.7	5.8%
3.5	1.5%
3.6	15.1%
3.7	54.7%
3.8	17.9%
3.9	5.3%
3.10	0.2%

This is based on downloads for 3,564 packages providing manylinux wheels. The full graph is interesting too. This of course is linux users that are using binary packages (but lots of packages have binaries, so a pretty wide sample). I'd expect Windows and macOS users to be much, much more highly biased toward recent versions.

The most interesting thing to me is how wildly popular 3.7 is. I read that 1.8, 2.7, and 3.6 were especially special/popular releases, but I don't know what's up with 3.7. I can think of some major OS's with 3.6, but 3.7 seems well out of proportion; it took the most popular python (from a 2.7/3.6 tie) in middle 2020 and it's still gaining in popularity (from 2.7-3.6 users, I hope!)

I personally don't have a strong opinion either way for a tool like Nox. I'll likely start moving min versions on core stuff around the time Ubuntu 22.04 comes out and is available as a GHA runner, or around the time manylinux drops 3.6. It may depend quite a bit on development - it's easier to develop with a higher min requirement. Less-core stuff will depend on if I'm actively developing it or not - I like to get a benefit from a Python drop if possible. Numerical stuff is supposed to be 3.8+ now, according to NEP 29, so might be doing that too.

[FollowTheProcess]

@henryiii that data is actually super interesting! Weird how 3.7 is the majority!

[DiddiLeija]

Yeah, why is 3.7 so popular? That's pretty weird for me 🤔

around the time manylinux drops 3.6

Actually, I already decided to ask at pypa/manylinux#1260, maybe that reminder could be helpful.

I like to get a benefit from a Python drop if possible.

Me either. We didn't win a lot with dropping Python 3.6 like dropping 3.5 or 2, I suppose. However, I think it's important to do it. But in the other hand, I'm starting to get concerned by the Ubuntu thing that @henryiii pointed out.

[DiddiLeija]

Ok. Probably, manylinux won't drop 3.6 until 2029. So, I want to know if somebody disagrees with pushing this. If somebody does, I will just close this PR, and recreate it when needed.

cc @theacodes @FollowTheProcess @cjolowicz @dhermes @henryiii

[henryiii]

manylinux won't drop 3.6 until 2029

? Pretty sure manylinux will drop 3.6 midway through the year, like always. The main issue I was referring to was ubuntu-18.04 has Python 3.6, so if you use pipx on GHA and use ubuntu-18.04 you get 3.6 (note that ubuntu-latest is ubuntu-20.04, which is fine). Since pip is dropping 3.6, this is becoming a bit nicer. I expect drops to be happening all year.

Note that conda-forge still supports 3.6 (not sure why), so https://github.com/conda-forge/nox-feedstock will need to be updated when this is dropped (https://github.com/conda-forge/pip-feedstock forgot. Both last year and this year!)

To be clear, I don't have any objections either way, I mostly wanted a release with the recent changes out, and that happened. If nice work has gone in supporting an old Python version, it's nice to release it before dropping.

[henryiii]

Wow, you are right. Just followed the issue. I'd be shocked if it lasts till 2029, but okay. 3.6 would be like 2.7 at that point...

[henryiii]

Ahh, I (we) misread. It says not before mid-year next year (2023 with "standard support" ending for 18.04). That's more reasonable, and inline with past drops.

By the miss-read logic, Python 3.5 would not have been dropped (Ubuntu 16.04, EOL 2026), or Python 3.4 (Ubuntu 14.04, EOL 2024) or Python 2.7 (any OS, including RHEL 8 and Ubuntu 20.04, just not "default" anymore).

[henryiii]

3.5 became very expensive to support in cibuildwheel & manylinux (more so than 2.7, actually, due to the ending of patch updates for it). 3.6 might become that way too, but currently, it's not at all hard to support there.

[FollowTheProcess]

I'm +1 for dropping 👍🏻

[dhermes]

I'm big 👍 for dropping. I feel like most projects have a policy of "last 3 major releases" so it would be nice to get rid of 3.7 too (the download numbers ruin this argument).

[henryiii]

I think I've seen NEP 29 (42 months) and official Python EOL (5 years), but not "last three" - the release cadence of Python changed around 3.8, so last three would change in size of support window to only 3 years, while it would have been 5.5 before. This is not a PyData project, so I'd not go drop Python versions that are not past official Python EOL. (I rather wish NumPy had adopted Python EOL, one rule would be nicer than several - I understand why, but doesn't mean I love it)

5 years is often "normal" LTS support, too.

[DiddiLeija]

Ok, since it seems like we can drop Python 3.6, I'm gonna merge this. I think we will drop Python 3.7 the next year or even later (I don't really like the "last 3 major releases" policy). Thanks everybody for the guidance!

[henryiii]

Python 3.7's EOL is Jun 27, 2023.

[henryiii]

After that, it will be every October, in sync with new Python releases.

[DiddiLeija]

@henryiii thanks for the information! I didn't know that new Pythons will EOL in October.