Dropping support for Python 2 and Python 3 below 3.5

[webknjaz]

🕒 It's time.

I'd like to implement dropping the support on Dec 31st, 2019 because I still hope to find some time to improve a few things that I'd like to make it into py2-supported version.

❓ I'm submitting a ...

• 🐞 bug report
• 🐣 feature request
• ❓ question about the decisions made in the repository

🐞 Is your feature request related to a problem? Please describe.

Maintenance burden. Inability to use new language features.

🐣 Describe the solution you'd like

Drop support for old Python versions.

📋 Describe alternatives you've considered

Keep suffering?

📋 Additional context

https://python3statement.org

[jaraco]

There's nothing to stop someone (especially maintainers) from contributing to cheroot prior to the 3.x-only release. We can even create a maint/8.x branch to mirror what CherryPy has for LTS.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[webknjaz]

Current status: I want to do this after #262 and #243. And maybe #256 if we can decide to go for it.

[webknjaz]

TODO: drop cryptography deprecation warning ignores from pytest.ini once this is done.

[Safihre]

Or, just drop 2.7.. The core usage of cheroot is still cherrypy, since cherrypy dropped support, why bother many months more? :)

[jaraco]

Or, just drop 2.7.. The core usage of cheroot is still cherrypy, since cherrypy dropped support, why bother many months more? :)

Good question. One case is that older versions of CherryPy still can and do pick up the latest cheroot.

I'd like to drop support for sunset Pythons (3.5, 2.7) ASAP, but I want cheroot to be stable. There were a couple of regressions identified with recent releases and I want to be sure those are addressed. Looking at the changelog, it does look as if several regressions were fixed. I'll scan the issues and see if there are any outstanding we need to address (or roll back).

Current status: I want to do this after #262 and #243. And maybe #256 if we can decide to go for it.

At one point, getting these features in for Python 2 support had merit, but it feels like that time has passed. Let's freeze feature development for 8.x and focus on getting 9.x out for Python 3.6+.

[webknjaz]

@jaraco I think that the most important thing is to fix the regressions introduced by the migration to selectors and race conditions related to that. There's a few PRs that have to be reviewed but they are complex and since I'm worried about introducing more regressions, I've been postponing reviewing them. I want to dedicate a sufficient amount of time for the review to make sure that fixes to the regression don't slip through our fingers again. I'd be extremely helpful to get your opinion on fixes by @liamstask. IIRC the PRs should be reviewed in the following order:

• connections: add threadsafe selector wrapper #325 connections: add threadsafe selector wrapper #337
• connections: put conns safely #319 (superseded)
• server: process connections as they become active, rather than on tick #311

After this is addressed, it should be safe to announce the feature freeze for 8.x and drop py2 in 9+. It looks like #262 needs me to take over it but if by any chance it'll get merged, that'd be cool to do in 8.x.

[jaraco]

It looks like #262 needs me to take over it but if by any chance it'll get merged, that'd be cool to do in 8.x.

I'd expect that such a change would be de-stabilizing. I'd rather not accept any more de-stabilizing changes in 8.x and focus on stabilizing changes so that the legacy support can be dropped. Let's get #311 (or similar) merged and then proceed with requiring Python 3.6.

[webknjaz]

Fair enough. But here's a few more things that may be considered as soft blockers:

• [TODO] 👷 Kill Travis CI in favor of GHA  #343
• 🐛 Suppress legit OS errors on socket shutdown #342

[jaraco]

It occurs to me that the changes applied to the head of the repo have not always been stabilizing, so I'd like to push forward with the freeze and dropping Python support. I'll get #311 merged and released and then proceed with #339. We'll still have a maintenance branch for 8.x where stabilizing changes and infrastructure updates can (and should) be targeted.

[jaraco]

I've created the maint/8.x branch to track changes that target Python 3.5 and 2.7 compatibility.

[webknjaz]

It occurs to me that the changes applied to the head of the repo have not always been stabilizing

Really? That wasn't my intention at all. I was trying to only put bugfixes there + some infra fixes, of course.

[jaraco]

It occurs to me that the changes applied to the head of the repo have not always been stabilizing

Really? That wasn't my intention at all. I was trying to only put bugfixes there + some infra fixes, of course.

Apologies. I didn't mean that to be critical. I was just observing that the stabilizing changes we were making, such as #337 were themselves causing regressions (#341), or #311 which is a minor redesign of the implementation. So although these were bugfixes, they weren't simple tweak bugfixes but rewrites of behavior.

I meant my comment as a matter-of-fact and not a critique and all in good spirit. Sorry if it came off as frustrated. I'm not.

[webknjaz]

@jaraco you are right, those changes are non-trivial and I think we did best we could given the circumstances. FWIW I'm trying to put in more time into making the CI green and I'm rather stuck in attempting to synchronize states in the flaky tests to make them more deterministic.

[Safihre]

When will Python 2 support be dropped finally?

[jaraco]

Yes. It's time to get this done, even if cheroot is unstable.

[dvzrv]

Hi! What's the status on this?

AFAICT this issue blocks moving to more recent version of setuptools-scm, which in turn blocks the removal of setuptools-scm-git-archive.

[webknjaz]

Why is it important to move to the newer version of the build system plugin? The current build deps are compatible across all supported versions of Python already.

[Safihre]

@webknjaz does it matter at all? Why is this still not done?
I don't understand this project at all, barely any improvements/fixes and only updates related to tests and test frameworks.
Sorry for sounding sour but I was just hoping for a bit of development...

[dvzrv]

Why is it important to move to the newer version of the build system plugin?

Because we would like to remove the use of setuptools-scm-git-archive on Arch Linux, as it is yet another package we would potentially need to bootstrap for each minor version upgrade of the Python interpreter.

The current project setup prevents doing this, as setuptools-scm >= 7.0.0 (which is required for dropping setuptools-scm-git-archive) is not compatible with python < 3.7 (and only >= 3.7 is supported by Python upstream).
Deprecating and removing the support for long unsupported Python interpreter versions (as attempted in #510 - but somehow not yet merged) would help a great deal in dropping dependencies (such as setuptools-scm-git-archive) and removing workarounds for old interpreter versions.

[webknjaz]

@Safihre I was hoping that it was obvious that this is blocked by the lack of time. Also, I find it more impactful to maintain stability, hence improvements in test deps locking and reproducibility of testing.
@dvzrv I know that Jason tried removing py2 support in one huge chunk, but probably didn't have time either. Both of us maintain a number of projects and are trying to come up with scalable ways of doing this, which means that improving things on the scale (and testing stability too, in my case) sometimes gets more priority.
AFAIK, all the downstream packaging ecosystems have standardized ways of applying patches to the upstream source. I think that in your case, it should be enough to delete the setuptools-scm-git-archive from the build deps in such a patch, followed by replacing .git_archival.txt with https://github.com/pypa/setuptools_scm/blob/main/.git_archival.txt.
Among other blockers is that I'd like to get some known bug fixes in, before completing this task. The problem is that these are quite complex and require substantial amounts of time, so when I have a little time, I do what I can in that time slot instead of doing nothing at all.

[Safihre]

In my personal opinion the number of test related commits in this and cherrypy repo has been way too many compared to actual fix/feature commits. The ratio seems completely off for a long time. Instead of small test related commits, the combined time could have been used to actually move things forward instead of keeping these projects in their stagnent state. What's the point of good tests if they are not used to test fixes or test new features?

[webknjaz]

Note that most of them are automated (the pin updates), hence they don't require almost any effort.

[jaraco]

I've merged the PR to remove Python 3.5 and earlier support and released it as v9.0.0. I don't trust the automated release processes to work, so I just cut the release manually.

[Safihre]

😍🎉

[webknjaz]

@jaraco I'm planning to document this but it should be possible to trigger the release automation by using the Run workflow button and entering the desired version at https://github.com/cherrypy/cheroot/actions/workflows/ci-cd.yml. Here's what the previous release workflow run looks like FYI: https://github.com/cherrypy/cheroot/actions/runs/1651628800. It is supposed to pause before running the "production" PyPI release, asking for an explicit permission (just click an approval button on the workflow run page). That automation also results in a gh release and a discussion attached to in, like #426.