Deprecate .egg in the importlib-metadata backend

[uranusjr]

This provides us a path to remove all pkg_resources usages on Python 3.11 or later, and thus avoid the problem that pkg_resources uses Python API deprecated in 3.12.

Note that the deprecation is fast-tracked since 3.12 is… soon. This is not exactly fast enough, since the schedule of the pip 23.3 release is a month after CPython 3.12.0. But maybe there’s a way to make it work…?

[pfmoore]

Just to note here as well, this will need to go into 23.2. It's not a bugfix so it's missed the boat for a 23.1 patch, sorry.

[uranusjr]

With #11997 merged and solving the immediate blocker for Python 3.12, should we still aim to fast-track remove egg support in the new backend in 23.3? A part of me wants to but technically there’s no concrete to do that instead of following the normal deprecation cycle instead.

[pradyunsg]

Let's do a normal deprecation cycle. Alternatively, we could drop this in 23.2 for 3.12+ and 24.0 for 3.11, within the beta period if we really really want to be eager about this.

[pfmoore]

Do we know (or can we find out) what setuptools' plans are? Presumably they don't plan on dropping pkg_resources in the near term, but can we get any indication of how much warning we'll get when that does happen? I'm personally inclined to expedite our removal, just to avoid getting messed around again by this whole situation.

On the other hand, in the past I think we've had good results from our deprecation warnings, so maybe it's worth a full deprecation cycle to get a good feel for the impact.

[pradyunsg]

Do we know (or can we find out) what setuptools' plans are? Presumably they don't plan on dropping pkg_resources in the near term, but can we get any indication of how much warning we'll get when that does happen?

Well, we can ask -- @pypa/setuptools-team any opinions?

[pradyunsg]

Given https://discuss.python.org/t/pep-715-disabling-bdist-egg-distribution-uploads-on-pypi/27610, let's just move forward on this?

[pfmoore]

Agreed, let's just do this.

[jaraco]

If there are known large-scale blockers to removing pkg_resources, I'd avoid removing it, as it would need to be backed out anyway. I don't have a good place yet to track such large-scale blockers, but I'm not surprised they might exist. I'm all in favor of changes that reduce the impedance to removing pkg_resources. I don't yet know if or when it will be feasible to remove pkg_resources from setuptools. I realize that's vague and largely unhelpful, but that's all the context I have.