Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release 24.1 #12613

Closed
sbidoul opened this issue Apr 6, 2024 · 52 comments
Closed

Release 24.1 #12613

sbidoul opened this issue Apr 6, 2024 · 52 comments
Assignees
Labels
type: maintenance Related to Development and Maintenance Processes
Milestone

Comments

@sbidoul
Copy link
Member

sbidoul commented Apr 6, 2024

It's April already.

Any @pypa/pip-committers up to take the RM hat this month?

@sbidoul sbidoul added this to the 24.1 milestone Apr 6, 2024
@sbidoul sbidoul pinned this issue Apr 6, 2024
@pfmoore
Copy link
Member

pfmoore commented Apr 6, 2024

Do we expect the packaging upgrade to be in 24.1? Because I don't have the bandwidth to deal with that change.

If the packaging upgrade isn't included, I could do it - but looking at the news fragments, it doesn't look like there's much in the release - do we even need a release? (I'm assuming we don't get a flurry of people merging changes to get them into the release - I don't have the bandwidth for that, either!)

@pradyunsg
Copy link
Member

Do we expect the packaging upgrade to be in 24.1?

Yes.


If no one else has bandwidth, I'd be willing to be the RM for this one.

I expect we're going to have a beta period, to get some user feedback on the legacy version removal situation (thank you to @sbidoul for doing most of the work on #12300!) and to also make follow up fixes we'd likely need.

@pfmoore
Copy link
Member

pfmoore commented Apr 7, 2024

OK, if we're putting the packaging upgrade in, I'll leave it with you. I'm happy to help where I can, so feel free to ping me if you think I can help. I'll try to find some time to review @sbidoul's PR in the next day or two.

@sbidoul
Copy link
Member Author

sbidoul commented Apr 8, 2024

Regarding a beta release, I'm not entirely convinced it will help (my instinct says not enough people will notice/care), but that's up to the RM, of course ;)

@pradyunsg pradyunsg self-assigned this Apr 8, 2024
@notatallshaw
Copy link
Member

notatallshaw commented Apr 9, 2024

I would test a beta and I'm sure @potiuk would as well, I would also try to spread the word on social media as much as possible. However, I will say, the one time that pip recently had a beta it didn't get much exposure.

On release though, I will again mention that resolving is not sound #12317 and I have a fix here if anyone has time to provide any more feed back sarugaku/resolvelib#144

Also this PR seems fairly small and would be a great performance news item for Pip #12453 (and more complex but would be a big UX improvement news item, would be #12388 and #12404 if they are considered close to ready)

@NeilBotelho
Copy link
Contributor

I'd say #12388 is mostly done, but I'm not sure if you would want to merge it into main by itself. It only adds support for parallel downloads without progress bars. #12404 adds support for parallel progress bars, but I'd like some more feedback on that before I'd be ready to call it done.

@notatallshaw
Copy link
Member

notatallshaw commented Apr 28, 2024

A summary on macos runners which are currently blocking CI runs:

  1. macos-latest has been updated to macos-14 running on ARM from macos-12 running on x86
  2. The subversion package is not available by default and must be installed on macos-14
  3. Until a couple of days ago Python 3.8 and 3.9 weren't available on macos-14
  4. Old versions of virtualenv do not work on macos-14, and pip test suite currently pins Python 3.8 and 3.9 to virtualenv<20
  5. Removing this restriction reveals 1 failure "Tests_UninstallUserSite::test_uninstall_editable_from_usersite" on macos-14 Python 3.8 and 3.9

So it needs to be decided whether to keep running x86 as well as ARM (#12593) or just stick with "latest" which is now ARM (#12617). And someone either needs to fix this test (I don't know what the issue is and have no way to reproduce locally) or whether to exclude Python 3.8 and 3.9 from MacOS ARM.

@notatallshaw
Copy link
Member

I'd also like to nominate #12657 for 24.1, it's very simple, and I've measured a significant performance improvement on large number of packages to resolve, and even a slight perfornance improvement on small number of packages to resolve.

@pradyunsg
Copy link
Member

Oh wow, this month has flown by. I'm catching up on my RM duties now. 😅

@vx1920
Copy link

vx1920 commented Apr 30, 2024

I did report small issue while ago: useless warnings in "pip list --outdated --verbose".
Possible fix with adding two lines to commands/list.py was described, it will allow suppress warnings optionally.
See #12540
Should I create pull request ?

@colesbury
Copy link

colesbury commented May 2, 2024

I know this was asked by Paul above and answered, but is the packaging upgrade still slotted to be in the 24.1 release? (From the outside, it's hard to tell how close or far away the PR is to being ready.)

Do you expect the 24.1 release before the Python 3.13 beta 1?

@sbidoul
Copy link
Member Author

sbidoul commented May 2, 2024

I posted a status update on the PR: #12300 (comment)

@notatallshaw
Copy link
Member

notatallshaw commented May 3, 2024

As it seems there's very likely to be a beta that hopefully will get some wide coverage, I would like to make the case for pip maintainers to be a little bit more agressive merging approved open PRs that are in a good state:

https://github.com/pypa/pip/pulls?q=is%3Apr+is%3Aopen+review%3Aapproved+draft%3Afalse+status%3Asuccess+sort%3Aupdated-desc+

I appreciate pip is under resourced and generally conservative in accepting PRs, but having a long open approved unmerged PR can be a little demoralizing to potential future contributors.

@pfmoore
Copy link
Member

pfmoore commented May 3, 2024

I've moved a few of those onto the 24.1 milestone, where I think they are clear-cut ones that should be merged. The others I think might need a little more consideration than I have time for right now. (I'm not arguing against them, though).

@pradyunsg
Copy link
Member

OK, we're almost all set for 24.1b1 to be released today. I have one outstanding PR to merge and then will cut a release after grabbing lunch today.

@pradyunsg
Copy link
Member

https://pypi.org/project/pip/24.1b1/ is up. I'm not updating get-pip.py or filing a PR to CPython with an updated wheel for this.

@pfmoore
Copy link
Member

pfmoore commented May 7, 2024

Are you going to post an announcement on discuss.python.org? It might be a good way to attract a few more beta testers.

@pradyunsg
Copy link
Member

Yep, I plan to do that along with a blog post that serves as a call for beta testing.

That'll all have to be after my work day today though, since pip maintainance isn't a part of my day job anymore.

@potiuk
Copy link
Contributor

potiuk commented May 7, 2024

FYI. I can confirm that the most complex case with installation and eager upgrade for latest airflow version with 24.1b1 works fine. Not super comprehensive, but passes as "smoke test" IMHO

@pradyunsg
Copy link
Member

@potiuk Is it meaningfully quicker? :)

@potiuk
Copy link
Contributor

potiuk commented May 7, 2024

Not too scientific but looks like ~ 15% (~550 seconds -> ~470 seconds) - on a very small sample.

@notatallshaw
Copy link
Member

Not too scientific but looks like ~ 15% (~550 seconds -> ~470 seconds) - on a very small sample.

Roughly what is that running?

Anything that gets stuck doing a bit of backtracking should see more significant performance gains. But if the versions here are all tightly constrained that is about in line with what I would expect.

@potiuk
Copy link
Contributor

potiuk commented May 7, 2024

Roughly what is that running?

Roughly this:

curl -fsSL https://github.com/apache/airflow/archive/main.tar.gz | tar xz -C /tmp/tmp.18llqGZdHv --strip 1
pip install --root-user-action ignore --editable '/tmp/tmp.18llqGZdHv[devel-ci]'

@henryiii
Copy link
Contributor

Probably actions/setup-python#886 ? You could manually set 3.13.0b1 for a bit to unblock. Looks like 3.12.4 might have the same issue.

@hugovk
Copy link
Contributor

hugovk commented Jun 10, 2024

Pin using "3.13.0-beta.1", but it will need some mangling to work with the nox -s test-${{ matrix.python }} call.

@henryiii
Copy link
Contributor

I’d mangle the setup-python call instead. Easier and doesn’t change the job name. Something like ${{ matrix.python == '3.13' && '3.13.0-beta.1' || matrix.python }}.

@mayeut
Copy link
Member

mayeut commented Jun 10, 2024

A workaround shouldn't be needed anymore. Broken versions were removed from the manifest.

@pradyunsg
Copy link
Member

And, we're green! Thanks all!

@pradyunsg
Copy link
Member

https://pypi.org/project/pip/24.1b2/ is up!

@edgarrmondragon
Copy link
Contributor

24.1b2 works great for us and fixes the problem described in #12613 (comment) 🙌

@potiuk
Copy link
Contributor

potiuk commented Jun 13, 2024

Also Airlfow build works nicely with pip set as default installer and 24.1.b2 apache/airflow#40184 👍

@pypa pypa deleted a comment Jun 20, 2024
@pradyunsg
Copy link
Member

Alright, #12777 is up. I'll push all the relevant knobs tonight / tomorrow night depending on my availability.

@pradyunsg
Copy link
Member

pradyunsg commented Jun 20, 2024

Alrighty, all done with the button clicking for now.

@hugovk
Copy link
Contributor

hugovk commented Jun 21, 2024

  • I haven't filed a CPython PR, but will file one next week.

If possible, it'd be great to have it in time for Tuesday's 3.13.0b3.

https://peps.python.org/pep-0719/#schedule

@pradyunsg
Copy link
Member

@hugovk Thanks for that nudge -- made the PR now, given that there's not really any major showstoppers AFAICT with this release.

@notatallshaw
Copy link
Member

I believe #12781 needs more attention.

My cursory analysis is that the new tar extraction logic fails for tars which contains hard links, but it wasn't clear to me why, and neither the regular Python extraction or the data filter extraction failed, perhaps I am wrong in my analysis.

@pfmoore
Copy link
Member

pfmoore commented Jun 25, 2024

I believe #12781 needs more attention.

See my comments there. I'm not sure we want to rush a fix for 24.1.1, as I think the subtleties involved warrant deeper consideration, and a "quick fix" might just leave us with further issues.

This was referenced Jun 26, 2024
@pradyunsg
Copy link
Member

pradyunsg commented Jun 27, 2024

FWIW, I guess I should mention this explicitlt: main is open for non-bugfix changes. I'll cherry pick for any follow up bugfix releases we need.

This was referenced Jul 2, 2024
@pradyunsg
Copy link
Member

Cutting a 24.1.2 with #12787 and #12799 cherry-picked. #12831 is the relevant PR.

@pradyunsg
Copy link
Member

OK, assuming I don't see any fires in the next few minutes about 24.1.2 (uploaded to PyPI and get-pip.py has also been updated), I'm gonna say this doozy of a release is done with.

And, I'll move on toward making 24.2 next weekend. 😅

@pradyunsg
Copy link
Member

Ok, gonna say about 25 minutes is enough time.

Thanks everyone for helping make this release happen -- this was a surprisingly long release cycle to manage. We had hiccups and a bunch of technical details needed to be figured out and yet this went by without major disruption... and, overall, quite smoothly!

@pradyunsg pradyunsg unpinned this issue Jul 7, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
type: maintenance Related to Development and Maintenance Processes
Projects
None yet
Development

No branches or pull requests