Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for astral-sh/uv #3825

Closed
arjxn-py opened this issue Feb 19, 2024 · 20 comments · Fixed by #4353
Closed

Add support for astral-sh/uv #3825

arjxn-py opened this issue Feb 19, 2024 · 20 comments · Fixed by #4353
Labels
difficulty: easy A good issue for someone new. Can be done in a few hours feature priority: medium To be resolved if time allows

Comments

@arjxn-py
Copy link
Member

Description

uv is a quick Python package installer and resolver, made with Rust. It's meant to work just like pip and pip-tools, but faster.

Motivation

The introduction of the uv tool (https://astral.sh/blog/uv) marks a significant leap in package management efficiency. Offering notable speed improvements over pip, uv demonstrates superior performance across various scenarios. This enhanced speed translates to quicker package installations, contributing to smoother and more efficient workflow experiences.

Possible Implementation

Additional context

No response

@arjxn-py arjxn-py added feature difficulty: easy A good issue for someone new. Can be done in a few hours priority: medium To be resolved if time allows labels Feb 19, 2024
@arjxn-py arjxn-py changed the title Add support for [astral-sh/uv](https://github.com/astral-sh/uv) Add support for astral-sh/uv Feb 19, 2024
@agriyakhetarpal
Copy link
Member

This would be a great addition even though uv is pretty new and possibly far from production use.

I would suggest waiting on using it with the PyPI publishing workflow (see astral-sh/uv#1663) and rather add it to the PR tests after astral-sh/uv#1646 is resolved from their end (I am indeed following uv's developments pretty closely haha)

@kratman
Copy link
Contributor

kratman commented Feb 19, 2024

Is using pip a bottleneck on any of our tests?

@arjxn-py
Copy link
Member Author

arjxn-py commented Feb 19, 2024

Is using pip a bottleneck on any of our tests?

No that's not the case, but as we have a large dependency tree uv can make significant improvements in workflow speeds, plus example notebooks can also take advantage of the same once we're done experimenting with this new tool and it is stable & production ready.

I anticipate that we'd not just jump on uv considering how fast & efficient it is but would take one step at a time while keeping the infrastructure stable.

@agriyakhetarpal
Copy link
Member

I anticipate that we'd not just jump on uv considering how fast & efficient it is but would take one step at a time while keeping the infrastructure stable.

I agree. Let's wait until wntrblm/nox#762 gets merged and receives some feedback by users.

Is using pip a bottleneck on any of our tests?

We don't face a bottleneck as such since pip already receives a lot of the wheels cached in CI (though that hasn't worked all the time for me). What it doesn't do is cache the installation processes, and therefore installations (copying files around with the right permissions, that is) are relatively slower. This is where using uv is of the essence!

@kratman
Copy link
Contributor

kratman commented Feb 19, 2024

I anticipate that we'd not just jump on uv considering how fast & efficient it is but would take one step at a time while keeping the infrastructure stable.

Yeah I would be against switching at an early stage for the tool. Pip is well tested and most users already have it installed. Maybe something to revisit in a year or so

@agriyakhetarpal
Copy link
Member

A new release for nox is out which includes a [uv] extra, and it's pretty cool: https://github.com/wntrblm/nox/releases/tag/2024.03.02. Whoever wishes to work on this could also consider adding a nox badge to our README.

@santacodes
Copy link
Contributor

Seems interesting, this could also potentially decrease the build times of docker containers and the project setup times in cookiecutter templates and virtual environments or nox.

pyproject.toml and noxfile.py can be initially worked upon to test this feature and then can be moved to CI/CD (publish_pypi.yml).

I anticipate that we'd not just jump on uv considering how fast & efficient it is but would take one step at a time while keeping the infrastructure stable.

Yeah I would be against switching at an early stage for the tool. Pip is well tested and most users already have it installed. Maybe something to revisit in a year or so

If this issue is not being already worked upon by someone, I would be happy to create a draft for future references.

@kratman
Copy link
Contributor

kratman commented Mar 25, 2024

I still think we should not do this one. I don't see the gain for changing to a new tool when pip is well tested

@kratman
Copy link
Contributor

kratman commented Mar 25, 2024

Just to reiterate, I don't think the pip install is a major bottleneck

@agriyakhetarpal
Copy link
Member

agriyakhetarpal commented Mar 25, 2024

I still think we should not do this one. I don't see the gain for changing to a new tool when pip is well tested

Just to reiterate, I don't think the pip install is a major bottleneck

@santacodes @kratman on the contrary – I feel we can try it now, uv seems more stable to me than before and there's no harm (users won't get it anyway, just the developers, and an year is probably too long a wait). I was looking at this earlier in the day today and with my hopefully thorough testing mostly everything seems to work perfectly – with the exception of the error I just posted on their tracker (see above).

@agriyakhetarpal
Copy link
Member

Properly using and caching uv can shave us off ~1–1.5 minutes in our CI pipelines, which doesn't sound like a lot, but coupled with #3857 we should be able to get under the 5-minute mark with that and improve local development. But yes, it has to be done properly (actions/setup-python#822 isn't done yet, but we know the uv cache locations).

@agriyakhetarpal
Copy link
Member

I see that nox has cut a release with uv now, should simplify things!

@kratman
Copy link
Contributor

kratman commented Mar 25, 2024

I just think that we can speed up CI in other ways than continually changing the build system. For instance when I removed the benchmarks from every PR it saved 30 minutes off the total runtime of the CI.

Right now we have 46 CI jobs, at most 10 seem to run at once, and (last time I checked) on average they run around 20 minutes -> (46/10) × 20 ~ 80 minutes to run CI. If we cut down our build matrix by half it would save us 40 minutes and we would not need to make backend changes.

To me it does not seem like pip is the bottle neck. For some of the builds we might save more time by trying to find ways to avoid building our dependencies from source

@agriyakhetarpal
Copy link
Member

Yes, that's right – removing a lot of the redundant CI would be quite helpful. But about the bottleneck part, I see that by replacing pip I see some insane speedups locally (it takes a minute in CI if one installs [all,dev,jax,odes] where having a cached wheel for scikits.odes would help – we recently removed those because it breaks sometimes, might be worth removing that hack)

@agriyakhetarpal
Copy link
Member

agriyakhetarpal commented Mar 25, 2024

Maybe @santacodes you can try out with a draft PR (or in a branch on your fork) and we can see to it if everything works well in CI? If it doesn't, we can always close it and look at this at another time

@santacodes
Copy link
Contributor

Maybe @santacodes you can try out with a draft PR (or in a branch on your fork) and we can see to it if everything works well in CI? If it doesn't, we can always close it and look at this at another time

Sure, happy to try it out in my fork with CI resources to test the performance

@agriyakhetarpal
Copy link
Member

The uv developers' implementation of their dependency resolver is correct here and pip's resolvelib is in the wrong (they have a lot of open issues about pre-release backtracking) – so we won't be able to install scikits.odes at this time with Python 3.11 unless either scikits.odes fixes the Cython upper pin on their end with a new release (which no one takes how long it can take), or we drop scikits.odes as a dependency.

We can keep this open for a while – with the exception of scikits.odes, all of our dependencies have pure Python or binary wheels, are maintained with no absurd upper bounds1 for their build-time dependencies, and installing them is much more trivial (I tested an editable installation without [odes] and it took 40 seconds instead of three minutes, with most of the time spent in compiling PyBaMM).

Footnotes

  1. The scikits.odes developers upper-pinned to cython<3.0.0a8, which means uv correctly picks an earlier pre-release (3.0a7), while pip picks a stable release (0.29.37). They should have pinned to cython<3 instead which would have had fixed all problems here. Note that the backtracking for pre-releases is ambiguous, so there is no right answer (but uv's is making more sense to me).

@Saransh-cpp
Copy link
Member

Some arguments for switching to uv (eventually; hence, keeping this issue open to keep an eye on the development) -

  1. Nox has a uv backend now

  2. PyPA's build now supports uv (making things kind of official)

  3. Pixi now supports uv

  4. A lot of conversation and experiments involving uv are taking place in Scikit-HEP:

    Switching to uv cut the total job times for hist in half.

    For example, using nox to build nox’s docs goes from 22 seconds to 4 seconds using uv.

@agriyakhetarpal
Copy link
Member

to add to the above: #3932 will remove scikits.odes, so all of our dependencies will have wheels for all platforms that we support (both platform-specific binaries or pure Python ones).

@valentinsulzer
Copy link
Member

Good to see this being adopted more. I'd be happy to switch to this once we see some of the big python packages (e.g. numpy) start to switch over. I do agree that pip isn't a major bottleneck in our testing.

@santacodes santacodes mentioned this issue Aug 16, 2024
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
difficulty: easy A good issue for someone new. Can be done in a few hours feature priority: medium To be resolved if time allows
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants