Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: switch to uv pip-compile for lockfiles #2084

Merged
merged 2 commits into from
Dec 4, 2024

Conversation

gotmax23
Copy link
Collaborator

This switches the nox pip-compile session and lockfiles to use uv pip compile (written in Rust and much faster) instead of pip-compile from pip-tools for our dependency update jobs.

As a side effect, this addresses the issue brought up in #1950 (comment), as we're no longer using pip-compile that can break anytime pip changes the private/internal APIs that pip-tools relies on.

I re-generated the lockfiles with --no-upgrade to avoid extraneous changes in this PR.
The only additions are the formatting of comments in the lockfile and colorama.
colorama was added as a result of uv pip-compile's --universal flag which creates "universal" lockfiles that include dependencies of other platforms, architectures, and Python versions than those of the system used to generate the lockfile.
--universal should create more consistent results for contributors who use other systems
(e.g., Mac OS or a newer Python version than the one used in CI).

Fixes: #1755

Copy link
Collaborator

@felixfontein felixfontein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this!

noxfile.py Show resolved Hide resolved
Copy link
Contributor

@oraNod oraNod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments from me but I really like this. Thanks for working on it @gotmax23

noxfile.py Outdated Show resolved Hide resolved
noxfile.py Outdated Show resolved Hide resolved
noxfile.py Show resolved Hide resolved
noxfile.py Show resolved Hide resolved
@oraNod
Copy link
Contributor

oraNod commented Nov 4, 2024

@gotmax23 Just checking but did you mark the other PRs to update dependencies as draft pending this one?

@gotmax23
Copy link
Collaborator Author

gotmax23 commented Nov 5, 2024

No, they were marked as draft because they were generated using the broken version of pip-tools. I'll check them again so they can be merged in the interim (and also rebase this PR) later. Thanks!

@gotmax23 gotmax23 marked this pull request as draft November 6, 2024 07:20
@gotmax23
Copy link
Collaborator Author

This is still on my radar. I've just been very busy. Thanks everyone for your feedback so far.

@gotmax23
Copy link
Collaborator Author

Ready for review

Copy link
Member

@webknjaz webknjaz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--universal should create more consistent results for contributors who use other systems

As long as you're fine with hitting corner cases and restrictions that arise from this...

@gotmax23
Copy link
Collaborator Author

As long as you're fine with hitting corner cases and restrictions that arise from this...

Can you elaborate?

Copy link
Contributor

@oraNod oraNod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great. Thanks @gotmax23

I'd also like to hear more about the --universal option and corner cases @webknjaz Do you mean there could be some platform or OS specific dependency that gets included / not included here?

@oraNod oraNod requested a review from felixfontein December 2, 2024 13:09
@oraNod
Copy link
Contributor

oraNod commented Dec 2, 2024

@gotmax23 FYI there's now a conflict with the static requirements file. If you could fix that I think we can go ahead and merge. I believe @webknjaz is on PTO and might not respond until later. Probably no need to hold up the merge.

@felixfontein If you could also provide a final review, that'd be appreciated.

Thanks!

@webknjaz
Copy link
Member

webknjaz commented Dec 2, 2024

Do you mean there could be some platform or OS specific dependency that gets included / not included here?

Incompatibilities with some platform might be included. I heard they're using Poetry-style lock file ish method, which is known to have this weakness.

This switches the nox pip-compile session and lockfiles to use uv pip
compile (written in Rust and much faster) instead of pip-compile from
pip-tools for our dependency update jobs.

As a side effect, this addresses the issue brought up in
ansible#1950 (comment),
as we're no longer using pip-compile that can break anytime pip changes
the private/internal APIs that pip-tools relies on.

I re-generated the lockfiles with `--no-upgrade` to avoid extraneous
changes in this PR.
The only additions are the formatting of comments in the lockfile and
colorama.
colorama was added as a result of `uv pip-compile`'s `--universal` flag
which creates "universal" lockfiles that include dependencies of other
platforms, architectures, and Python versions than those of the system
used to generate the lockfile.
`--universal` should create more consistent results for contributors who
use other systems
(e.g., Mac OS or a newer Python version than the one used in CI).

Fixes: ansible#1755
Suggested-by: Don Naro <dnaro@redhat.com>
@oraNod
Copy link
Contributor

oraNod commented Dec 4, 2024

@gotmax23 I think it's time to merge this. How far do you think we should backport? Or maybe we keep it on devel until we cut stable-2.19?

Thanks! 🚀

@oraNod oraNod merged commit efd49d7 into ansible:devel Dec 4, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ci: switch pip-compile jobs to use uv pip compile
4 participants