build: Deduplicate mismatched pip.txt files that caused build failure #32081
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
There was a `requirements/pip.txt` with old versions, and a newer `requirements/edx/pip.txt` managed via a `pip.in` file. The old one was used in most places, but came out of sync with pip-tools.txt, which was managed properly. Eventually this caused a `pip check` failure due to the mismatch. This should resolve at least part of edx/edx-arch-experiments#267 This PR moves pip.in and pip-tools.in and their corresponding pin files up to the `requirements/` dir, since they should be shared between the edx and sandbox environments. This also has the effect of upgrading pip to match the version in the file we've been uselessly upgrading. Other changes: - Remove `-q` option from pip and pip-sync calls, as it was hiding some debugging information that would have resolved this sooner. - Depend on `pre-requirements` from `compile-requirements`, rather than from `upgrade`. (The base target is the one that actually needs it.) This also lets us remove the explicit `pip install pip-tools` line. - Install the recompiled pip and pip-tools files right away, not after the loop. When we upgrade pip-tools, we want to use the upgraded version, not the previous version. This requires moving the pip-tools.txt recompilation outside of the loop and into its own explicit line. - Don't upgrade pip if we're not running `make upgrade` (respect the compile options). - Remove apparently-unneeded `--no-emit-trusted-host --no-emit-index-url` options (we don't pass trusted-host or index-url options).
UsamaSadiq
approved these changes
Apr 17, 2023
There was a problem hiding this comment.
I was trying to fix mysql migration error along with this issue in the PR #32077.
Since the tests are passing in this PR, I'll close mine in favour of this PR.
robrap
approved these changes
Apr 17, 2023
|
EdX Release Notice: This PR has been deployed to the staging environment in preparation for a release to production. |
|
EdX Release Notice: This PR has been deployed to the production environment. |
52 tasks
mariajgrimaldi
pushed a commit
that referenced
this pull request
May 31, 2023
…#32081) There was a `requirements/pip.txt` with old versions, and a newer `requirements/edx/pip.txt` managed via a `pip.in` file. The old one was used in most places, but came out of sync with pip-tools.txt, which was managed properly. Eventually this caused a `pip check` failure due to the mismatch. This should resolve at least part of edx/edx-arch-experiments#267 This PR moves pip.in and pip-tools.in and their corresponding pin files up to the `requirements/` dir, since they should be shared between the edx and sandbox environments. This also has the effect of upgrading pip to match the version in the file we've been uselessly upgrading. Other improvements: - Remove `-q` option from pip and pip-sync calls, as it was hiding some debugging information that would have resolved this sooner. - Depend on `pre-requirements` from `compile-requirements`, rather than from `upgrade`. (The base target is the one that actually needs it.) This also lets us remove the explicit `pip install pip-tools` line. - Install the recompiled pip and pip-tools files right away, not after the loop. When we upgrade pip-tools, we want to use the upgraded version, not the previous version. This requires moving the pip-tools.txt recompilation outside of the loop and into its own explicit line. - Don't upgrade pip if we're not running `make upgrade` (respect the compile options). - Remove apparently-unneeded `--no-emit-trusted-host --no-emit-index-url` options (we don't pass trusted-host or index-url options). (cherry picked from commit b852344)
mtyaka
pushed a commit
that referenced
this pull request
Jun 12, 2023
…#32081) There was a `requirements/pip.txt` with old versions, and a newer `requirements/edx/pip.txt` managed via a `pip.in` file. The old one was used in most places, but came out of sync with pip-tools.txt, which was managed properly. Eventually this caused a `pip check` failure due to the mismatch. This should resolve at least part of edx/edx-arch-experiments#267 This PR moves pip.in and pip-tools.in and their corresponding pin files up to the `requirements/` dir, since they should be shared between the edx and sandbox environments. This also has the effect of upgrading pip to match the version in the file we've been uselessly upgrading. Other improvements: - Remove `-q` option from pip and pip-sync calls, as it was hiding some debugging information that would have resolved this sooner. - Depend on `pre-requirements` from `compile-requirements`, rather than from `upgrade`. (The base target is the one that actually needs it.) This also lets us remove the explicit `pip install pip-tools` line. - Install the recompiled pip and pip-tools files right away, not after the loop. When we upgrade pip-tools, we want to use the upgraded version, not the previous version. This requires moving the pip-tools.txt recompilation outside of the loop and into its own explicit line. - Don't upgrade pip if we're not running `make upgrade` (respect the compile options). - Remove apparently-unneeded `--no-emit-trusted-host --no-emit-index-url` options (we don't pass trusted-host or index-url options). (cherry picked from commit b852344)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a
requirements/pip.txtwith old versions, and a newerrequirements/edx/pip.txtmanaged via apip.infile. The old one was used in most places, but came out of sync with pip-tools.txt, which was managed properly. Eventually this caused apip checkfailure due to the mismatch.This should resolve at least part of edx/edx-arch-experiments#267
This PR moves pip.in and pip-tools.in and their corresponding pin files up to the
requirements/dir, since they should be shared between the edx and sandbox environments. This also has the effect of upgrading pip to match the version in the file we've been uselessly upgrading.Other improvements:
-qoption from pip and pip-sync calls, as it was hiding some debugging information that would have resolved this sooner.pre-requirementsfromcompile-requirements, rather than fromupgrade. (The base target is the one that actually needs it.) This also lets us remove the explicitpip install pip-toolsline.make upgrade(respect the compile options).--no-emit-trusted-host --no-emit-index-urloptions (we don't pass trusted-host or index-url options).