Avoid re-using old ~/.pex/code/ caches.
#1444
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The original `$PEX_ROOT/code/<code_hash>` cache scheme was broken since the `<code_hash>` was calculated just using user code but the contents of the cache included Pex's `__main__.py` and `PEX-INFO` files. The latter was luckily never loaded by the Pex runtime to figure out how to run and so the bug never surfaced. With the introduction of the zipapp and packed runtime layout support, these bad cache entries are now used and the `PEX-INFO` is read. This means, for the case of PEXes with no user code that were also `--not-zip-safe`, all such PEXes share the `PEX-INFO` of the 1st such PEX executed in a given `$PEX_ROOT`. This leads to unexpected dependencies and crashes when executing any such PEX other than the seeding PEX. Fix this by simply re-naming the `code/` cache dir root to `user_code`, allowing previous PEXes to work as prior and new PEXes to also work by completely ignoring the old buggy `code/` caches. Fixes pex-tool#1443
stuhood
approved these changes
Sep 10, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The original
$PEX_ROOT/code/<code_hash>cache scheme was broken sincethe
<code_hash>was calculated just using user code but the contentsof the cache included Pex's
__main__.pyandPEX-INFOfiles. Thelatter was luckily never loaded by the Pex runtime to figure out how to
run and so the bug never surfaced.
With the introduction of the zipapp and packed runtime layout support,
these bad cache entries are now used and the
PEX-INFOis read. Thismeans, for the case of PEXes with no user code that were also
--not-zip-safe, all such PEXes share thePEX-INFOof the 1st suchPEX executed in a given
$PEX_ROOT. This leads to unexpecteddependencies and crashes when executing any such PEX other than the
seeding PEX.
Fix this by simply re-naming the
code/cache dir root touser_code,allowing previous PEXes to work as prior and new PEXes to also work by
completely ignoring the old buggy
code/caches.Fixes #1443