PDM will unconditionally install packages with MSYS2 Python rather than compile C/Rust extensions. #3158
Closed
1 task done
Labels
🐛 bug
Something isn't working
Comments
Interesting, the built wheel is also named |
|
I think the MSYS2 |
|
That's tricky since I don't have a msys2 environment for testing, can you paste the output of |
William@DESKTOP-3H1DSBV MINGW64 ~
$ python3 -m pip debug --verbose
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
pip version: pip 24.1.2 from C:/msys64/mingw64/lib/python3.11/site-packages/pip (python 3.11)
sys.version: 3.11.9 (main, Apr 12 2024, 09:55:31) [GCC 13.2.0 64 bit (AMD64)]
sys.executable: C:/msys64/mingw64/bin/python3.exe
sys.getdefaultencoding: utf-8
sys.getfilesystemencoding: utf-8
locale.getpreferredencoding: cp1252
sys.platform: win32
sys.implementation:
name: cpython
'cert' config value: Not specified
REQUESTS_CA_BUNDLE: None
CURL_CA_BUNDLE: None
pip._vendor.certifi.where(): C:/msys64/mingw64/lib/python3.11/site-packages/pip/_vendor/certifi/cacert.pem
pip._vendor.DEBUNDLED: False
vendored library versions:
CacheControl==0.14.0
distlib==0.3.8
distro==1.9.0
msgpack==1.0.8
packaging==24.1
platformdirs==4.2.1
pyproject-hooks==1.0.0
requests==2.32.3
certifi==2024.02.02
idna==3.7
urllib3==1.26.18
rich==13.7.1 (Unable to locate actual module version, using vendor.txt specified version)
pygments==2.17.2
typing_extensions==4.11.0 (Unable to locate actual module version, using vendor.txt specified version)
resolvelib==1.0.1
setuptools==69.5.1 (Unable to locate actual module version, using vendor.txt specified version)
tenacity==8.2.3 (Unable to locate actual module version, using vendor.txt specified version)
tomli==2.0.1
truststore==0.9.1
Compatible tags: 39
cp311-cp311-mingw_x86_64
cp311-abi3-mingw_x86_64
cp311-none-mingw_x86_64
cp310-abi3-mingw_x86_64
cp39-abi3-mingw_x86_64
cp38-abi3-mingw_x86_64
cp37-abi3-mingw_x86_64
cp36-abi3-mingw_x86_64
cp35-abi3-mingw_x86_64
cp34-abi3-mingw_x86_64
cp33-abi3-mingw_x86_64
cp32-abi3-mingw_x86_64
py311-none-mingw_x86_64
py3-none-mingw_x86_64
py310-none-mingw_x86_64
py39-none-mingw_x86_64
py38-none-mingw_x86_64
py37-none-mingw_x86_64
py36-none-mingw_x86_64
py35-none-mingw_x86_64
py34-none-mingw_x86_64
py33-none-mingw_x86_64
py32-none-mingw_x86_64
py31-none-mingw_x86_64
py30-none-mingw_x86_64
cp311-none-any
py311-none-any
py3-none-any
py310-none-any
py39-none-any
py38-none-any
py37-none-any
py36-none-any
py35-none-any
py34-none-any
py33-none-any
py32-none-any
py31-none-any
py30-none-anyDo you still have the |
|
Just to make things more interesting, |
|
I have installed EDIT: Ignore the |
|
Upgrade |
|
I've run into a different behavior. So that's progress :D! |
it's not, what is the output of There are only two places that raise such error in dep-logic, but you didn't provide the full traceback, which is always a good habbit. I can't find why. |
Apologies, you are of course correct. I have updated the traceback.
William@DESKTOP-3H1DSBV MINGW64 ~/Projects/MCVE/pdm-install
$ python
Python 3.11.10 (main, Sep 10 2024, 13:02:13) [GCC 14.2.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sysconfig
>>> sysconfig.get_platform()
'mingw_x86_64'
>>> |
That's somehome surprising, sorry I don't work on msys2 and didn't test the parser on this platfrom. |
|
Starting from line 109 in dep-logic/tags/platform.py, I rewrote the code as the following: if len(platform_info) == 1:
if platform_info[0] in "win32":
return cls(os.Windows(), Arch.X86)
elif platform_info[0] in "mingw_x86_64":
return cls(os.Generic("mingw"), Arch.parse("x86_64"))
else:
raise PlatformError(f"Unsupported platform {platform_}")This got the expected behavior of compiling from source :D. Feel free to steal/modify/ignore. Why Not Just Use
|
|
Upgrade dep-logic to 0.4.9 to fix this: |
|
Beautiful, this fixed everything! Thank you for the quick response and fix, sorry for not providing a traceback the first time :). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
If a Python package has non-Python code, generally with MSYS2 Python, the package must be compiled from source due to technically incompatible ABIs1. Since commit c190e14,
pdmwill install such packages immediately using a wheel for MSVC rather than compile the wheel from source.c190e14 is 2000+ lines of changes; I imagine this particular behavior can be fixed in a few lines (I hope!). I would be happy to bisect further, but the
feat/lock-targetsbranch in #2995 doesn't seem to exist anymore.jupyter notebookfails to properly initialize due to a missing Cython-compiledzmq,ruffworks fine.Steps to reproduce
Checkout a version of
pdmafter/during commit c190e14 and install it using a Python provided by the MSYS2 project:Create a dummy
pdmproject:Install any non-pure-Python package (i.e. requires C or Rust code). For this example, I chose ruff:
Actual behavior
ruff/your Python package of choice is immediately installed from an upstream wheel with an incompatible ABI.Expected behavior
ruff/your Python package of choice should be compiled from source. This is what happens in 7d39569, the commit immediately preceding c190e14:(
cargo/rustcoutput has been stripped for brevity)With a recent
pip, compiling from source is also what happens:Environment Information
I tested against c190e14, but can confirm that the issue still exists in the current HEAD.
The text was updated successfully, but these errors were encountered: