Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not build wheels for XXX which use PEP 517 #1516

Closed
2 tasks done
albireox opened this issue Oct 29, 2019 · 15 comments · Fixed by python-poetry/poetry-core#12
Closed
2 tasks done

Could not build wheels for XXX which use PEP 517 #1516

albireox opened this issue Oct 29, 2019 · 15 comments · Fixed by python-poetry/poetry-core#12
Labels
kind/bug Something isn't working as expected

Comments

@albireox
Copy link

albireox commented Oct 29, 2019

  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • OS version and name: macOS 10.15
  • Poetry version: 1.0.0b4

Issue

#11 provides a workaround for custom build systems (e.g., Cython extensions) that works well with poetry. However, when you try to do pip install . on a project that has a custom build.py the build process fails with

ERROR: Could not build wheels for seiya which use PEP 517 and cannot be installed directly

This means, among other things, that one cannot use ReadTheDocs to document the code. This happens with my code but I have also tested it with pendulum.

Is there a workaround for this?

@albireox albireox added the kind/bug Something isn't working as expected label Oct 29, 2019
@albireox
Copy link
Author

Doing some more Googling on this I came up with the solution described here. One can create a setup.py file such as

from distutils.core import setup

import numpy

from Cython.Build import cythonize
from setuptools.extension import Extension


global setup_kwargs


extensions = [
    Extension('seiya.cube.cubify',
              ['seiya/cube/cubify.pyx'],
              include_dirs=[numpy.get_include()])
]


setup_kwargs = {}
setup_kwargs.update({'ext_modules': cythonize(extensions, annotate=True)})


setup(**setup_kwargs)

and remove the need of using build.py. As far as I can see this works fine (even allows editable installs!). When you do poetry install it creates an UNKNOWN.egg-info directory with PKG-INFO in which all the fields are UNKNOWN. That does not seem to affect poetry but I'm not sure what will happen when you build a distribution (I have run poetry build and the results look ok, with the correct metadata, but I haven't tested it all the way by uploading it to PyPI and installing from a fresh environment).

IMHO poetry really needs to define a way to deal with extensions. There are many projects that depend on a custom build system and this kind of workarounds are getting old ...

@albireox
Copy link
Author

@sdispater Any comments on this? I'm seeing problems with this workaround with a recent project that requires compilation of extensions and being able to install using pip install . and I cannot find a good solution.

@flokno
Copy link

flokno commented Mar 31, 2020

+1 from me and probably related to #2214

I use numpy to compile a small fortran module. Installation works fine with poetry install, but with pip install ., pip complains that it does not find a setup.py. From what I can see this should be easy to fix since poetry install builds a temporary setup.py as far as I understand.

abn added a commit to abn/poetry-core that referenced this issue Mar 31, 2020
abn added a commit to abn/poetry-core that referenced this issue Mar 31, 2020
abn added a commit to abn/poetry-core that referenced this issue Mar 31, 2020
abn added a commit to abn/poetry-core that referenced this issue Mar 31, 2020
abn added a commit to abn/poetry-core that referenced this issue Mar 31, 2020
abn added a commit to abn/poetry-core that referenced this issue Apr 7, 2020
abn added a commit to abn/poetry-core that referenced this issue Apr 9, 2020
abn added a commit to python-poetry/poetry-core that referenced this issue Apr 12, 2020
@albireox
Copy link
Author

albireox commented May 15, 2020

Thanks @abn for fixing this. However, I cannot completely make it work. If I add

[build-system]
build-backend = "poetry.core.masonry.api"
requires = ["poetry-core>=1.0.0a6"]

to my pyproject.toml it fails with

  Building wheel for sdss-flicamera (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/bin/python3.8 /Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/tmpzqz9ab7n
       cwd: /Users/albireo/Documents/Code/sdss5/flicamera
  Complete output (24 lines):
  Traceback (most recent call last):
    File "setup.py", line 2, in <module>
      from setuptools import setup
  ModuleNotFoundError: No module named 'setuptools'
  Traceback (most recent call last):
    File "/Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
      main()
    File "/Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 204, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/private/var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/pip-build-env-p_gl0npf/overlay/lib/python3.8/site-packages/poetry/core/masonry/api.py", line 57, in build_wheel
      return unicode(WheelBuilder.make_in(poetry, Path(wheel_directory)))
    File "/private/var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/pip-build-env-p_gl0npf/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 56, in make_in
      wb.build()
    File "/private/var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/pip-build-env-p_gl0npf/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 82, in build
      self._build(zip_file)
    File "/private/var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/pip-build-env-p_gl0npf/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 101, in _build
      self._run_build_command(setup)
    File "/private/var/folders/gp/vsl4fq4d47s9mpywmcbrsj7c0000gn/T/pip-build-env-p_gl0npf/overlay/lib/python3.8/site-packages/poetry/core/masonry/builders/wheel.py", line 129, in _run_build_command
      subprocess.check_call(
    File "/Users/albireo/.pyenv/versions/3.8.2/Python.framework/Versions/3.8/lib/python3.8/subprocess.py", line 364, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['/Users/albireo/.pyenv/versions/3.8.2/envs/flicamera/bin/python3.8', 'setup.py', 'build', '-b', 'build']' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for sdss-flicamera
Failed to build sdss-flicamera
ERROR: Could not build wheels for sdss-flicamera which use PEP 517 and cannot be installed directly

However, if I do

[build-system]
build-backend = "poetry.core.masonry.api"
requires = ["poetry-core>=1.0.0a6", "setuptools>=46.0.0"]

Then it works. Is this expected and if so, should the build-system documentation be updated?

I'm running this on a virtual environment using pyenv with Python 3.8.2. Note that the virtual environment has setuptools installed but that doesn't seem to make a difference.

@abn
Copy link
Member

abn commented May 15, 2020

@albireox what's the version of pip in your environment? Can you try updating it?

@albireox
Copy link
Author

With the latest pip 20.1 the problem still happens (that's what I had been using originally).

@albireox
Copy link
Author

Maybe related to #2276 (comment)

@virtustate
Copy link

Same problem working through https://www.pythoncheatsheet.org/blog/python-projects-with-poetry-and-vscode-part-1/ installing pendulum. Conda installs pendulum fine. I use Conda as a pip fallback for things like scipy on Windows.

@abn
Copy link
Member

abn commented Sep 30, 2020

@albireox the issue is the isolated environment created by pip when building th wheel. #2666 might help mitigate this issue and so will #2826 since that will mean we rely on virtualenv everytime to build packages.

If he issue persists, please create a new issue, so we can track it and discuss it correctly. (#3001 maybe?).

@albireox
Copy link
Author

@abn I tried this again, without the setuptools requires, and now it seems to work fine. If I see the issue pop up again I'll open a dedicated issue.

@ag-hcoder
Copy link

Any solution yet?

@finswimmer
Copy link
Member

@ag-hcoder: I believe the problem here is setuptools itself. See #3153 (comment)

@syahdeini
Copy link

I use python 3.8.0, and this problem still occurs


    error: Command "clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/bzip2/include -I/usr/local/opt/readline/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/bzip2/include -I/usr/local/opt/readline/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DNPY_INTERNAL_BUILD=1 -DHAVE_NPY_CONFIG_H=1 -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE=1 -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/core/src/common -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/core/src/umath -Inumpy/core/include -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/core/include/numpy -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/distutils/include -Inumpy/core/src/common -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -Inumpy/core/src/_simd -I/Users/XXX/Library/Caches/pypoetry/virtualenvs/aep-T5QCnjDx-py3.8/include -I/Users/XXX/.pyenv/versions/3.8.0/include/python3.8 -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/core/src/common -Ibuild/src.macosx-11.1-x86_64-3.8/numpy/core/src/npymath -c build/src.macosx-11.1-x86_64-3.8/numpy/core/src/umath/loops_unary_fp.dispatch.c -o build/temp.macosx-11.1-x86_64-3.8/build/src.macosx-11.1-x86_64-3.8/numpy/core/src/umath/loops_unary_fp.dispatch.o -MMD -MF build/temp.macosx-11.1-x86_64-3.8/build/src.macosx-11.1-x86_64-3.8/numpy/core/src/umath/loops_unary_fp.dispatch.o.d -msse -msse2 -msse3" failed with exit status 1
    ----------------------------------------
    ERROR: Failed building wheel for numpy
  Failed to build numpy
  ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly
  WARNING: You are using pip version 21.0.1; however, version 21.1.2 is available.
  You should consider upgrading via the '/Users/XXX/Library/Caches/pypoetry/virtualenvs/aep-T5QCnjDx-py3.8/bin/python -m pip install --upgrade pip' command.

@jordi-crespo
Copy link

Any updates on this error?

Copy link

github-actions bot commented Mar 2, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants