Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BuildError module not found 'pbr' (editable install of project itself, pdm 1.8.0) #601

Closed
1 task done
pawamoy opened this issue Aug 16, 2021 · 16 comments
Closed
1 task done
Labels
🐛 bug Something isn't working

Comments

@pawamoy
Copy link
Contributor

pawamoy commented Aug 16, 2021

  • I have searched the issue tracker and believe that this is not a duplicate.

Steps to reproduce

Logs:

% pdm install -v
All packages are synced to date, nothing to do.

Installing the project as an editable package...
Preparing isolated env for PEP 517 build...
Collecting setuptools
  Using cached setuptools-57.4.0-py3-none-any.whl (819 kB)
Installing collected packages: setuptools
Successfully installed setuptools-57.4.0
running develop
running egg_info
writing src/failprint.egg-info/PKG-INFO
writing dependency_links to src/failprint.egg-info/dependency_links.txt
writing entry points to src/failprint.egg-info/entry_points.txt
writing requirements to src/failprint.egg-info/requires.txt
writing top-level names to src/failprint.egg-info/top_level.txt
Traceback (most recent call last):
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/_editable_install.py", line 24, in <module>
    exec(compile(code, __file__, "exec"))
  File "/media/data/dev/python/failprint/setup.py", line 54, in <module>
    setup(**setup_kwargs)
  File "/tmp/pdm-build-env-z5ymx0jf-overlay/lib/python3.6/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.6.13/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.6.13/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.6.13/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/tmp/pdm-build-env-z5ymx0jf-overlay/lib/python3.6/site-packages/setuptools/command/develop.py", line 34, in run
    self.install_for_development()
  File "/tmp/pdm-build-env-z5ymx0jf-overlay/lib/python3.6/site-packages/setuptools/command/develop.py", line 132, in install_for_development
    self.run_command('egg_info')
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.6.13/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.6.13/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/tmp/pdm-build-env-z5ymx0jf-overlay/lib/python3.6/site-packages/setuptools/command/egg_info.py", line 291, in run
    writer = ep.resolve()
  File "/tmp/pdm-build-env-z5ymx0jf-overlay/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2456, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
ModuleNotFoundError: No module named 'pbr'
Install failprint 0.8.0+d20210816 failed
Traceback (most recent call last):
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/builders/base.py", line 78, in log_subprocessor
    subprocess.check_call(
  File "/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.9.5/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.6.13/bin/python', '-u', '/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/_editable_install.py', '/media/data/dev/python/failprint/setup.py', '/media/data/dev/python/failprint/__pypackages__/3.6', '/media/data/dev/python/failprint/__pypackages__/3.6/lib', '/media/data/dev/python/failprint/__pypackages__/3.6/bin']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pawamoy/.local/bin/pdm", line 8, in <module>
    sys.exit(main())
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/core.py", line 195, in main
    return Core().main(args)
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/core.py", line 153, in main
    raise cast(Exception, err).with_traceback(traceback)
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/core.py", line 148, in main
    f(options.project, options)
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/cli/commands/install.py", line 44, in handle
    actions.do_sync(
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/cli/actions.py", line 168, in do_sync
    handler.synchronize()
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/synchronizers.py", line 349, in synchronize
    self.install_candidate(self_key)
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/synchronizers.py", line 167, in install_candidate
    self.manager.install(can)
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/manager.py", line 46, in install
    installer(candidate)
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/installers.py", line 114, in install_editable
    builder.subprocess_runner(install_args, candidate.source_dir)
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/builders/base.py", line 212, in subprocess_runner
    return log_subprocessor(cmd, cwd, extra_environ=env)
  File "/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/builders/base.py", line 86, in log_subprocessor
    raise BuildError(f"Call command {cmd} return non-zero status.")
pdm.exceptions.BuildError: Call command ['/home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.6.13/bin/python', '-u', '/home/pawamoy/.local/pipx/venvs/pdm/lib/python3.9/site-packages/pdm/installers/_editable_install.py', '/media/data/dev/python/failprint/setup.py', '/media/data/dev/python/failprint/__pypackages__/3.6', '/media/data/dev/python/failprint/__pypackages__/3.6/lib', '/media/data/dev/python/failprint/__pypackages__/3.6/bin'] return non-zero status.

pyproject.toml:

[build-system]
requires = ["pdm-pep517"]
build-backend = "pdm.pep517.api"

[project]
name = "failprint"
version = {use_scm = true}
description = "Run a command, print its output only if it fails."
authors = [{name = "Timothée Mazzucotelli", email = "pawamoy@pm.me"}]
license = {file = "LICENSE"}
readme = "README.md"
requires-python = ">=3.6"
keywords = ["cli", "failure", "output", "runner"]
dynamic = ["version", "classifiers"]
classifiers = ["Development Status :: 4 - Beta"]
dependencies = [
    "ansimarkup~=1.4",
    "jinja2>=2.11, <4",
    "ptyprocess~=0.6; sys_platform != 'win32'",
]

[project.scripts]
failprint = "failprint.cli:main"

[tool.pdm]
package-dir = "src"

[tool.pdm.dev-dependencies]
duty = ["duty~=0.7"]
docs = [
    "mkdocs~=1.1; python_version >= '3.7'",
    "mkdocs-coverage~=0.2; python_version >= '3.7'",
    "mkdocs-gen-files~=0.3; python_version >= '3.7'",
    "mkdocs-literate-nav~=0.4; python_version >= '3.7'",
    "mkdocs-material~=7.1; python_version >= '3.7'",
    "mkdocstrings~=0.15; python_version >= '3.7'",
    "toml~=0.10; python_version >= '3.7'",
]
format = [
    "autoflake~=1.4",
    "black~=20.8b1",
    "isort~=5.8",
]
maintain = [
    "git-changelog~=0.4",
]
quality = [
    "darglint~=1.7",
    "flake8-bandit~=2.1",
    "flake8-black~=0.2",
    "flake8-bugbear~=21.3",
    "flake8-builtins~=1.5",
    "flake8-comprehensions~=3.4",
    "flake8-docstrings~=1.6",
    "flake8-pytest-style~=1.4",
    "flake8-string-format~=0.3",
    "flake8-tidy-imports~=4.2",
    "flake8-variables-names~=0.0",
    "pep8-naming~=0.11",
    "wps-light~=0.15",
]
tests = [
    "pytest~=6.2",
    "pytest-cov~=2.11",
    "pytest-randomly~=3.6",
    "pytest-sugar~=0.9",
    "pytest-xdist~=2.2",
    "hypothesis~=6.13",
]
typing = ["mypy~=0.812"]

Actual behavior

The installation of the project itself in editable mode seems to fail. Apparently it misses a pbr module/package. Same result with higher Python versions.

Note that I have enabled the feature.install_cache option. I also tried installing pbr in the environments of the different Python interpreters I'm using, without success.

Expected behavior

No errors 😊

Environment Information

% pdm info && pdm info --env
PDM version:        1.8.0                                                                      
Python Interpreter: /home/pawamoy/.basher-packages/pyenv/pyenv/versions/3.6.13/bin/python (3.6)
Project Root:       /media/data/dev/python/failprint                                           
Project Packages:   /media/data/dev/python/failprint/__pypackages__/3.6                        
{
  "implementation_name": "cpython",
  "implementation_version": "3.6.13",
  "os_name": "posix",
  "platform_machine": "x86_64",
  "platform_release": "5.13.10-arch1-1",
  "platform_system": "Linux",
  "platform_version": "#1 SMP PREEMPT Thu, 12 Aug 2021 21:59:14 +0000",
  "python_full_version": "3.6.13",
  "platform_python_implementation": "CPython",
  "python_version": "3.6",
  "sys_platform": "linux"
}
@pawamoy pawamoy added the 🐛 bug Something isn't working label Aug 16, 2021
@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 16, 2021

Same result with feature.install_cache disabled.

EDIT: I did not reinstall packages. It actually works with that feature disabled, sorry.

@frostming
Copy link
Collaborator

Hmm, do you have a setup.py in the project root?

@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 17, 2021

No I don't. Isn't PDM generating one on-the-fly for the editable install?

@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 17, 2021

Here are the contents of the intermediary setup.py:

# -*- coding: utf-8 -*-
from setuptools import setup

import codecs

with codecs.open('README.md', encoding="utf-8") as fp:
    long_description = fp.read()
INSTALL_REQUIRES = [
    'ansimarkup~=1.4',
    'jinja2>=2.11, <4',
    "ptyprocess~=0.6; sys_platform != 'win32'",
]
ENTRY_POINTS = {
    'console_scripts': [
        'failprint = failprint.cli:main',
    ],
}

setup_kwargs = {
    'name': 'failprint',
    'version': '0.8.0+d20210817',
    'description': 'Run a command, print its output only if it fails.',
    'long_description': long_description,
    'license': 'UNKNOWN',
    'author': '',
    'author_email': 'Timothée Mazzucotelli <pawamoy@pm.me>',
    'maintainer': None,
    'maintainer_email': None,
    'url': '',
    'packages': [
        'failprint',
    ],
    'package_dir': {'': 'src'},
    'package_data': {'': ['*']},
    'long_description_content_type': 'text/markdown',
    'keywords': ['cli', 'failure', 'output', 'runner'],
    'classifiers': [
        'Development Status :: 4 - Beta',
        'License :: Other/Proprietary License',
        'Programming Language :: Python :: 3',
        'Programming Language :: Python :: 3.6',
        'Programming Language :: Python :: 3.7',
        'Programming Language :: Python :: 3.8',
        'Programming Language :: Python :: 3.9',
    ],
    'install_requires': INSTALL_REQUIRES,
    'python_requires': '>=3.6',
    'entry_points': ENTRY_POINTS,

}


setup(**setup_kwargs)

@frostming
Copy link
Collaborator

Can't reproduce on my side, it seems pbr incorrectly contaminates the build env.

@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 17, 2021

Indeed. pbr is an indirect dev dependency in my project.

% python3.9 -m pip uninstall -y pbr  # removing it from __pypackages__
Found existing installation: pbr 5.6.0
Uninstalling pbr-5.6.0:
  Successfully uninstalled pbr-5.6.0

% pdm install -v --prod             
All packages are synced to date, nothing to do.

Installing the project as an editable package...
Preparing isolated env for PEP 517 build...
Collecting setuptools
  Using cached setuptools-57.4.0-py3-none-any.whl (819 kB)
Installing collected packages: setuptools
Successfully installed setuptools-57.4.0
running develop
running egg_info
writing src/failprint.egg-info/PKG-INFO
writing dependency_links to src/failprint.egg-info/dependency_links.txt
writing entry points to src/failprint.egg-info/entry_points.txt
writing requirements to src/failprint.egg-info/requires.txt
writing top-level names to src/failprint.egg-info/top_level.txt
reading manifest file 'src/failprint.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'src/failprint.egg-info/SOURCES.txt'
running build_ext
Creating /media/data/dev/python/failprint/__pypackages__/3.9/lib/failprint.egg-link (link to src)
Adding failprint 0.8.0+d20210817 to easy-install.pth file
Installing failprint script to /media/data/dev/python/failprint/__pypackages__/3.9/bin

Installed /media/data/dev/python/failprint/src
Install failprint 0.8.0+d20210817 successful

🎉 All complete!

@frostming
Copy link
Collaborator

Yeah, i had pbr in __pypackages__ but it worked fine.

@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 17, 2021

So, I guess the install_cache feature is at fault here, because without it, the installation of the project itself works fine (edited my previous comment stating otherwise).
Isn't putting all these .pth files in pypackages telling Python to load every dependency at startup?

@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 17, 2021

Hmmm OK no, it only adds the parent directories of the cached packages to sys.path.

@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 17, 2021

There's this line in setuptools/command/easy_install.py:

2031   │ # For pbr compat; will be removed in a future version.
2032   │ sys_executable = CommandSpec._sys_executable()

@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 17, 2021

Yeah I found other references to pbr in setuptools itself so this might play a role.

@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 17, 2021

It works if I replace pbr.pth with a pbr symlink to /home/pawamoy/.cache/pdm/packages/pbr-5.6.0-py2.py3-none-any/lib/pbr. It confirms that the .pth file is somehow contaminating the installation subprocess 😕

@frostming
Copy link
Collaborator

Hm, i reproduced it. The real issue is the editable build shouldn't see the dependencies under __pypackages__

@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 17, 2021

And it sees these dependencies because of the shell-enabled PEP 582 PYTHONPATH?

@frostming
Copy link
Collaborator

frostming commented Aug 17, 2021

The package's path must be present in sys.path or setup.py develop will refuse to install. It was solved this in a bit hacky way which should be cleaned after switching to PEP 660. The install cache somewhat exposed this issue.

@pawamoy
Copy link
Contributor Author

pawamoy commented Aug 17, 2021

Thanks 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants