Install of git package fails on Raspberry Pi (succeeds elsewhere) with PackageInfoError

[ChasNelson1990]

• I am on the latest Poetry version.

  • I have run poetry self update --preview

• I have searched the issues of this repo and believe that this is not a duplicate.

  • Closest is: Poetry 1.1.0b2 PackageInfoError #2850

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

  • Gist: https://gist.github.com/ChasNelson1990/6964153b4d5277518334716ac79f3a81

• OS version and name: Linux raspberrypi 5.4.51-v7+ Running poetry install should not install the current project into site-packages #1333 SMP Mon Aug 10 16:45:19 BST 2020 armv7l GNU/Linux

• Poetry version: Poetry version 1.1.0b2

• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/ChasNelson1990/6964153b4d5277518334716ac79f3a81

Issue

Hi there,

So I'm using a colleague's public bitbucket repository (j_py_sad_correlation) as a dependency for some of my own work. I'm able to use poetry to install this as a dependency on my Arch Linux laptop; and they're also able to use poetry to install this as a dependency on their OsX system. However, when I try to use poetry to install my dependencies on a Raspberry Pi v4 with the latest Raspberry Pi OS and preview Poetry I get a PackageInfoError and, to be honest, I'm just not expert enough on packages to understand what's going on here.

I initially thought it might relate to an architecture problem in setup.py that we used to have (ARM processors required an additional flag at some point in the past) but I created a branch to test this (not shown) and got the same error. So it's not that.

Based on where in poetry this error is calling, it seems that poetry has recognised this is an old module and not pep517 compliant, has fallen back to using setup.py but, when calling setup.py to get egg_info, it fails. So that must work on both Arch Linux and OsX but not on Raspberry Pi OS? I have this vague recollection that pypi use to have some RPi issues due to eggs vs wheels but, again, not a package expert.

I'm expecting this is probably a problem in my colleague's setup.py, etc. that is not RPi compatible (although I can install this by cloning the repo and running python setup.py build; python setup.py install inside a clean poetry env). But the error isn't informative enough for me to understand whether this is true or not. One thing that could be useful is for the actual EnvCommandError to be passed back to the user on line 476 of inspection/info.py (although, tbh, I don't know what this error means either).

Any suggestions? Happy to provide more info if needed.

Thanks!
Chas

P.S. If this is a RPi-specific issue with install, could that also explain why on my laptop Pillow seems to be installed from pip in a matter of seconds but on the RPi it seems to be building from source and takes so long I gave up?

[BeebBenjamin]

I came across something like this yesterday using pip to install numpy on a RPi3B+, for me it was because there wasn't a wheel available for an ARM processor, so the default is to build from source, I think. I tried to fix it using https://www.piwheels.org/, but they don't currently support the python version I have compiled (3.6). In the end I had a think about if I needed numpy or just one function. So I coded a replacement. But it is nice to know about the issue and piwheels, if I use a different python version on a RPI. I wonder if poetry could make use of piwheels.

[abn]

@ChasNelson1990 can you try with the latest pre-release please? Additionally, it will also be helpful if you can provide the stack trace by running the poetry command with -vvv.

I wonder if poetry could make use of piwheels.

@BeebBenjamin I am unclear on what you mean by this. Do you want poetry to publish there? Or use this as a index when installing packages? If the latter, you can already do this.

[BeebBenjamin]

That's exactly what I mean. I probably won't be using piwheels for my deployments though as they don't have wheels for python 3.6 on ARM. But being able use my own packages might be the way to fix this, as I could build my own wheels and keep them in a repo. Thanks for the tip.

[ChasNelson1990]

Hi there, thanks for responding. Your response was nicely timed as I'm back on this project this week.

I think, based on the above and my own issues I have two issues here (possibly both related to ARM but I think separate issues):

1. Pillow (and possibly other packages) are trying to build from source. This might be solved by using piwheels as a repository for installing packages (see below).
2. The custom package, which installs fine within a poetry env on an RPi4, isn't installing correctly as part of a poetry install.

In response to my issue 1:

@abn said:

@BeebBenjamin I am unclear on what you mean by this. Do you want poetry to publish there? Or use this as a index when installing packages? If the latter, you can already do this.

Following the Poetry documentation and this idea I did: poetry config repositories https://www.piwheels.org/simple. I assumed this would enable piwheels as an alternative (primary?) repository. However, I get a ValueError saying: You cannot remove the [repositories] section from line 155 of ~/.poetry/lib/poetry/console/commands/config.py. (Sorry for the lack of trace - this RPi is headless but I can recreate it on another RPi if needed.)

In response to my issue 2:
@abn said:

@ChasNelson1990 can you try with the latest pre-release please? Additionally, it will also be helpful if you can provide the stack trace by running the poetry command with -vvv.

I did post the trace for the original call in the linked to Gist (https://gist.github.com/ChasNelson1990/6964153b4d5277518334716ac79f3a81) if that's what you were looking for?

I have done poetry self update --preview and I seem to have the same issues. The trace for this seems to be the same for the latest pre-release as it was in my original Gist.

[ChasNelson1990]

P.S. I also note that if I install using python3 -m pip install pillow outside a poetry env it is already using piwheels where appropriate. So it seems like there's a global config in the RPi python that is, perhaps unsurprisingly, being ignored by poetry env pythons.

[abn]

Following the Poetry documentation and this idea I did: poetry config repositories https://www.piwheels.org/simple.

The command you are using is for adding repositories for publishing. What you need is source configuration in your pyproject.toml.

[[tool.poetry.source]]
name = "piwheels"
url = "https://www.piwheels.org/simple/"

Adding this, poetry will search this index first before looking on PyPI.

I did post the trace for the original call in the linked to Gist (https://gist.github.com/ChasNelson1990/6964153b4d5277518334716ac79f3a81) if that's what you were looking for?

Apologies I missed this earlier. So the issue here is that we are unable to determine the package metadata for py_sad_correlation. The root cause is that it is not possible to build the package such that we can extract the core package metadata. See error below.

Command ['/tmp/tmpzqianky9/.venv/bin/python', 'setup.py', 'egg_info'] errored with the following return code 1, and output: 
Traceback (most recent call last):
  File "setup.py", line 2, in <module>
    import numpy  # So we can work out where the numpy headers live!
ModuleNotFoundError: No module named 'numpy'

A fix might be to add a pyproject.toml file to that project with the following content.

[build-system]
requires = ["setuptools", "wheel", "numpy"]
build-backend = "setuptools.build_meta:__legacy__"

[dimbleby]

poetry now insists on pep517 builds, which almost surely either fixes this or makes all of the particulars from the old failures irrelevant.

suggest closing this, and invite a new issue for new problems