"Unable to find installation candidates for" packages cached from a LegacyRepository (/simple)

[awilkins]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

  • Caveat : I have insufficient detail to decide if "Unable to find installation candidates for" for packages that are already installed #4679 is a dupe of this issue, but it is possible since it has similar symptoms

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• CodeBuild image aws/codebuild/standard:5.0 (Ubuntu 20.04)

• Poetry 1.1.11 : as installed by pip

  • Yes, this is not the recommended way to install Poetry
  • However, the recommended way ought to have the same issue (installs same version of cachecontrol, at present)
  • Confirmed that using install-poetry.py on a fresh install reproduces this issue

Issue

Have been debugging this inside a container based on aws/codebuild/standard:5.0 (since this is the target build container for our CI pipelines).

Pipelines are installing poetry via pip and failing. Build works fine on my workstation, even after a poetry self update, which doesn't seem to update dependencies if they already meet the version constraints (like pip).

After a fresh install of Poetry 1.1.11 via install-poetry.py, now also having similar issues (with different symptoms but I presume the same cause) locally.

Issues are resolved by activating Poetry's venv and running

pip install cachecontrol==0.12.6

This only applies to packages installed (from a Nexus repo) via the /simple interface, which uses the LegacyRepository class.

Suspected issue

Working on a deeper diagnosis, but thought I'd get the bug report opened.

cachecontrol released 0.12.7 as of 12 hours ago, clearly there is a bug or change in behaviour (aka - bug, for a patch version bump) that breaks Poetry for this use case.

The issue seems to be caching of the HTML metadata from the /simple type repo ; while the cache file on disk is successfully updated each time it is fetched, the cache-wrapped response object for the metadata returns an empty byte buffer to the program. This results in the Page object not finding any links and get_links_for_package failing.

In a tree with a lockfile and in an environment with an empty cache, this means you will be able to install that package once and once only - the first time caches the metadata, which then subsequently triggers this bug and prevents further access to the metadata. Sadly all my pipelines build multiple packages with common dependencies.

Now going to look and see what the problem in cachecontrol is.

Might I humbly suggest that as a tool which promotes stable builds ... that Poetry use some of it's own medicine to install itself, and use a lockfile or output of pip freeze to create a reproducible install for a given version. This is definitely not the first occasion on which a reproducible installation would have prevented issues - see #4523 and possibly also #4528

Terminal output

root@d829437ec7b5:/home/src/python/account# poetry install -vvv
Debug wait
Debug attached
Using virtualenv: /root/.cache/pypoetry/virtualenvs/account-D8L0MJuz-py3.9
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 45 installs, 0 updates, 0 removals, 2 skipped

  • Removing importlib-metadata (4.8.1): Pending...
  • Removing importlib-metadata (4.8.1): Skipped for the following reason: Not currently installed
  • Removing zipp (3.5.1): Pending...
  • Removing zipp (3.5.1): Skipped for the following reason: Not currently installed
  • Installing six (1.16.0): Pending...
  • Installing six (1.16.0): Failed

  RuntimeError

  Unable to find installation candidates for six (1.16.0)

  at ~/.pyenv/versions/3.9.1/lib/python3.9/site-packages/poetry/installation/chooser.py:72 in choose_for
       68│ 
       69│             links.append(link)
       70│ 
       71│         if not links:
    →  72│             raise RuntimeError(
       73│                 "Unable to find installation candidates for {}".format(package)
       74│             )
       75│ 
       76│         # Get the best link

also

❯ poetry update
Updating dependencies
Resolving dependencies... (0.8s)

  PackageNotFound

  Package six (1.16.0) not found.

  at ~/.local/share/pypoetry/venv/lib/python3.9/site-packages/poetry/repositories/pool.py:149 in package
      145│                     self._packages.append(package)
      146│ 
      147│                     return package
      148│ 
    → 149│         raise PackageNotFound("Package {} ({}) not found.".format(name, version))
      150│ 
      151│     def find_packages(
      152│         self, dependency,
      153│     ):

[f0ff886f]

Thank you for finding this out so quickly, we were just in the middle of triaging our CI failing to build anything. I can confirm on Py 3.9 that pinning cachecontrol==0.12.6 helps, however it isn't enough for Py 3.6 and 3.7 (but Py 3.8 and 3.9 were fine), so cachecontrol[filecache]==0.12.6 was enough to fix it across all versions.

Might I humbly suggest that as a tool which promotes stable builds ... that Poetry use some of it's own medicine to install itself, and use a lockfile or output of pip freeze to create a reproducible install for a given version.

💯

[viceice]

Does a temporary downgrade to v1.1.10 fix the issue too?

[mkniewallner]

Does a temporary downgrade to v1.1.10 fix the issue too?

Unfortunately I don't think so, as 1.1.10 also relies on cachecontrol = "^0.12.4" (see code).

I guess that one option to fix the issue (for the short-term) would be to specify an high range for cachecontrol (that only goes up to 0.12.6) or to keep the current range, but skip both 0.12.7 and 0.12.8 (pending a fix coming in 0.12.9 🤞).

[mkniewallner]

Version 0.12.9 of cachecontrol including a fix that should solve the issue has just been released..

[mkniewallner]

FWIW, I've manually triggered builds from Dependabot and Renovate (which both freshly install Poetry either through pip install or through install-poetry.py) on some projects, and both tools were now able to resolve a private dependency they were previously failing to solve, before the release of 0.12.9 of cachecontrol.
So I believe that with freshly installed Poetry which use 0.12.9, it now works as expected.

If Poetry was installed during the issue (so with either 0.12.7 or 0.12.8 of cachecontrol), I believe that either a pip install -U cachecontrol or a reinstallation of Poetry should fix the issue.

[awilkins]

As noted above, cachecontrol==0.12.9 fixes this ; there's also an open discussion #4693 about whether an installer with greater stability (fixed requirements spec) can be provided.

I feel like leaving this issue open for a short while will be useful to people with installs that still have broken versions of cachecontrol as open issues are easier to discover than closed ones, but the issue should be resolved for all new installs of Poetry.

[numine777]

This is broken again.

Poetry 1.1.11 installed via curl script and pip
cachecontrol 0.12.9 and 0.12.10 tested. Attempting to install on Ubuntu 20.04 LTS and in Docker using 18.04 image. Both displaying the same behavior as above. Python version is 3.8.10

[awilkins]

@numine777 Not seeing this problem recurring ; I'm building multi-project python build jobs in CodeBuild that reliably triggered this cache problem, and I'm installing Poetry only via pip (the script essentially makes a special venv and then installs via pip). Do you have any stale caches hanging around? I recommend you clear them if so.

cachecontrol 0.12.10 only seems to amend the package metadata surrounding licenses to make certain license auditing use cases work better, no functional changes that ought to have broken it again.

If you like, maybe try this alternative install script : this is the normal install-poetry.sh script, but with a fixed version of every dependency baked into it. It will only install version 1.1.11 at present.

[b0lle]

Hi,

I am facing exact same issue as described, using cachecontrol 0.12.10 and two private repos. The issue happens on gitlab ci pipeline but on local machine everything works as expected:

$ poetry install --remove-untracked
Creating virtualenv mypackage in /builds/.../.venv
Installing dependencies from lock file
Package operations: 80 installs, 0 updates, 0 removals
  • Installing cachecontrol (0.12.10)
  • Installing foo (1.1.0)
  • Installing bar (1.1.2)
Unable to find installation candidates for bar (1.1.2)

Repository config:

$ echo $(poetry config repositories)
{'foo': {'url': 'https://.../pypi/simple'}, 'bar': {'url': 'https://.../pypi/simple'}}

Poetry version

$ echo $(poetry --version)
Poetry version 1.1.12

Python 3.10

[b0lle]

My issue was related to:
#3802

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.