Poetry formula updates its own virtualenv when using it inside a project

[oliverandrich]

• ran brew update and can still reproduce the problem?
• ran brew doctor, fixed all issues and can still reproduce the problem?
• ran brew gist-logs <formula> (where <formula> is the name of the formula that failed) and included the output link?
• if brew gist-logs didn't work: ran brew config and brew doctor and included their output with your issue?

What you were trying to do (and why)

Create a new python project using the poetry tool from the formula poetry.

What happened (include command output)

The creation of the fresh project works, but calling poetry install does not install packages into the fresh projects, but some packages are updated inside the virtualenv of poetry itself. This is a bug, cause a project should not update the virtualenv of the tool used. A poetry installed via pipx does not behave like that.

Command output

~/tmp

❯ poetry new new-project

Created package new_project in new-project
~/tmp

❯ cd new-project
~/tmp/new-project

❯ poetry install

Creating virtualenv new-project in /Users/oa/tmp/new-project/.venv

Updating dependencies

Resolving dependencies... (0.2s)
Writing lock file
Package operations: 5 installs, 6 updates, 0 removals

Updating more-itertools (8.0.2 /usr/local/Cellar/poetry/1.0.2/libexec/vendor/lib/python3.7/site-packages -> 8.0.2)
Updating zipp (0.6.0 /usr/local/Cellar/poetry/1.0.2/libexec/vendor/lib/python3.7/site-packages -> 0.6.0)
Updating importlib-metadata (1.1.3 /usr/local/Cellar/poetry/1.0.2/libexec/vendor/lib/python3.7/site-packages -> 1.4.0)
Updating pyparsing (2.4.6 /usr/local/Cellar/poetry/1.0.2/libexec/vendor/lib/python3.7/site-packages -> 2.4.6)
Updating six (1.13.0 /usr/local/Cellar/poetry/1.0.2/libexec/vendor/lib/python3.7/site-packages -> 1.13.0)
Updating attrs (19.3.0 /usr/local/Cellar/poetry/1.0.2/libexec/vendor/lib/python3.7/site-packages -> 19.3.0)
Installing packaging (20.0)
Installing pluggy (0.13.1)
Installing py (1.8.1)
Installing wcwidth (0.1.8)
Installing pytest (5.3.2)
Installing new-project (0.1.0)

  

What you expected to happen

What I expect is, that the formula based poetry behaves like a pipx-installed poetry or like a poetry installed with the official installer. Here is an output of the same command sequence. You can easily see the difference on the first 5 or 6 commands. Here the project is isolated and the files are installed inside the actual virtualenv of the project and no updates occur in the virtualenv of the formula.

Command output

~/tmp
❯ poetry new new-project
Created package new_project in new-project
~/tmp

❯ cd new-project
~/tmp/new-project

❯ poetry install

Creating virtualenv new-project in /Users/oa/tmp/new-project/.venv

Updating dependencies

Resolving dependencies... (0.2s)
Writing lock file
Package operations: 11 installs, 0 updates, 0 removals

Installing more-itertools (8.0.2)
Installing zipp (0.6.0)
Installing importlib-metadata (1.4.0)
Installing pyparsing (2.4.6)
Installing six (1.13.0)
Installing attrs (19.3.0)
Installing packaging (20.0)
Installing pluggy (0.13.1)
Installing py (1.8.1)
Installing wcwidth (0.1.8)
Installing pytest (5.3.2)
Installing new-project (0.1.0)

  

Step-by-step reproduction instructions (by running brew install commands)

brew install poetry
poetry new some-new-project
cd some-new-project
poetry install

Output of brew config and brew doctor commands

❯ brew config
HOMEBREW_VERSION: 2.2.2-80-g4bdce47
ORIGIN: https://github.com/Homebrew/brew
HEAD: 4bdce4701fa4490a415c04fd8d0030cde24fbe44
Last commit: 27 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 3dfe6ba274492a06a2116effef37353a800e5052
Core tap last commit: 6 hours ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_BREW_GIT_REMOTE: https://github.com/Homebrew/brew
HOMEBREW_CORE_GIT_REMOTE: https://github.com/Homebrew/homebrew-core
HOMEBREW_DEV_CMD_RUN: 1
CPU: dodeca-core 64-bit kabylake
Homebrew Ruby: 2.6.3 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
Clang: 11.0 build 1100
Git: 2.21.0 => /Library/Developer/CommandLineTools/usr/bin/git
Curl: 7.64.1 => /usr/bin/curl
macOS: 10.15.2-x86_64
CLT: 11.3.0.0.1.1574140115
Xcode: 11.3

~/tmp
❯ brew doctor
Your system is ready to brew.

[bayandin]

It seems poetry doesn't like that we set PYTHONPATH for poetry executable:

homebrew-core/Formula/poetry.rb

Lines 169 to 182 in 8cb3662

	xy = Language::Python.major_minor_version "python3"
	ENV.prepend_create_path "PYTHONPATH", libexec/"vendor/lib/python#{xy}/site-packages"
	resources.each do |r|
	r.stage do
	system "python3", *Language::Python.setup_install_args(libexec/"vendor")
	end
	end
	ENV.prepend_create_path "PYTHONPATH", libexec/"lib/python#{xy}/site-packages"
	system "python3", *Language::Python.setup_install_args(libexec)
	bin.install libexec/"bin/poetry"
	bin.env_script_all_files(libexec/"bin", :PYTHONPATH => ENV["PYTHONPATH"])

I've tried to change the installation to using virtualenv_install_with_resources but it didn't work since one of the resources (clikit) requires poetry itself

[oliverandrich]

I think something like this is necessary for some modules. This is taken from the flake8 formula. I tried this myself, but I didn't succeed with it. So, this is no easy solution but just a hint.

def install
    venv = virtualenv_create(libexec, "python3")
    resource("entrypoints").stage do
      # Without removing this file, `pip` will ignore the `setup.py` file and
      # attempt to download the [`flit`](https://github.com/takluyver/flit)
      # build system.
      rm_f "pyproject.toml"
      venv.pip_install Pathname.pwd
    end
    (resources.map(&:name).to_set - ["entrypoints"]).each do |r|
      venv.pip_install resource(r)
    end
    venv.pip_install_and_link buildpath
  end

[AlJohri]

@oliverandrich I think pipx ran into the same issue: pypa/pipx#233
And hacked around it by explicitly nulling the PYTHONPATH in child processes when invoked with subprocess.run: https://github.com/pipxproject/pipx/pull/252/files#diff-42b69df5d9f71de41d5bc8cc22a7a5a8

we didn't go with the virtualenv_install_with_resources because pipx for some reason just doesn't work properly when packaged in side a virtualenv (as its a tool designed to create new virtualenvs)

[oliverandrich]

Sounds reasonable. May it would be a good idea to remove the formula for the time being. At least one of my team members corrupted a project.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.