Should installing pipx shared libs use pip_args from the latest package?

[itsayellow]

Describe the bug

Currently when installing or upgrading the pipx shared libs, the functions _SharedLibs.upgrade() and _SharedLibs.create() accept pip_args arguments.
https://github.com/pipxproject/pipx/blob/08bd15610bc406dabe8bf0db4adaba4a3a8b89ea/src/pipx/shared_libs.py#L33
https://github.com/pipxproject/pipx/blob/08bd15610bc406dabe8bf0db4adaba4a3a8b89ea/src/pipx/shared_libs.py#L61

They are used if a normal pipx install needs a shared library create/upgrade, and the pipx install pip args are used to also create the shared libs.

They are NOT used when creating/upgrading the shared libs during a pipx list or any access of an existing venv. (!!)

Recently this failed for me in the following situation: In a Python 3.6 test I was trying to install a package for the first time using the pip arguments --use-feature=2020-resolver. It caused pipx to fail with an error because before installing the package, it went to install the shared libs using pip argument --use-feature=2020-resolver. Since the original Python 3.6 pip doesn't have that option, it couldn't even upgrade pip in the first place and crashed.

I'm wondering if we should just remove pip_args from _SharedLibs.upgrade() and _SharedLibs.create(). It seems kind of arbitrary that only when upgrading the shared libs during a package install do they get the pip_args that happen to be used for that package, but not otherwise.

Is there a reason a user would need custom pip args to be used with the installation of the shared libs? If so, maybe we need a different more reliable mechanism.

How to reproduce

Expected behavior

[cs01]

In almost all cases it's probably fine to not use pip_args. I can see issues if someone is trying to use a custom or private PyPI url. I suppose the change could be made, and if something breaks people will file issues.

[itsayellow]

Yeah a private index was the only thing I could think of. But I couldn't rationalize why they would want to use a private index for pip, setuptools, etc. unless they had no internet access.

[cs01]

Yeah agreed. I guess in that case they couldn't install in the first place? There are also environment variables for pip to proxy requests, i.e. https_proxy.

[itsayellow]

I'm also kind of assuming that since we inconsistently apply pip_args (sometimes we don't at all) that if were an issue for people it would've come up by now.

[cs01]

#shipit

[itsayellow]

I almost hate to bring this up because I really want to just #shipit, but I'm wondering about the use of pip --ignore-installed in _SharedLibs.create(). It really seems like it should be --force-reinstall instead.

Indeed @cs01 seems to have brought up the issue in the PR that added this code and it was never addressed unfortunately:
#394 (comment)

For reference, here's the stackoverflow article that I seem to have found independently after @cs01:
https://stackoverflow.com/questions/51913361/difference-between-pip-install-options-ignore-installed-and-force-reinstall

I like the clean reinstall on the first-time creation of the shared libs, (even if we think Debian has fixed its problems), but I'm thinking that maybe it should be --force-reinstall.

I did try it on my Ubuntu system with no ill effects, so unless somebody has a problem I'm going to make this change in the PR I'm writing.

[pawamoy]

...aaaaaand I'm here 😎 I have exactly this issue: using a private index, and pypi.org is blocked. It's not that bad, there are alternative solutions (#604).

More context: I'm a bit facing a chicken-and-egg problem. Devs here work on Linux and Windows, so I wrote a Python script (cross-compatible) that will configure their environment so they can use our private index, etc. But then... this script is published on our private index 😑 So how do I install it? By passing the right options to pip to install pipx, and the same options to pipx to install the script. It led me to #604 😄

Conclusion: it's fine. Maybe there are possible improvements to do in pipx regarding pip args. Surely this should simply be documented, in a troubleshooting note for example 🙂 I can open a PR for that!

[itsayellow]

If this was a one time command and you didn't want to modify the env variables persistently, you could do something like:

PIP_INDEX_URL="https://redacted.com/simple/" PIP_CERT="ca-bundle.crt" PIP_TRUSTED_HOST="redacted.com" pipx install myutility

(Or depending on your shell you might have to prefix that with the env command.)

I see the need here for some kind of mechanism. But I can't think of any easier way to set this for pipx, than there is to set the environment variables for pip. So to reduce complexity I think I tend to agree that what is best is to add a note in the docs somewhere on how to alter pip behavior in this case.