Need documentation or fix to use pip configuration with shared libs

[itsayellow]

How would this feature be useful?

Related: #544 #604.

Right now users can use pip_args to install a package from a private index. But there is no pipx-specific argument that directs the installation of the shared libs.

We used to borrow the pip_args from a package install, but this was an inconsistent mechanism, because the shared libs can be upgraded during pipx list and other commands which have no pip_args.

The most reliable way for a user to change such behavior is to set PIP_* environment variables, or to set the pip global config (e.g. see #604 (comment))

Describe the solution you'd like

My preference would be simply to document what is needed to configure pip.

Describe alternatives you've considered

We could add persistent pipx configuration, but this is a complicated solution because pipx doesn't have global persistent configuration at the moment, and it would have to be added. Plus it's not clear this is easier or better than just configuring pip.

We could also have a dedicated option? or command? that would leave a persistent config in the pipx/shared directory, possibly similar or identical to pipx_metadata.json. However this would add complexity and have to be recreated by the user if the shared directory were ever deleted. Again, it's not clear this is better or easier than configuring pip.

[uranusjr]

This is difficult because the user would expect the answer to be yes for some and no for some. There’s an argument to inherit --proxy etc., but for --no-binary, --ignore-installed, etc. the answer is very likely no. So the safe answer is likely no, and provide a separate command to update shared libs with --pip-args applied.

[itsayellow]

This is difficult because the user would expect the answer to be yes for some and no for some. There’s an argument to inherit --proxy etc., but for --no-binary, --ignore-installed, etc. the answer is very likely no. So the safe answer is likely no, and provide a separate command to update shared libs with --pip-args applied.

I'm not following--what's the "yes" or "no" questions you're talking about?

Are you advocating for something like pipx shared --pip-args="..."? That would then leave a pipx_metadata.json in the shared/ venv?

[itsayellow]

My assumption is that the most likely thing config needed to be used with the shared libraries are things like mentioned in #604, like options affecting index, cert, trusted host, proxy. I can't imagine anything else being useful for shared libs. And those things seem pretty broadly applicable to me.

[pawamoy]

Maybe --pip-args should simply be removed in favor or pip environment variables. Or maybe --pip-args could be used by pipx to re-export these variables! At least it would make it work for pipx install.

[Arpafaucon]

woah, I discovered all this issue thread by pure accident (a lucky git-blame) ... when working on a similar issue #1256 .

Me working on this was triggered by a similar issue as #604 - we have a private proxy for packages, and we'd like to use it for all python install

• mostly for network performance reasons
• but also for internal IT security reasons (I don't control those - I guess to limit supply chain attacks ?)

First of all, I 100% agree about the need to document it - it took me several hours of trial and errors to understand what was happening behind the hood. The shared libs feature is an effective internal implementation detail, but it's never documented and, as we've seen, it leaks a bit in the public interface.
My humble take is that we should accept it and let advanced users know about it; I'd suggest

• a first step would be to have part of a doc mentioning the concept, that would have helped me understand pipx behaviour
• provide a pipx upgrade-shared-libs command that can be proactively called with the right pip options, so that advanced users can avoid the surprise background updates - while keeping simpler use cases simple to maintain

I had a go at fixing it by doing the full-reverse operation of what @itsayellow did

• propagating pip_args to shared lib operations
• avoid implicit default arguments by typing : pip_args: List[str] instead of the existing List[str] | None = None
• ... with the general rule of avoiding "default" empty arguments when the user is not able to provide them explicitly
  • pipx list does not do any maintenance task anymore
  • pipx reinstall-all uses the config from the first venv (I don't like this really much but that fixes the use case for me - I expect users will install all their pipx packages with roughly the same network / proxy options).

There is also #1278 incoming that can give advanced users more control

[chrysle]

We now have an upgrade-shared command (#1316) into which pip arguments can be passed. That still leaves the automatic upgrade to handle, but IMO the current fix is good for now, especially with a global --skip-maintenance flag to be introduced.