Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Providing pip configuration in sys.base_prefix #9752

Open
1 task done
pelson opened this issue Apr 1, 2021 · 40 comments
Open
1 task done

Providing pip configuration in sys.base_prefix #9752

pelson opened this issue Apr 1, 2021 · 40 comments
Labels
C: configuration Configuration management and loading state: awaiting PR Feature discussed, PR is needed type: feature request Request for a new feature

Comments

@pelson
Copy link
Contributor

pelson commented Apr 1, 2021

pip version

main

Python version

all

OS

all

Additional information

When a pip config is installed in an installation prefix $PREFIX/pip.conf, $PREFIX/bin/python -m pip correctly picks up the config. When one makes a virtual environment with $PREFIX/bin/python -m venv ./my-venv then ./my-venv/bin/pip does not pick up the config.

It is questionable if this is a bug or a feature request, but essentially, I believe that pip should be looking in sys.base_prefix as well as sys.prefix for a config file.

Description

No response

Expected behavior

No response

How to Reproduce

Starting with a non-virtual environment (e.g. a conda environment):

$ touch ./env/pip.conf
$ pip config debug
env_var:
env:
global:
  /etc/xdg/xdg-ubuntu/pip/pip.conf, exists: False
  /etc/xdg/pip/pip.conf, exists: False
  /etc/pip.conf, exists: False
site:
  /media/important/github/pypa/pip/env/pip.conf, exists: True
user:
  /home/pelson/.pip/pip.conf, exists: False
  /home/pelson/.config/pip/pip.conf, exists: False

$ python -m venv ./venv

## BEWARE THAT YOUR VENV HAS THE BUNDLED PIP, SO INSTALL A NEWER PIP FOR DEBUGGING
$ ./venv/bin/pip install -e /path/to/pip/repo

$ ./venv/bin/pip config debug
env_var:
env:
global:
  /etc/xdg/xdg-ubuntu/pip/pip.conf, exists: False
  /etc/xdg/pip/pip.conf, exists: False
  /etc/pip.conf, exists: False
site:
  /media/important/github/pypa/pip/venv/pip.conf, exists: False
user:
  /home/pelson/.pip/pip.conf, exists: False
  /home/pelson/.config/pip/pip.conf, exists: False

What I want to see for a venv is that the base_prefix is searched as well as the prefix.

Output

No response

Code of Conduct

  • I agree to follow the PSF Code of Conduct
@pelson pelson added S: needs triage Issues/PRs that need to be triaged type: bug A confirmed bug or unintended behavior labels Apr 1, 2021
@pradyunsg
Copy link
Member

Hi! Thanks for filing this issue and PR!

Could you please elaborate on why you want this? What use case does this help with?

@pradyunsg pradyunsg added S: awaiting response Waiting for a response/more information and removed S: needs triage Issues/PRs that need to be triaged labels Apr 1, 2021
@uranusjr
Copy link
Member

uranusjr commented Apr 1, 2021

I think this is by design? A virtual environment is supposed to isolated things from the global environment, so configuration from sys.base_prefix should not be picked up.

@pelson
Copy link
Contributor Author

pelson commented Apr 1, 2021

Could you please elaborate on why you want this? What use case does this help with?

Sure, I'll try with my specific case, and then hopefully we can draw parallels elsewhere.

I an the responsible for a Python distribution to a controlled network. There are other Python distributions on the controlled network (both accessible from the same machine) for which I am not responsible, and which need specific pip configurations. I too have a specific pip configuration (e.g. index-url), I therefore configure them in the pip.conf of the distribution. It is designed that users (developers) take virtual environments from this distribution in order to extend it as needed - pip is a fundamental part of that work flow to allow them to extend as required. I therefore want those users to have the appropriate pip configuration within their virtual environments (out of the box).

To re-iterate, it is not acceptable to have the config be global (e.g. in /etc/pip.conf) nor user enabled (e.g. in ~/pip.conf) as they need to also be able to use another Python distribution with different pip configurations (i.e. the pip config needs to be isolated to the environment).

Environment variables defining the config aren't acceptable because the two Python distributions wouldn't be able to inter-operate (you can only have one PIP_INDEX_URL.

To me the idea that a virtual environment should inherit the pip configuration from the base environment is natural. After all, a virtual environment inherits the Python and its standard library from the base environment - if you change the config of how Python is built such config propagates/leaks to the virtual environment.

@no-response no-response bot removed the S: awaiting response Waiting for a response/more information label Apr 1, 2021
@pfmoore
Copy link
Member

pfmoore commented Apr 1, 2021

IMO, the only sort of environment that should inherit is a --use-system-site-packages one, as that's explicitly not isolated from the system environment. Certainly changing the default behaviour to inherit would be a significant backward incompatibility, and would likely cause problems for people who rely on global settings not leaking into a virtualenv (e.g., testing).

@pelson
Copy link
Contributor Author

pelson commented Apr 1, 2021

Thank you for your input so far.

would likely cause problems for people who rely on global settings not leaking into a virtualenv (e.g., testing)

We should try to be more precise with the terminology here. global settings do leak into a virtualenv:

$ cat /etc/pip.conf 
[global]
no-cache-dir = true

$ cat ./venv/pip.conf 
[global]
no-dependencies = yes

$ pip config list
global.no-cache-dir='true'
global.no-dependencies='yes'

What I'm suggesting is, just like Python config (e.g. compile flags) leaks into a virtual environment, so too should the pip config. i.e. site configuration should leak into a virtual env.

To turn this discussion around a little, perhaps I could ask you how you would control a virtual environment's pip configuration if global, user, and environment settings are not acceptable (because (a) a pip config doesn't apply to all Python installations on a machine, (b) it isn't a user configuration to define how a Python installation should interact with the distribution)?

I'm trying to not be hypothetical here, so my concrete scenario:

  • 1 machine
  • No internet, but internal repositories exist
  • 2 completely different Python distributions
  • Each python distribution has different pip configuration requirements (e.g. index urls)
  • Users use virtual environments to give them extensibility

The problem occurs fairly quickly as users take a virtual environment from one of the distributions and they immediately have no pip configuration. Even worse, they try to use a PEP517 tool such as build to create a wheel python -m build ./ using the correctly (pip) configured Python distribution, but because the build isolation creates a new virtual environment the pip isn't configured and you get an error.

@piotr-dobrogost
Copy link

piotr-dobrogost commented Apr 5, 2021

@pelson
I have to admit the scenario you brought up is interesting. :)

To turn this discussion around a little, perhaps I could ask you how you would control a virtual environment's pip configuration if global, user, and environment settings are not acceptable (because (a) a pip config doesn't apply to all Python installations on a machine, (b) it isn't a user configuration to define how a Python installation should interact with the distribution)?

After taking a quick look at docs of venv it looks like you could extend EnvBuilder class and define post_setup() method.

From https://docs.python.org/3/library/venv.html#venv.EnvBuilder.post_setup

post_setup(context)
A placeholder method which can be overridden in third party implementations to pre-install packages in the virtual environment or perform other post-creation steps. (emphasis mine)

The post-creation step in your case could be creation of appropriate pip configuration file in the newly created virtual environment.

@gaborbernat
Copy link

After taking a quick look at docs of venv it looks like you could extend EnvBuilder class and define post_setup() method.

That's my recommendation for using build too, to extend virtualenv via its plugins- pypa/build#270 (comment).

@pelson
Copy link
Contributor Author

pelson commented Apr 11, 2021

Thanks for the suggestion of extending EnvBuilder. Indeed this does work to automatically configure a venv with pip.conf, and I've been running with such a tweak for quite some time. Unfortunately the python -m venv and python -m build invocations aren't something that you can override in this way (without monkeypatching during Python startup 🙊).

Perhaps I could have been a bit clearer here: I'm not looking for a workaround per se (I have one, though I do appreciate ideas of other potential workaround solutions), I'm looking to address what I believe is a genuine shortfall (i.e. unexpected behaviour) in the way pip behaves with virtual environments.

That's my recommendation for using build too, to extend virtualenv via its plugins

I'm not using virtualenv, I'm using venv, the standard library way to make virtual environments. So too does build (https://github.com/pypa/build/blob/0.3.1/src/build/env.py#L188), no? There is no equivalent plugin system for venv.

the only sort of environment that should inherit is a --use-system-site-packages one

I disagree: I am not proposing that you inherit a single extra package with this change, merely that a locally (site) scoped pip.conf from a parent environment should be inherited in the child (virtual) one. I think considering configuration inheritance as the same as package inheritance (system site packages) would be a conflation of the --use-system-site-packages flag.

I'm trying to figure out real-world use cases where you wouldn't want to inherit the pip.conf from the parent environment. The one provided by @pfmoore "isolation testing" is a reasonable one, but I don't believe it should necessarily trump the one in which a user wants to create a virtual environment on a Python distribution which has been correctly pip configured and expect that their virtual env pip to "just work™".

In my PR (#9753) I implemented the easier approach of blending together the pip.conf in sys.base_prefix and sys.prefix (just like all of the other config search path items). It would totally solve my use case if, instead of doing a blend, it simply didn't look in sys.base_prefix if sys.prefix / pip.conf existed. This would solve the "isolated testing" usecase, as you would simply touch sys.prefix / pip.conf to avoid sys.base_prefix / pip.conf being considered. This may be best of both worlds - you can easily enough have isolated testing, but still get working behaviour out of the box.

@uranusjr
Copy link
Member

build uses virtualenv if it’s available, and falls back to venv when it’s not.

@pfmoore
Copy link
Member

pfmoore commented Apr 11, 2021

"isolation testing" is a reasonable one, but I don't believe it should necessarily trump the one in which a user wants to create a virtual environment on a Python distribution which has been correctly pip configured and expect that their virtual env pip to "just work™".

Also, isolation is important from the point of view of bug triage - "can you reproduce your issue in an empyth virtualenv" is much easier than "can you reproduce your issue if you create a new virtualenv, hunt down and disable any config files you may have created in the past and forgotten about, ..."

But the real problem is that both are valid requirements. As a result, we're not discussing what the behaviour should be, but rather which behaviour should be default, and which should be opt-in.

Backward compatibility is a significant weight in favour of ignoring the site config by default.

IMO you've made some good arguments that your use case is worth considering, but your arguments are not strong enough to switch the default. If you want to continue arguing for your behaviour being the default, I suggest you focus on how to address the backward compatibility issue.

@layday
Copy link
Member

layday commented Apr 11, 2021

I don't understand how #9753 is going to help with build inheriting a parent venv's configuration. When you create a venv from another venv, sys.base_prefix does not point to the parent venv's prefix; it always points to the installed Python's prefix. pip does not look inside sys.base_prefix I assume because (a) they would not want to encourage users to drop pip configuration directly in e.g. /usr and (b) because, in most cases, global configuration serves the same purpose.

@pelson
Copy link
Contributor Author

pelson commented Apr 12, 2021

Also, isolation is important from the point of view of bug triage - "can you reproduce your issue in an empyth virtualenv" is much easier than "can you reproduce your issue if you create a new virtualenv, hunt down and disable any config files you may have created in the past and forgotten about, ..."

I'm only semi-convinced about this one. You still have to deal with peoples environment variables, global config and user config. In reality you'd tell them to run pip config debug and, if it existed, perhaps tell users to PIP_DISABLE_CONFIG=1 for a truly clean config-free reproducer.

IMO you've made some good arguments that your use case is worth considering, but your arguments are not strong enough to switch the default.

Thank you for your insight and pointing out where I should focus the discussion.

But the real problem is that both are valid requirements. As a result, we're not discussing what the behaviour should be, but rather which behaviour should be default, and which should be opt-in.

If you want to continue arguing for your behaviour being the default, I suggest you focus on how to address the backward compatibility issue.

My proposal to read sys.base_prefix / pip.conf IFF sys.prefix doesn't exist is the easiest way to enable both behaviours conveniently, but it is indeed backwards incompatible.

The problem is that the two use cases are in conflict - so we need to be able to configure which behaviour we want. But in order to configure this using non-global config, we need to read the base_prefix's config...

Backward compatibility is a significant weight in favour of ignoring the site config by default.

The previous statement I made is the strongest I have in favour of changing the default. If we change the default, it is easy to then make an isolated virtual environment (touch sys.prefix / pip.conf) if you need it. The existing pip config debug is clear and continues to apply. If we don't change the default then it requires some more complexity in pip to decide if it should consider sys.base_prefix / pip.conf.

So let's take a look at what that might actually entail if we don't change the default:

We need some non-global, non-venv and non-envvar means to tell pip to blend sys.base_prefix / pip.conf with sys.prefix / pip.conf. So we could perhaps look at sys.base_prefix / pip.conf and look for a config item called, say use-base-pip-conf: true (default to false). If it is set, we include the rest of sys.base_prefix / pip.conf when building the pip configuration.

It isn't the nicest behaviour because we need to pre-read the config to figure if we want to read the rest of the config. pip config debug needs to become more nuanced ("the base config file exists, but we don't use it because it is not enabled"). This is the cost of keeping the "isolated by default" behaviour though. I can imagine this feature leading to a bit of confusion, if I'm honest.

Do you have different ideas about how we might be able to satisfy the two use cases without changing the default? Fundamentally I'm struggling because the "isolation" behaviour a subset of the "non-isolation" one (i.e. you can configure the "non-isolated by default" one to be isolated, but you can't configure the "isolated by default" one to be non-isolated).

@pradyunsg pradyunsg added state: needs discussion This needs some more discussion type: feature request Request for a new feature and removed type: bug A confirmed bug or unintended behavior labels Apr 12, 2021
@layday
Copy link
Member

layday commented Apr 12, 2021

If we can step away from discussing the implementation for a minute, what's not been made clear through the course of this conversation is that this change would only be applicable to Conda environments, because Conda apparently mangles the value of sys.base_prefix of a Python installed in a Conda environment. This means that you can keep a development environment-local pip configuration in the "global" Python prefix, a peculiarity only found in Conda environments.

@piotr-dobrogost
Copy link

piotr-dobrogost commented Apr 12, 2021

@layday
I'm not familiar with Conda but the problem seems perfectly valid in the realm of the standard Python. If I understand correctly the original issue was "Isolated venv does not inherit PIP index " (pypa/build#270). This has nothing to do with Conda. Actually I am surprised this issue has not come up way earlier. Just think of all these corporate environments when one is required to use specific pip index. It is natural to set this pip index as part of the base Python installation (often on a machine shared by many users) and hope it would propagate to virtual environments based on the base installation.
I am in favour of reevaluating what the isolation of virtual environment is supposed to mean. I guess the original intend was to isolate virtual environment from packages installed in the base installation. We should consider if isolation from the base installation's configuration was kind of an accident making more harm than good.

@layday
Copy link
Member

layday commented Apr 13, 2021

I'd question whether this is a valid configuration vector outside of a Conda environment. The intention with loading configuration from {sys.prefix}/pip.conf was to allow configuring pip in a virtual environment (and extended to support Conda environments in #6268), i.e. from a development-local config file. Corporate entities can configure pip globally instead of on a Python prefix basis - sys.base_prefix can be shared by but it can also vary (!) for multiple Python installations on the same system. What this means in practice is that you can ensure neither that a pip config file located in sys.base_prefix applies to only one specific Python installation nor that it is global.

@pelson
Copy link
Contributor Author

pelson commented Apr 13, 2021

what's not been made clear through the course of this conversation is that this change would only be applicable to Conda environments

I don't think this conversation has anything to do with conda. Here is a clean build of Python:

$ wget https://www.python.org/ftp/python/3.9.4/Python-3.9.4.tgz
$ tar xzf Python-3.9.4.tgz 
$ ./Python-3.9.4/configure --prefix $(pwd)/py39
$ make
$ make install

With this:

$ ./py39/bin/python3 -m venv ./my-venv

$ ./py39/bin/python3
>>> import sys
>>> sys.base_prefix
'/home/pelson/Downloads/cpython_clean/py39'
>>> sys.prefix
'/home/pelson/Downloads/cpython_clean/py39'

$ ./my-venv/bin/python
>>> import sys
>>> sys.base_prefix
'/home/pelson/Downloads/cpython_clean/py39'
>>> sys.prefix
'/home/pelson/Downloads/cpython_clean/my-venv'

And the pip config:

$ touch ./py39/pip.conf
$ ./py39/bin/pip3 config debug
env_var:
env:
global:
  /etc/xdg/xdg-ubuntu/pip/pip.conf, exists: False
  /etc/xdg/pip/pip.conf, exists: False
  /etc/pip.conf, exists: True
    global.no-cache-dir: true
site:
  /home/pelson/Downloads/cpython_clean/py39/pip.conf, exists: True
user:
  /home/pelson/.pip/pip.conf, exists: False
  /home/pelson/.config/pip/pip.conf, exists: False

$ ./my-venv/bin/pip config debug
env_var:
env:
global:
  /etc/xdg/xdg-ubuntu/pip/pip.conf, exists: False
  /etc/xdg/pip/pip.conf, exists: False
  /etc/pip.conf, exists: True
    global.no-cache-dir: true
site:
  /home/pelson/Downloads/cpython_clean/my-venv/pip.conf, exists: False
user:
  /home/pelson/.pip/pip.conf, exists: False
  /home/pelson/.config/pip/pip.conf, exists: False

To "step away from the implementation", as you say, I want the pip config for the base environment to apply to the virtual environment one. sys.base_prefix is the documented way to identify "base environment" (https://docs.python.org/3/library/sys.html#sys.base_prefix).

Perhaps I missed something in what you were saying though.


To keep the conversation on track, the 3 proposals that have so far been discussed:

  1. pip always looks at sys.base_prefix/pip.conf and blends it together with sys.prefix/pip.conf if they exist
  2. pip looks at sys.base_prefix if-and-only-if sys.prefix/pip.conf doesn't exist
  3. pip looks at sys.base_prefix/pip.conf to determine if it should consider the rest of sys.base_prefix/pip.conf to be blended with sys.prefix/pip.conf. By default this would be false so the current default behaviour would remain.

My preference is option 2 as it represents a far simpler implementation and is far less confusing to explain, and therefore less likely to generate support requests to pip. Unfortunately this is a breaking change as pointed out by @pfmoore. To get back to the old behaviour, one just needs to touch sys.prefix/pip.conf (or remove the sys.base_prefix/pip.conf!).

As far as I can see right now there is no other way for us to use non-global, non-user, non-environmental means to allow a venv to be configured out of the box - I have an open question to hear of other viable implementation options. To re-iterate, for the use case it is essential that a venv can be configured out of the box (e.g. isolated wheel building), it is completely acceptable that the base environment needs to be configured to do this (i.e. it isn't the default behaviour) but is not OK for it to be a post-venv creation step as this would preclude the use of venv using tools.

@layday
Copy link
Member

layday commented Apr 13, 2021 via email

@piotr-dobrogost
Copy link

piotr-dobrogost commented Apr 13, 2021

It is relevant to Conda because only with Conda will you have a different base prefix for every development environment.

No one is talking about different base prefix for every environment. The point is to have all virtual environments based on one, specific base Python installation to inherit pip configuration from this one, specific base Python installation. Also no one besides you mentioned Conda in any way.

Corporate entities can configure pip globally instead of on a Python prefix basis - sys.base_prefix can be shared by but it can also vary (!) for multiple Python installations on the same system.

That's exactly what OP stated:

There are other Python distributions on the controlled network (both accessible from the same machine) for which I am not responsible, and which need specific pip configurations.

As to:

What this means in practice is that you can ensure neither that a pip config file located in sys.base_prefix applies to only one specific Python installation nor that it is global.

Exactly the opposite is true. sys.base_prefix is unique per Python installation thus taking its pip's configuration into consideration in virtual environments allows all such environments to share common configuration tailored to the needs of this specific base Python installation.

@pfmoore
Copy link
Member

pfmoore commented Apr 13, 2021

To keep the conversation on track, the 3 proposals that have so far been discussed

... and to provide context, pip's current behaviour

  • pip only looks at sys.prefix and never looks at sys.base_prefix.

This boils down to, pip looks at the currently active environment, the site configuration, and the user configuration. This is also simple to explain.

Do you have any examples of other software that looks up configuration from both sys.prefix and sys.base_prefix? I'm not aware of any.

🤷 I guess what it boils down to is that for me, the change is more difficult to describe than the current behaviour, it's not backward compatible, and it doesn't seem useful enough to justify the maintenance cost (without trying to be dismissive, your situation is clearly a fairly rare special case).

@layday
Copy link
Member

layday commented Apr 13, 2021 via email

@pfmoore
Copy link
Member

pfmoore commented Sep 7, 2022

I currently represent 400+ scientific and engineering Python users, and have quite extensive experience of deploying Python to large scientific organisations who regularly have network isolated (internet-free) machines with internal package indexes (HPCs, critical op infrastructure, etc.). I'm not saying I'm correct or my opinion is worth more as a result, but I'm genuinely doing this in the interest of a significant user population who you would otherwise never see as they are behind an organisational firewall.

I really appreciate the fact that you've taken the time on this issue to represent that user base. As you say, it's extremely hard for us to find out about such use cases and the difficulties they face, and the input is really important to us. Please don't take the pushback you're receiving on this issue as a lack of interest in your use case. In particular, one reason I'm personally so involved in this issue is because I want to make sure we don't forget about these "hidden" use cases (I don't otherwise have much opinion on pip's config mechanisms 😉).

One question I asked above, which hasn't yet been answered:

Do you have any examples of other software that looks up configuration from both sys.prefix and sys.base_prefix? I'm not aware of any.

This is a key point - pip shouldn't be breaking new ground here, the problem our config system is addressing isn't unique to pip, so we shouldn't be implementing unique solutions. So I would expect you'd have similar problems with other software, and if you don't, I'd want pip to do things similarly to those tools, for consistency across the ecosystem if nothing else.

The PR I produced offered concrete evidence (#9753).

I apologise. I had somehow missed that you had provided this PR. It does indeed look OK to me, in terms of clearly explaining what you're proposing and demonstrating that the change is relatively straightforward at a technical level.

I personally think the complexity is low (please feel free to leave comments on the complexity in the PR if you disagree).

The amount of discussion and differing opinions here demonstrates (to me, at least) that this isn't the case. It may be simple in implementation terms, and maybe even in terms of explaining the technical implications ("you can now pick up config from sys.base_prefix as well") but in user terms, the fact that we're having this amount of discussion, suggests that calling the complexity "low" is missing a key point.

Regarding the chance of moving this forward, I still think it's mostly down to you. There have been a number of key questions/issues raised here. Maybe you feel that you have addressed them, but I think the lack of progress is essentially because the rest of the participants wouldn't agree...

You've said that the plugin mechanism for virtualenv solves the issue for that case, your problem is really only with the stdlib venv. That in itself limits the benefit of this change. And I'm not clear why, given that you control the Python installation, you don't patch the stdlib venv to fix the problem there. So what about this issue necessitates that it gets solved in pip? Is it just that you think the solution "fits better" in pip? If so, you haven't really made your point on that yet.

We currently have the following config locations in pip:

  • global: system-wide configuration file, shared across users.
  • user: per-user configuration file.
  • site: per-environment configuration file; i.e. per-virtualenv.

You're proposing changing "site" to mean something very different (per-virtualenv plus inherited config from the base environment, I think). Could you explain how you'd re-word the description of "site" (without making it significantly longer than the other two!) to say what you intend it to now mean?

In my view, what you're after sounds more like you want to change "global". If so, that's something we inherit from platformdirs (where it's confusingly called "site" 🙁), so you should be proposing that change with them. I don't think we'd want to add a special case just for pip here.

If you want to add an extra level, then why? You've never (as far as I can see) said what precisely you want to put in this "shared base environment config". I can't think of anything that I'd want to set that would depend on which Python installation I used to create my virtual environment. If I have both your shared environment on my PC, plus a locally installed Python that's the same version, why would it matter which one I used to create a virtualenv? That's actually the biggest fundamental question here - what is the actual problem you're trying to solve here? Not the solution or approach you'd like to take, but what (in end-user terms) is the underlying problem?

I hope this helps. Sorry I can't give you a simple "yes, this seems like a good addition" or "no, sorry, we're not going to accept this", but I really don't think you've given us enough information about your requirement to let us make such a decision yet. I know you've tried, but hopefully the above helps you understand where there's still gaps in our understanding.

@pelson
Copy link
Contributor Author

pelson commented Sep 7, 2022

Maybe you feel that you have addressed them, but I think the lack of progress is essentially because the rest of the participants wouldn't agree...

Thank you for being explicit. This is also my feeling, and I appreciate having a concrete list of those things to focus on.

Unfortunately addressing those things is going to result in a long message - so apologies in advance. I will keep it as short as I can whilst addressing all that you have put to me.

You've said that the plugin mechanism for virtualenv solves the issue for that case, your problem is really only with the stdlib venv

In fact, right now I support venv (by patching it), but not virtualenv. Whilst virtualenv has hook support, I don't think it is any more easy to do by default than venv - the functionality to change the default seeder without explicit opt-in doesn't (yet) exist.

This isn't really about the tool though - I ideally want to support any tool that can produce virtual environments and which follows the standard sys.base_prefix and sys.prefix concepts. I'm advocating that pip knows how to handle virtual environments, not that the tools know how to make virtual environments with pip appropriately configured. The difference is subtle, but I would like to be able to create a virtual environment without pip, run python -m ensurepip in the venv and have a suitably configured pip as per the base, for example.

Is it just that you think the solution "fits better" in pip? If so, you haven't really made your point on that yet.

To be explicit: Yes exactly. That, and the fact that the tool used to create the virtual env isn't the important part - there are multiple valid ways to produce a virtual environment (a standard library concept), and it shouldn't be necessary to patch all tools which create them in order to get a working pip.

Do you have any examples of other software that looks up configuration from both sys.prefix and sys.base_prefix? I'm not aware of any.

pip shouldn't be breaking new ground here, the problem our config system is addressing isn't unique to pip, so we shouldn't be implementing unique solutions

I empathise with this point, but I disagree - IMO pip is special in this regard. pip is a fundamental part of an isolated virtual environment's subsequent population. There is no other library or tool which fundamentally needs to be appropriately configured immediately upon venv creation in order to allow tools to subsequently extend the newly created venv. To put this another way: for every other tool it is possible to manually configure a tool before running it (even if not ideal), but pip is so fundamental that tools (such as build, pre-commit) bake into their design the idea of "create venv + install packages" as a single step - there is no opportunity for users to configure pip manually in-between.

Finding something that does config like this is going to be tough as a result. Perhaps the most similar behaviour, whilst not using virtual environments precisely, is found in conda. Their config search path follows:

  • CONDARC env var
  • Currently active environment
  • $HOME/
  • Root environment (akin to base)
  • System (/var/lib/, /etc/conda, C:/ProgramData/conda)

But finding other projects that read their config from sys.prefix, never mind sys.base_prefix, is challenging. Often it is based on $XDG_HOME, or /etc.

You're proposing changing "site" to mean something very different

I accept this directly. Both platformdirs and site.py (std-lib) have similar naming issues, IMO. The term "site" to mean virtual environment doesn't make much sense to me, since you tend to provide a "site-wide" (base) Python installation, rather than a site-wide virtual environment - but I am happy to fall into line on the terminology 👍.

Perhaps, given the conflation in many places, it is best to move away from the term "site" altogether. Python environment would be good enough to convey that it is coming from sys.prefix. I leave that to another discussion (on another issue) though. For now site is good enough to mean current Python environment.

In my view, what you're after sounds more like you want to change "global".

I'm explicitly not after this - I do not want to influence the pip.config of all Python invocations on a machine. I only want to influence those that use the Python installation I provide on a network drive - two reasons: the Python invocations aren't all happening on the same machine, and I don't want to touch what happens to other Python distributions running on that machine (platform managed Python, or other Python installations that may exist).

This latter point you've probed specifically about, so will talk about later on.

If you want to add an extra level, then why?

Given the naming confusion around site, I agree that I am fundamentally requesting a new level.

I think the motivation is coming through in my points above, but I believe that venv specific config should be relatively rare. I will try to highlight what I see as the advantages of each:

  • Environment variable: Good for changing something quickly without writing a file
  • Global: Multiple users on the same machine (in /etc) have pip configured 👍
  • User: Same user on multiple machines (if homespace network mounted) have pip configured 👍
    Also good if you want to change a global setting, but don't have the permission to change global settings.
  • Site (venv): Useful for testing pip.config and avoiding unwanted common config. Especially good if you have system python (owned by root) which isn't correctly pip configured, and want others to be able to be able to use pip from your venv (this sound fairly contrived).

The new level that I would put in between Global and Site:

  • Base: Multiple users on multiple machines (if prefix network mounted) have pip configured 👍

(FWIW, I would have put Environment variable last, proceeded by User, as that would have been more consistent with PYTHONPATH, and would maximise the utility of the benefits I highlighted - that is a different discussion though 😉)

If I have both your shared environment on my PC, plus a locally installed Python that's the same version, why would it matter which one I used to create a virtualenv?

My shared environment doesn't support your arbitrary locally installed Python version. If you have problems with your Python version with my pip.conf, I am going to tell you to use my shared Python environment to see if you can reproduce it there. Concretely, my shared environment patches Python in certain ways, including overriding ensurepip - this ensures that when a new venv is created a reasonably up-to-date pip is installed out of the box. This was essential at a certain point in order to pick up the site config (which I think was added around pip 9), but now is a convenience and ensures that people are using the improved resolver and benefiting from all of the other improvements that come with modern pip.

I think your question is essentially asking: What is the use case for having multiple pip.config files on the same machine for the same Python version.

Hypothetically I could imaging wanting to differentiate the package index to use. For example, one could timestamp the index being used based on the Python distribution release date, in order to provide reproducible and snapshotted pip install calls - I don't do this, but it is something that my sys-admins and security team have requested, and a strategy that is employed for Linux distributions w. RPMs.

Perhaps a list of reasons that I can't use the existing config locations would be useful:

  • Global: I don't control the machines upon which the Python is mounted. I can't touch /etc/. In most cases, nor can the user (it is managed by sys admins).
  • User: It isn't my user running Python, it is others - not desirable to have each user configure their pip manually.
  • Site: The virtual environments are created by the user. I use this location currently by patching venv creation.
  • Env var: There is no activation step - the Python distribution works without activation, and not desirable to request each user set environment variables.

but what (in end-user terms) is the underlying problem?

Simply put, as a user:

  • I don't have permission to change my global config.
  • I use multiple Python distributions. Some in a restricted network zone.
  • My homespace is mounted on all machines, no matter the zone.
  • I don't want to deal with pip.config - I want to let the experts set it up for me so that when I am in the restricted zone, it uses a specific package index.
  • When I'm not in a restricted zone, I can't use the restricted package config (e.g. no config, or one managed by another Python environment, is needed)
  • There are multple Python distributions in the restricted zone - not all of them should be configured (mostly because they aren't supported). The security team insists that we only configure the Python distributions that are being maintained.

If you are partially convinced, I can revive the PR to see what it actually entails to implement the extra level being discussed. If not, I think I've probably over-spent on getting a return on investment already 😄. Either way, I appreciate that this is also very time consuming for you, and want to say thank you for taking the time to engage.

@pfmoore
Copy link
Member

pfmoore commented Sep 7, 2022

If you are partially convinced

I'm not convinced that your proposed solution is the right one, but I do see that you have a problem that isn't easily solved by existing pip functionality. But equally, I don't want to let you think that convincing me is all that's involved here. I'd like to see you get some feedback from other pip developers if we're to do anything with this. And assuming the other devs are receptive, I'd be wanting to explore options that don't have the problems that this approach has (which have been discussed at length above).

I hope that helps - I'll let you decide whether you want to pursue this further.

I might follow up with another response addressing some of your particular points. But please treat that as simply exploring the problem a little further - there's no need for you to respond to it.

@pradyunsg pradyunsg changed the title pip config of base not picked up in venv Providing pip configuration in sys.base_prefix Sep 7, 2022
@pradyunsg pradyunsg added the C: configuration Configuration management and loading label Sep 7, 2022
@pradyunsg
Copy link
Member

pradyunsg commented Sep 7, 2022

I've retitled this issue, to better reflect what is being asked for.

As useful context/background, I wrote pip config and wrote/refactored most of the configuration logic a few years ago (stating this to establish I'm reasonably familiar with it, and not to imply that I somehow "own" this piece within pip or have any sort of authority over this beyond what my fellow maintainers do).

I think the rationale provided here for adding a new level is reasonable, and I'm fine with adding an additional config file at base_prefix.

I will say this: this is a weird organisational situation you're in, where you can't/don't want to influence the global configuration (i.e. it's fine to have pip reach out to whatever is configured there, or the default PyPI) and you don't trust/want your users to configure their tools to work correctly (i.e. by documenting/educating about a standard config file for the use case). I'm not sure what pieces are specific choices by you vs being forced upon you. :)

Coming back: I don't think an additional config file layer is particularly that expensive for us in terms of the code + maintenance, and it feels like the right level of abstraction to solve this particular class of organisational/usage-pattern problems at -- the main concern I have is around (a) documenting this clearly+completely, and (b) a clean migration to this. Since it's effectively a new feature, (b) is less of a concern. (a) is something that will end up needing to be a part of the PR adding this. I'll guess that this is something that's mostly useful for organisational users, where there's some shared configuration for Python development that only applies to a subset of users (likely the package index?).

I'd be happy to review and (assuming no other maintainers are strongly opposed) merge a PR for this.

To err on the side of overcommunicating.... I'll explicitly state that I'm not promising that I'd take a look at such a PR quickly though -- trying to juggle quite a few things at the moment. I also haven't been looking at this thread closely due to limited bandwidth on my end. I've quickly skimmed through the initial bits of this thread and the most recent comments (sorry, there's way too much text here and I have limited mental energy right now) and that's the context I had when I wrote this. Sorry if I've missed some nuances/points that have been brought up already. 🙈

@pradyunsg pradyunsg added state: awaiting PR Feature discussed, PR is needed and removed state: needs discussion This needs some more discussion labels Sep 7, 2022
@pradyunsg
Copy link
Member

I'll tentatively mark this as awaiting PR, to reflect that we'd review a PR for this and that's the next step forward here. I'll also note that I likely won't have the bandwidth to actually write this PR but likely will for reviewing one.

To my fellow maintainers, please feel free to remove that label if you think I applied that too eagerly. :)

@pelson
Copy link
Contributor Author

pelson commented Jan 31, 2023

We can close this out thanks to #11487 having been merged 🎉. Should be in pip 23.1 when it is released.

Thanks to all who contributed to the discussion. It was a long thread, over a long time span, but I believe we came out of it with a very worthwhile improvement, particularly for the case where a common base environment is supplied (e.g. by sys-admins) and where virtual environments are expected to be correctly configured from the base.

I personally will be happy to delete some ugly workarounds as a result of this 😄

@pfmoore
Copy link
Member

pfmoore commented Apr 22, 2023

This change caused issues in the 23.1 release (see #11982). As a result it is being reverted and a new PR will need to be submitted to re-implement the functionality, with a new design.

The design here was explicitly noted (above) as being backward incompatible, and indeed it was this incompatibility that caused the issues. Any replacement PR will need to either be fully backward compatible, or will need to have a suitable transition plan, including a deprecation period for the existing behaviour and a plan for what we will do if further compatibility issues occur.

Furthermore, this design breaks the isolation between virtual environments and their "base" environment - again, something that was noted in the discussion but which wasn't really addressed as a potential issue. Any replacement proposal needs to avoid doing this somehow. IMO, pip should not break the "virtual environments are isolated from their base environment" principle.

On a personal note, as a result of the issues, I'm now even less convinced that this is a feature that pip should accept. I understand there's a use case here, but there are other ways of addressing it. I'm very aware that "outsiders explaining how you can solve your problem" is generally more annoying than useful, but I would think that the following are possibilities to consider:

  1. A wrapper for creating virtual environments that populates the environment's pip.ini appropriately. The venv module is explicitly designed for this sort of extension. Yes, I know that one constraint is that the site can't control how users create environmnts, but "do it this way or we can't support you" is often a reasonable compromise.
  2. A global "Python startup" hook (implemented using one of the core Python mechanisms for such a thing) that checks the environment for a pip.ini and if there isn't one present, creates it. Again, this is using supported methods for customising your installation and therefore should not be a problem.

To be clear, my reason for preferring a solution using existing mechanisms is that if you take that route, you don't have to consider all of the other ways in which people can use (or abuse!) the existing mechanisms, and so you can implement a simpler, dedicated solution that doesn't have to take an "extreme defensiveness" position - which is what a new feature in pip has to take.

@pfmoore pfmoore reopened this Apr 22, 2023
@pypa pypa unlocked this conversation Apr 22, 2023
ranjith-jagadeesh pushed a commit to ranjith-jagadeesh/jsii that referenced this issue Apr 22, 2023
…s/@jsii/python-runtime (aws#4060)

Updates the requirements on [pip](https://github.com/pypa/pip) to permit the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p>
<blockquote>
<h1>23.1 (2023-04-15)</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li>Remove support for the deprecated <code>--install-options</code>. (<code>[#11358](pypa/pip#11358) &lt;https://github.com/pypa/pip/issues/11358&gt;</code>_)</li>
<li><code>--no-binary</code> does not imply <code>setup.py install</code> anymore. Instead a wheel will be
built locally and installed. (<code>[#11451](pypa/pip#11451) &lt;https://github.com/pypa/pip/issues/11451&gt;</code>_)</li>
<li><code>--no-binary</code> does not disable the cache of locally built wheels anymore. It only
means &quot;don't download wheels&quot;. (<code>[#11453](pypa/pip#11453) &lt;https://github.com/pypa/pip/issues/11453&gt;</code>_)</li>
<li>Deprecate <code>--build-option</code> and <code>--global-option</code>. Users are invited to switch to
<code>--config-settings</code>. (<code>[#11859](pypa/pip#11859) &lt;https://github.com/pypa/pip/issues/11859&gt;</code>_)</li>
<li>Using <code>--config-settings</code> with projects that don't have a <code>pyproject.toml</code> now print
a deprecation warning. In the future the presence of config settings will automatically
enable the default build backend for legacy projects and pass the setttings to it. (<code>[#11915](pypa/pip#11915) &lt;https://github.com/pypa/pip/issues/11915&gt;</code>_)</li>
<li>Remove <code>setup.py install</code> fallback when building a wheel failed for projects without
<code>pyproject.toml</code>. (<code>[#8368](pypa/pip#8368) &lt;https://github.com/pypa/pip/issues/8368&gt;</code>_)</li>
<li>When the <code>wheel</code> package is not installed, pip now uses the default build backend
instead of <code>setup.py install</code> for project without <code>pyproject.toml</code>. (<code>[#8559](pypa/pip#8559) &lt;https://github.com/pypa/pip/issues/8559&gt;</code>_)</li>
</ul>
<h2>Features</h2>
<ul>
<li>Specify egg-link location in assertion message when it does not match installed location to provide better error message for debugging. (<code>[#10476](pypa/pip#10476) &lt;https://github.com/pypa/pip/issues/10476&gt;</code>_)</li>
<li>Present conflict information during installation after each choice that is rejected (pass <code>-vv</code> to <code>pip install</code> to show it) (<code>[#10937](pypa/pip#10937) &lt;https://github.com/pypa/pip/issues/10937&gt;</code>_)</li>
<li>Display dependency chain on each Collecting/Processing log line. (<code>[#11169](pypa/pip#11169) &lt;https://github.com/pypa/pip/issues/11169&gt;</code>_)</li>
<li>Support a per-requirement <code>--config-settings</code> option in requirements files. (<code>[#11325](pypa/pip#11325) &lt;https://github.com/pypa/pip/issues/11325&gt;</code>_)</li>
<li>The <code>--config-settings</code>/<code>-C</code> option now supports using the same key multiple
times. When the same key is specified multiple times, all values are passed to
the build backend as a list, as opposed to the previous behavior, where pip would
only pass the last value if the same key was used multiple times. (<code>[#11681](pypa/pip#11681) &lt;https://github.com/pypa/pip/issues/11681&gt;</code>_)</li>
<li>Add <code>-C</code> as a short version of the <code>--config-settings</code> option. (<code>[#11786](pypa/pip#11786) &lt;https://github.com/pypa/pip/issues/11786&gt;</code>_)</li>
<li>Reduce the number of resolver rounds, since backjumping makes the resolver more efficient in finding solutions. This also makes pathological cases fail quicker. (<code>[#11908](pypa/pip#11908) &lt;https://github.com/pypa/pip/issues/11908&gt;</code>_)</li>
<li>Warn if <code>--hash</code> is used on a line without requirement in a requirements file. (<code>[#11935](pypa/pip#11935) &lt;https://github.com/pypa/pip/issues/11935&gt;</code>_)</li>
<li>Stop propagating CLI <code>--config-settings</code> to the build dependencies. They already did
not propagate to requirements provided in requirement files. To pass the same config
settings to several requirements, users should provide the requirements as CLI
arguments. (<code>[#11941](pypa/pip#11941) &lt;https://github.com/pypa/pip/issues/11941&gt;</code>_)</li>
<li>Support wheel cache when using <code>--require-hashes</code>. (<code>[#5037](pypa/pip#5037) &lt;https://github.com/pypa/pip/issues/5037&gt;</code>_)</li>
<li>Add <code>--keyring-provider</code> flag. See the Authentication page in the documentation for more info. (<code>[#8719](pypa/pip#8719) &lt;https://github.com/pypa/pip/issues/8719&gt;</code>_)</li>
<li>In the case of virtual environments, configuration files are now also included from the base installation. (<code>[#9752](pypa/pip#9752) &lt;https://github.com/pypa/pip/issues/9752&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fix grammar by changing &quot;A new release of pip available:&quot; to &quot;A new release of pip is available:&quot; in the notice used for indicating that. (<code>[#11529](pypa/pip#11529) &lt;https://github.com/pypa/pip/issues/11529&gt;</code>_)</li>
<li>Normalize paths before checking if installed scripts are on PATH. (<code>[#11719](pypa/pip#11719) &lt;https://github.com/pypa/pip/issues/11719&gt;</code>_)</li>
<li>Correct the way to decide if keyring is available. (<code>[#11774](pypa/pip#11774) &lt;https://github.com/pypa/pip/issues/11774&gt;</code>_)</li>
<li>More consistent resolution backtracking by removing legacy hack related to setuptools resolution (<code>[#11837](pypa/pip#11837) &lt;https://github.com/pypa/pip/issues/11837&gt;</code>_)</li>
</ul>

</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pypa/pip/commit/6424ac4600265490462015c2fc7f9a402dba9ed8"><code>6424ac4</code></a> Bump for release</li>
<li><a href="https://github.com/pypa/pip/commit/868338f9f79b58eff34dafb168aed65480d080d5"><code>868338f</code></a> Update AUTHORS.txt</li>
<li><a href="https://github.com/pypa/pip/commit/4f3a4f72697299da1a412cf10c919a989e0692f5"><code>4f3a4f7</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/11919">#11919</a> from sbidoul/deprecate-legacy-ignore-config-setting...</li>
<li><a href="https://github.com/pypa/pip/commit/dbf4e6842c9603792f6d3944a5c9cec17bd0a92a"><code>dbf4e68</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/11897">#11897</a> from sbidoul/cache-hash-checking-sbi</li>
<li><a href="https://github.com/pypa/pip/commit/efe2d27451d50b165df78093bf5885da713fbdf8"><code>efe2d27</code></a> Further refactor is_wheel_from_cache</li>
<li><a href="https://github.com/pypa/pip/commit/4beca6b4c9c510b19dbb6180e962425b89e8c839"><code>4beca6b</code></a> Improve test</li>
<li><a href="https://github.com/pypa/pip/commit/bd746e3136e5e1be2374a079bac66071dd967a8c"><code>bd746e3</code></a> Introduce ireq.cached_wheel_source_link</li>
<li><a href="https://github.com/pypa/pip/commit/caafe6e87d4f2998a77b194297e1c204cf6e10c2"><code>caafe6e</code></a> Add a couple of asserts</li>
<li><a href="https://github.com/pypa/pip/commit/a6ef6485be9512f18121298b058797c578f65d45"><code>a6ef648</code></a> Rename original_link_is_in_wheel_cache to is_wheel_from_cache</li>
<li><a href="https://github.com/pypa/pip/commit/ff8c8e38887880ad81ffd7cfc6a8373213c087b7"><code>ff8c8e3</code></a> Cosmetics</li>
<li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/23.0...23.1">compare view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)


</details>
@pelson
Copy link
Contributor Author

pelson commented Apr 25, 2023

Firstly, thanks to @pfmoore for handling the fallout from this. I appreciate that it has caused grief and a lot of expended effort, so thank you.

my reason for preferring a solution using existing mechanisms is that if you take that route, you don't have to consider all of the other ways in which people can use (or abuse!) the existing mechanisms

To re-iterate what we already discussed (in admittedly very long thread above), I do have local solutions and am proposing this into pip because I believe it is a valuable requirement for providers of Python distributions who aren't shipping as a system Python.

The truth is: this situation is unfortunate and was not practically foreseeable. I believe the change that was introduced was reasonable given all that we knew, and to be entirely frank, I consider the fundamental problem here is that the Windows Store builds are configuring pip in such a way (I would be happy to elaborate if it is considered a contentious statement). That being said, we now know more than we did before, and that may influence our design going forwards.

In practice, the workaround to this issue is quite simple (you documented a number of them yourself - simply setting an environment variable, for example, addresses the problem). Would suitable and specific documentation of this breaking change, plus a transition guide not be considered sufficient in this case? ("if you wish to use Windows Store Python with pip and venv, you must configure pip in your virtual environment, else it has been pre-configured to install into --user").

Any replacement PR will need to either be fully backward compatible, or will need to have a suitable transition plan, including a deprecation period for the existing behaviour

The deprecation period would presumably warn if a base pip config file is detected, and proceed to ignore it until a future release of pip? As well as a future flag to acknowledge the ignore behaviour, and enable the new. This is something I could buy-in to, and would be willing to proceed with, if it is agreed that this is the desired approach.

@pfmoore
Copy link
Member

pfmoore commented Apr 25, 2023

The deprecation period would presumably warn if a base pip config file is detected, and proceed to ignore it until a future release of pip? As well as a future flag to acknowledge the ignore behaviour, and enable the new. This is something I could buy-in to, and would be willing to proceed with, if it is agreed that this is the desired approach.

No, that's not sufficient (IMO). While I agree that the Windows Store distribution's approach is problematic, it is a real-world situation we have to deal with. As such, the problem is that the end user has no way of addressing any such deprecation warning - they can't remove the "base" config file, nor can they be expected to switch Python distributions.

If you can persuade the Windows Store distribution to change their behaviour (see python/cpython#103646) then when they have done so and there's a reasonable expectation that the majority of their users have upgraded to a version that no longer has the config file, then we could try again with this approach. But that will be a long time happening, and even then I still object to the idea of base environment configuration "leaking" into derived virtual environments that don't have --use-system-site set.

I think what's needed here is a different approach that solves your use case without using an existing config location. And it should probably also block setting configuration values (like --user) that don't behave the same in "system" and virtual environments. Or something. Remember that I'm the person who has been consistently uncomfortable with this feature - I never approved it, I simply stopped objecting in the face of other pip maintainers being willing to support it. I'm definitely not the person you want to be advising on how it should be designed. I'm the person who will pick apart any design you propose 😉

Hugo-Inmanta pushed a commit to inmanta/inmanta-core that referenced this issue Apr 27, 2023
Bumps [pip](https://github.com/pypa/pip) from 23.0.1 to 23.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p>
<blockquote>
<h1>23.1 (2023-04-15)</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li>Remove support for the deprecated <code>--install-options</code>. (<code>[#11358](pypa/pip#11358) &lt;https://github.com/pypa/pip/issues/11358&gt;</code>_)</li>
<li><code>--no-binary</code> does not imply <code>setup.py install</code> anymore. Instead a wheel will be
built locally and installed. (<code>[#11451](pypa/pip#11451) &lt;https://github.com/pypa/pip/issues/11451&gt;</code>_)</li>
<li><code>--no-binary</code> does not disable the cache of locally built wheels anymore. It only
means &quot;don't download wheels&quot;. (<code>[#11453](pypa/pip#11453) &lt;https://github.com/pypa/pip/issues/11453&gt;</code>_)</li>
<li>Deprecate <code>--build-option</code> and <code>--global-option</code>. Users are invited to switch to
<code>--config-settings</code>. (<code>[#11859](pypa/pip#11859) &lt;https://github.com/pypa/pip/issues/11859&gt;</code>_)</li>
<li>Using <code>--config-settings</code> with projects that don't have a <code>pyproject.toml</code> now print
a deprecation warning. In the future the presence of config settings will automatically
enable the default build backend for legacy projects and pass the setttings to it. (<code>[#11915](pypa/pip#11915) &lt;https://github.com/pypa/pip/issues/11915&gt;</code>_)</li>
<li>Remove <code>setup.py install</code> fallback when building a wheel failed for projects without
<code>pyproject.toml</code>. (<code>[#8368](pypa/pip#8368) &lt;https://github.com/pypa/pip/issues/8368&gt;</code>_)</li>
<li>When the <code>wheel</code> package is not installed, pip now uses the default build backend
instead of <code>setup.py install</code> for project without <code>pyproject.toml</code>. (<code>[#8559](pypa/pip#8559) &lt;https://github.com/pypa/pip/issues/8559&gt;</code>_)</li>
</ul>
<h2>Features</h2>
<ul>
<li>Specify egg-link location in assertion message when it does not match installed location to provide better error message for debugging. (<code>[#10476](pypa/pip#10476) &lt;https://github.com/pypa/pip/issues/10476&gt;</code>_)</li>
<li>Present conflict information during installation after each choice that is rejected (pass <code>-vv</code> to <code>pip install</code> to show it) (<code>[#10937](pypa/pip#10937) &lt;https://github.com/pypa/pip/issues/10937&gt;</code>_)</li>
<li>Display dependency chain on each Collecting/Processing log line. (<code>[#11169](pypa/pip#11169) &lt;https://github.com/pypa/pip/issues/11169&gt;</code>_)</li>
<li>Support a per-requirement <code>--config-settings</code> option in requirements files. (<code>[#11325](pypa/pip#11325) &lt;https://github.com/pypa/pip/issues/11325&gt;</code>_)</li>
<li>The <code>--config-settings</code>/<code>-C</code> option now supports using the same key multiple
times. When the same key is specified multiple times, all values are passed to
the build backend as a list, as opposed to the previous behavior, where pip would
only pass the last value if the same key was used multiple times. (<code>[#11681](pypa/pip#11681) &lt;https://github.com/pypa/pip/issues/11681&gt;</code>_)</li>
<li>Add <code>-C</code> as a short version of the <code>--config-settings</code> option. (<code>[#11786](pypa/pip#11786) &lt;https://github.com/pypa/pip/issues/11786&gt;</code>_)</li>
<li>Reduce the number of resolver rounds, since backjumping makes the resolver more efficient in finding solutions. This also makes pathological cases fail quicker. (<code>[#11908](pypa/pip#11908) &lt;https://github.com/pypa/pip/issues/11908&gt;</code>_)</li>
<li>Warn if <code>--hash</code> is used on a line without requirement in a requirements file. (<code>[#11935](pypa/pip#11935) &lt;https://github.com/pypa/pip/issues/11935&gt;</code>_)</li>
<li>Stop propagating CLI <code>--config-settings</code> to the build dependencies. They already did
not propagate to requirements provided in requirement files. To pass the same config
settings to several requirements, users should provide the requirements as CLI
arguments. (<code>[#11941](pypa/pip#11941) &lt;https://github.com/pypa/pip/issues/11941&gt;</code>_)</li>
<li>Support wheel cache when using <code>--require-hashes</code>. (<code>[#5037](pypa/pip#5037) &lt;https://github.com/pypa/pip/issues/5037&gt;</code>_)</li>
<li>Add <code>--keyring-provider</code> flag. See the Authentication page in the documentation for more info. (<code>[#8719](pypa/pip#8719) &lt;https://github.com/pypa/pip/issues/8719&gt;</code>_)</li>
<li>In the case of virtual environments, configuration files are now also included from the base installation. (<code>[#9752](pypa/pip#9752) &lt;https://github.com/pypa/pip/issues/9752&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fix grammar by changing &quot;A new release of pip available:&quot; to &quot;A new release of pip is available:&quot; in the notice used for indicating that. (<code>[#11529](pypa/pip#11529) &lt;https://github.com/pypa/pip/issues/11529&gt;</code>_)</li>
<li>Normalize paths before checking if installed scripts are on PATH. (<code>[#11719](pypa/pip#11719) &lt;https://github.com/pypa/pip/issues/11719&gt;</code>_)</li>
<li>Correct the way to decide if keyring is available. (<code>[#11774](pypa/pip#11774) &lt;https://github.com/pypa/pip/issues/11774&gt;</code>_)</li>
<li>More consistent resolution backtracking by removing legacy hack related to setuptools resolution (<code>[#11837](pypa/pip#11837) &lt;https://github.com/pypa/pip/issues/11837&gt;</code>_)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pypa/pip/commit/6424ac4600265490462015c2fc7f9a402dba9ed8"><code>6424ac4</code></a> Bump for release</li>
<li><a href="https://github.com/pypa/pip/commit/868338f9f79b58eff34dafb168aed65480d080d5"><code>868338f</code></a> Update AUTHORS.txt</li>
<li><a href="https://github.com/pypa/pip/commit/4f3a4f72697299da1a412cf10c919a989e0692f5"><code>4f3a4f7</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/11919">#11919</a> from sbidoul/deprecate-legacy-ignore-config-setting...</li>
<li><a href="https://github.com/pypa/pip/commit/dbf4e6842c9603792f6d3944a5c9cec17bd0a92a"><code>dbf4e68</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/11897">#11897</a> from sbidoul/cache-hash-checking-sbi</li>
<li><a href="https://github.com/pypa/pip/commit/efe2d27451d50b165df78093bf5885da713fbdf8"><code>efe2d27</code></a> Further refactor is_wheel_from_cache</li>
<li><a href="https://github.com/pypa/pip/commit/4beca6b4c9c510b19dbb6180e962425b89e8c839"><code>4beca6b</code></a> Improve test</li>
<li><a href="https://github.com/pypa/pip/commit/bd746e3136e5e1be2374a079bac66071dd967a8c"><code>bd746e3</code></a> Introduce ireq.cached_wheel_source_link</li>
<li><a href="https://github.com/pypa/pip/commit/caafe6e87d4f2998a77b194297e1c204cf6e10c2"><code>caafe6e</code></a> Add a couple of asserts</li>
<li><a href="https://github.com/pypa/pip/commit/a6ef6485be9512f18121298b058797c578f65d45"><code>a6ef648</code></a> Rename original_link_is_in_wheel_cache to is_wheel_from_cache</li>
<li><a href="https://github.com/pypa/pip/commit/ff8c8e38887880ad81ffd7cfc6a8373213c087b7"><code>ff8c8e3</code></a> Cosmetics</li>
<li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/23.0.1...23.1">compare view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=23.0.1&new-version=23.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

</details>
CedricGatay pushed a commit to CedricGatay/pr_custom_component that referenced this issue Apr 30, 2023
Updates the requirements on [pip](https://github.com/pypa/pip) to permit
the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's
changelog</a>.</em></p>
<blockquote>
<h1>23.1 (2023-04-15)</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li>Remove support for the deprecated <code>--install-options</code>.
(<code>[#11358](pypa/pip#11358)
&lt;https://github.com/pypa/pip/issues/11358&gt;</code>_)</li>
<li><code>--no-binary</code> does not imply <code>setup.py
install</code> anymore. Instead a wheel will be
built locally and installed.
(<code>[#11451](pypa/pip#11451)
&lt;https://github.com/pypa/pip/issues/11451&gt;</code>_)</li>
<li><code>--no-binary</code> does not disable the cache of locally built
wheels anymore. It only
means &quot;don't download wheels&quot;.
(<code>[#11453](pypa/pip#11453)
&lt;https://github.com/pypa/pip/issues/11453&gt;</code>_)</li>
<li>Deprecate <code>--build-option</code> and
<code>--global-option</code>. Users are invited to switch to
<code>--config-settings</code>.
(<code>[#11859](pypa/pip#11859)
&lt;https://github.com/pypa/pip/issues/11859&gt;</code>_)</li>
<li>Using <code>--config-settings</code> with projects that don't have a
<code>pyproject.toml</code> now prints
a deprecation warning. In the future the presence of config settings
will automatically
enable the default build backend for legacy projects and pass the
setttings to it.
(<code>[#11915](pypa/pip#11915)
&lt;https://github.com/pypa/pip/issues/11915&gt;</code>_)</li>
<li>Remove <code>setup.py install</code> fallback when building a wheel
failed for projects without
<code>pyproject.toml</code>.
(<code>[#8368](pypa/pip#8368)
&lt;https://github.com/pypa/pip/issues/8368&gt;</code>_)</li>
<li>When the <code>wheel</code> package is not installed, pip now uses
the default build backend
instead of <code>setup.py install</code> and <code>setup.py
develop</code> for project without
<code>pyproject.toml</code>.
(<code>[#8559](pypa/pip#8559)
&lt;https://github.com/pypa/pip/issues/8559&gt;</code>_)</li>
</ul>
<h2>Features</h2>
<ul>
<li>Specify egg-link location in assertion message when it does not
match installed location to provide better error message for debugging.
(<code>[#10476](pypa/pip#10476)
&lt;https://github.com/pypa/pip/issues/10476&gt;</code>_)</li>
<li>Present conflict information during installation after each choice
that is rejected (pass <code>-vv</code> to <code>pip install</code> to
show it) (<code>[#10937](pypa/pip#10937)
&lt;https://github.com/pypa/pip/issues/10937&gt;</code>_)</li>
<li>Display dependency chain on each Collecting/Processing log line.
(<code>[#11169](pypa/pip#11169)
&lt;https://github.com/pypa/pip/issues/11169&gt;</code>_)</li>
<li>Support a per-requirement <code>--config-settings</code> option in
requirements files.
(<code>[#11325](pypa/pip#11325)
&lt;https://github.com/pypa/pip/issues/11325&gt;</code>_)</li>
<li>The <code>--config-settings</code>/<code>-C</code> option now
supports using the same key multiple
times. When the same key is specified multiple times, all values are
passed to
the build backend as a list, as opposed to the previous behavior, where
pip would
only pass the last value if the same key was used multiple times.
(<code>[#11681](pypa/pip#11681)
&lt;https://github.com/pypa/pip/issues/11681&gt;</code>_)</li>
<li>Add <code>-C</code> as a short version of the
<code>--config-settings</code> option.
(<code>[#11786](pypa/pip#11786)
&lt;https://github.com/pypa/pip/issues/11786&gt;</code>_)</li>
<li>Reduce the number of resolver rounds, since backjumping makes the
resolver more efficient in finding solutions. This also makes
pathological cases fail quicker.
(<code>[#11908](pypa/pip#11908)
&lt;https://github.com/pypa/pip/issues/11908&gt;</code>_)</li>
<li>Warn if <code>--hash</code> is used on a line without requirement in
a requirements file.
(<code>[#11935](pypa/pip#11935)
&lt;https://github.com/pypa/pip/issues/11935&gt;</code>_)</li>
<li>Stop propagating CLI <code>--config-settings</code> to the build
dependencies. They already did
not propagate to requirements provided in requirement files. To pass the
same config
settings to several requirements, users should provide the requirements
as CLI
arguments. (<code>[#11941](pypa/pip#11941)
&lt;https://github.com/pypa/pip/issues/11941&gt;</code>_)</li>
<li>Support wheel cache when using <code>--require-hashes</code>.
(<code>[#5037](pypa/pip#5037)
&lt;https://github.com/pypa/pip/issues/5037&gt;</code>_)</li>
<li>Add <code>--keyring-provider</code> flag. See the Authentication
page in the documentation for more info.
(<code>[#8719](pypa/pip#8719)
&lt;https://github.com/pypa/pip/issues/8719&gt;</code>_)</li>
<li>In the case of virtual environments, configuration files are now
also included from the base installation.
(<code>[#9752](pypa/pip#9752)
&lt;https://github.com/pypa/pip/issues/9752&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fix grammar by changing &quot;A new release of pip available:&quot;
to &quot;A new release of pip is available:&quot; in the notice used for
indicating that.
(<code>[#11529](pypa/pip#11529)
&lt;https://github.com/pypa/pip/issues/11529&gt;</code>_)</li>
<li>Normalize paths before checking if installed scripts are on PATH.
(<code>[#11719](pypa/pip#11719)
&lt;https://github.com/pypa/pip/issues/11719&gt;</code>_)</li>
<li>Correct the way to decide if keyring is available.
(<code>[#11774](pypa/pip#11774)
&lt;https://github.com/pypa/pip/issues/11774&gt;</code>_)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pypa/pip/commit/6424ac4600265490462015c2fc7f9a402dba9ed8"><code>6424ac4</code></a>
Bump for release</li>
<li><a
href="https://github.com/pypa/pip/commit/868338f9f79b58eff34dafb168aed65480d080d5"><code>868338f</code></a>
Update AUTHORS.txt</li>
<li><a
href="https://github.com/pypa/pip/commit/4f3a4f72697299da1a412cf10c919a989e0692f5"><code>4f3a4f7</code></a>
Merge pull request <a
href="https://redirect.github.com/pypa/pip/issues/11919">#11919</a> from
sbidoul/deprecate-legacy-ignore-config-setting...</li>
<li><a
href="https://github.com/pypa/pip/commit/dbf4e6842c9603792f6d3944a5c9cec17bd0a92a"><code>dbf4e68</code></a>
Merge pull request <a
href="https://redirect.github.com/pypa/pip/issues/11897">#11897</a> from
sbidoul/cache-hash-checking-sbi</li>
<li><a
href="https://github.com/pypa/pip/commit/efe2d27451d50b165df78093bf5885da713fbdf8"><code>efe2d27</code></a>
Further refactor is_wheel_from_cache</li>
<li><a
href="https://github.com/pypa/pip/commit/4beca6b4c9c510b19dbb6180e962425b89e8c839"><code>4beca6b</code></a>
Improve test</li>
<li><a
href="https://github.com/pypa/pip/commit/bd746e3136e5e1be2374a079bac66071dd967a8c"><code>bd746e3</code></a>
Introduce ireq.cached_wheel_source_link</li>
<li><a
href="https://github.com/pypa/pip/commit/caafe6e87d4f2998a77b194297e1c204cf6e10c2"><code>caafe6e</code></a>
Add a couple of asserts</li>
<li><a
href="https://github.com/pypa/pip/commit/a6ef6485be9512f18121298b058797c578f65d45"><code>a6ef648</code></a>
Rename original_link_is_in_wheel_cache to is_wheel_from_cache</li>
<li><a
href="https://github.com/pypa/pip/commit/ff8c8e38887880ad81ffd7cfc6a8373213c087b7"><code>ff8c8e3</code></a>
Cosmetics</li>
<li>Additional commits viewable in <a
href="https://github.com/pypa/pip/compare/21.0...23.1">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
fredck pushed a commit to fredck/lightener that referenced this issue May 10, 2023
Updates the requirements on [pip](https://github.com/pypa/pip) to permit
the latest version.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's
changelog</a>.</em></p>
<blockquote>
<h1>23.1 (2023-04-15)</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li>Remove support for the deprecated <code>--install-options</code>.
(<code>[#11358](pypa/pip#11358)
&lt;https://github.com/pypa/pip/issues/11358&gt;</code>_)</li>
<li><code>--no-binary</code> does not imply <code>setup.py
install</code> anymore. Instead a wheel will be
built locally and installed.
(<code>[#11451](pypa/pip#11451)
&lt;https://github.com/pypa/pip/issues/11451&gt;</code>_)</li>
<li><code>--no-binary</code> does not disable the cache of locally built
wheels anymore. It only
means &quot;don't download wheels&quot;.
(<code>[#11453](pypa/pip#11453)
&lt;https://github.com/pypa/pip/issues/11453&gt;</code>_)</li>
<li>Deprecate <code>--build-option</code> and
<code>--global-option</code>. Users are invited to switch to
<code>--config-settings</code>.
(<code>[#11859](pypa/pip#11859)
&lt;https://github.com/pypa/pip/issues/11859&gt;</code>_)</li>
<li>Using <code>--config-settings</code> with projects that don't have a
<code>pyproject.toml</code> now prints
a deprecation warning. In the future the presence of config settings
will automatically
enable the default build backend for legacy projects and pass the
setttings to it.
(<code>[#11915](pypa/pip#11915)
&lt;https://github.com/pypa/pip/issues/11915&gt;</code>_)</li>
<li>Remove <code>setup.py install</code> fallback when building a wheel
failed for projects without
<code>pyproject.toml</code>.
(<code>[#8368](pypa/pip#8368)
&lt;https://github.com/pypa/pip/issues/8368&gt;</code>_)</li>
<li>When the <code>wheel</code> package is not installed, pip now uses
the default build backend
instead of <code>setup.py install</code> and <code>setup.py
develop</code> for project without
<code>pyproject.toml</code>.
(<code>[#8559](pypa/pip#8559)
&lt;https://github.com/pypa/pip/issues/8559&gt;</code>_)</li>
</ul>
<h2>Features</h2>
<ul>
<li>Specify egg-link location in assertion message when it does not
match installed location to provide better error message for debugging.
(<code>[#10476](pypa/pip#10476)
&lt;https://github.com/pypa/pip/issues/10476&gt;</code>_)</li>
<li>Present conflict information during installation after each choice
that is rejected (pass <code>-vv</code> to <code>pip install</code> to
show it) (<code>[#10937](pypa/pip#10937)
&lt;https://github.com/pypa/pip/issues/10937&gt;</code>_)</li>
<li>Display dependency chain on each Collecting/Processing log line.
(<code>[#11169](pypa/pip#11169)
&lt;https://github.com/pypa/pip/issues/11169&gt;</code>_)</li>
<li>Support a per-requirement <code>--config-settings</code> option in
requirements files.
(<code>[#11325](pypa/pip#11325)
&lt;https://github.com/pypa/pip/issues/11325&gt;</code>_)</li>
<li>The <code>--config-settings</code>/<code>-C</code> option now
supports using the same key multiple
times. When the same key is specified multiple times, all values are
passed to
the build backend as a list, as opposed to the previous behavior, where
pip would
only pass the last value if the same key was used multiple times.
(<code>[#11681](pypa/pip#11681)
&lt;https://github.com/pypa/pip/issues/11681&gt;</code>_)</li>
<li>Add <code>-C</code> as a short version of the
<code>--config-settings</code> option.
(<code>[#11786](pypa/pip#11786)
&lt;https://github.com/pypa/pip/issues/11786&gt;</code>_)</li>
<li>Reduce the number of resolver rounds, since backjumping makes the
resolver more efficient in finding solutions. This also makes
pathological cases fail quicker.
(<code>[#11908](pypa/pip#11908)
&lt;https://github.com/pypa/pip/issues/11908&gt;</code>_)</li>
<li>Warn if <code>--hash</code> is used on a line without requirement in
a requirements file.
(<code>[#11935](pypa/pip#11935)
&lt;https://github.com/pypa/pip/issues/11935&gt;</code>_)</li>
<li>Stop propagating CLI <code>--config-settings</code> to the build
dependencies. They already did
not propagate to requirements provided in requirement files. To pass the
same config
settings to several requirements, users should provide the requirements
as CLI
arguments. (<code>[#11941](pypa/pip#11941)
&lt;https://github.com/pypa/pip/issues/11941&gt;</code>_)</li>
<li>Support wheel cache when using <code>--require-hashes</code>.
(<code>[#5037](pypa/pip#5037)
&lt;https://github.com/pypa/pip/issues/5037&gt;</code>_)</li>
<li>Add <code>--keyring-provider</code> flag. See the Authentication
page in the documentation for more info.
(<code>[#8719](pypa/pip#8719)
&lt;https://github.com/pypa/pip/issues/8719&gt;</code>_)</li>
<li>In the case of virtual environments, configuration files are now
also included from the base installation.
(<code>[#9752](pypa/pip#9752)
&lt;https://github.com/pypa/pip/issues/9752&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fix grammar by changing &quot;A new release of pip available:&quot;
to &quot;A new release of pip is available:&quot; in the notice used for
indicating that.
(<code>[#11529](pypa/pip#11529)
&lt;https://github.com/pypa/pip/issues/11529&gt;</code>_)</li>
<li>Normalize paths before checking if installed scripts are on PATH.
(<code>[#11719](pypa/pip#11719)
&lt;https://github.com/pypa/pip/issues/11719&gt;</code>_)</li>
<li>Correct the way to decide if keyring is available.
(<code>[#11774](pypa/pip#11774)
&lt;https://github.com/pypa/pip/issues/11774&gt;</code>_)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/pypa/pip/commit/6424ac4600265490462015c2fc7f9a402dba9ed8"><code>6424ac4</code></a>
Bump for release</li>
<li><a
href="https://github.com/pypa/pip/commit/868338f9f79b58eff34dafb168aed65480d080d5"><code>868338f</code></a>
Update AUTHORS.txt</li>
<li><a
href="https://github.com/pypa/pip/commit/4f3a4f72697299da1a412cf10c919a989e0692f5"><code>4f3a4f7</code></a>
Merge pull request <a
href="https://redirect.github.com/pypa/pip/issues/11919">#11919</a> from
sbidoul/deprecate-legacy-ignore-config-setting...</li>
<li><a
href="https://github.com/pypa/pip/commit/dbf4e6842c9603792f6d3944a5c9cec17bd0a92a"><code>dbf4e68</code></a>
Merge pull request <a
href="https://redirect.github.com/pypa/pip/issues/11897">#11897</a> from
sbidoul/cache-hash-checking-sbi</li>
<li><a
href="https://github.com/pypa/pip/commit/efe2d27451d50b165df78093bf5885da713fbdf8"><code>efe2d27</code></a>
Further refactor is_wheel_from_cache</li>
<li><a
href="https://github.com/pypa/pip/commit/4beca6b4c9c510b19dbb6180e962425b89e8c839"><code>4beca6b</code></a>
Improve test</li>
<li><a
href="https://github.com/pypa/pip/commit/bd746e3136e5e1be2374a079bac66071dd967a8c"><code>bd746e3</code></a>
Introduce ireq.cached_wheel_source_link</li>
<li><a
href="https://github.com/pypa/pip/commit/caafe6e87d4f2998a77b194297e1c204cf6e10c2"><code>caafe6e</code></a>
Add a couple of asserts</li>
<li><a
href="https://github.com/pypa/pip/commit/a6ef6485be9512f18121298b058797c578f65d45"><code>a6ef648</code></a>
Rename original_link_is_in_wheel_cache to is_wheel_from_cache</li>
<li><a
href="https://github.com/pypa/pip/commit/ff8c8e38887880ad81ffd7cfc6a8373213c087b7"><code>ff8c8e3</code></a>
Cosmetics</li>
<li>Additional commits viewable in <a
href="https://github.com/pypa/pip/compare/21.0...23.1">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
inmantaci pushed a commit to inmanta/inmanta-core that referenced this issue Jun 30, 2023
Bumps [pip](https://github.com/pypa/pip) from 23.0.1 to 23.1.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/pypa/pip/blob/main/NEWS.rst">pip's changelog</a>.</em></p>
<blockquote>
<h1>23.1 (2023-04-15)</h1>
<h2>Deprecations and Removals</h2>
<ul>
<li>Remove support for the deprecated <code>--install-options</code>. (<code>[#11358](pypa/pip#11358) &lt;https://github.com/pypa/pip/issues/11358&gt;</code>_)</li>
<li><code>--no-binary</code> does not imply <code>setup.py install</code> anymore. Instead a wheel will be
built locally and installed. (<code>[#11451](pypa/pip#11451) &lt;https://github.com/pypa/pip/issues/11451&gt;</code>_)</li>
<li><code>--no-binary</code> does not disable the cache of locally built wheels anymore. It only
means &quot;don't download wheels&quot;. (<code>[#11453](pypa/pip#11453) &lt;https://github.com/pypa/pip/issues/11453&gt;</code>_)</li>
<li>Deprecate <code>--build-option</code> and <code>--global-option</code>. Users are invited to switch to
<code>--config-settings</code>. (<code>[#11859](pypa/pip#11859) &lt;https://github.com/pypa/pip/issues/11859&gt;</code>_)</li>
<li>Using <code>--config-settings</code> with projects that don't have a <code>pyproject.toml</code> now prints
a deprecation warning. In the future the presence of config settings will automatically
enable the default build backend for legacy projects and pass the setttings to it. (<code>[#11915](pypa/pip#11915) &lt;https://github.com/pypa/pip/issues/11915&gt;</code>_)</li>
<li>Remove <code>setup.py install</code> fallback when building a wheel failed for projects without
<code>pyproject.toml</code>. (<code>[#8368](pypa/pip#8368) &lt;https://github.com/pypa/pip/issues/8368&gt;</code>_)</li>
<li>When the <code>wheel</code> package is not installed, pip now uses the default build backend
instead of <code>setup.py install</code> and <code>setup.py develop</code> for project without
<code>pyproject.toml</code>. (<code>[#8559](pypa/pip#8559) &lt;https://github.com/pypa/pip/issues/8559&gt;</code>_)</li>
</ul>
<h2>Features</h2>
<ul>
<li>Specify egg-link location in assertion message when it does not match installed location to provide better error message for debugging. (<code>[#10476](pypa/pip#10476) &lt;https://github.com/pypa/pip/issues/10476&gt;</code>_)</li>
<li>Present conflict information during installation after each choice that is rejected (pass <code>-vv</code> to <code>pip install</code> to show it) (<code>[#10937](pypa/pip#10937) &lt;https://github.com/pypa/pip/issues/10937&gt;</code>_)</li>
<li>Display dependency chain on each Collecting/Processing log line. (<code>[#11169](pypa/pip#11169) &lt;https://github.com/pypa/pip/issues/11169&gt;</code>_)</li>
<li>Support a per-requirement <code>--config-settings</code> option in requirements files. (<code>[#11325](pypa/pip#11325) &lt;https://github.com/pypa/pip/issues/11325&gt;</code>_)</li>
<li>The <code>--config-settings</code>/<code>-C</code> option now supports using the same key multiple
times. When the same key is specified multiple times, all values are passed to
the build backend as a list, as opposed to the previous behavior, where pip would
only pass the last value if the same key was used multiple times. (<code>[#11681](pypa/pip#11681) &lt;https://github.com/pypa/pip/issues/11681&gt;</code>_)</li>
<li>Add <code>-C</code> as a short version of the <code>--config-settings</code> option. (<code>[#11786](pypa/pip#11786) &lt;https://github.com/pypa/pip/issues/11786&gt;</code>_)</li>
<li>Reduce the number of resolver rounds, since backjumping makes the resolver more efficient in finding solutions. This also makes pathological cases fail quicker. (<code>[#11908](pypa/pip#11908) &lt;https://github.com/pypa/pip/issues/11908&gt;</code>_)</li>
<li>Warn if <code>--hash</code> is used on a line without requirement in a requirements file. (<code>[#11935](pypa/pip#11935) &lt;https://github.com/pypa/pip/issues/11935&gt;</code>_)</li>
<li>Stop propagating CLI <code>--config-settings</code> to the build dependencies. They already did
not propagate to requirements provided in requirement files. To pass the same config
settings to several requirements, users should provide the requirements as CLI
arguments. (<code>[#11941](pypa/pip#11941) &lt;https://github.com/pypa/pip/issues/11941&gt;</code>_)</li>
<li>Support wheel cache when using <code>--require-hashes</code>. (<code>[#5037](pypa/pip#5037) &lt;https://github.com/pypa/pip/issues/5037&gt;</code>_)</li>
<li>Add <code>--keyring-provider</code> flag. See the Authentication page in the documentation for more info. (<code>[#8719](pypa/pip#8719) &lt;https://github.com/pypa/pip/issues/8719&gt;</code>_)</li>
<li>In the case of virtual environments, configuration files are now also included from the base installation. (<code>[#9752](pypa/pip#9752) &lt;https://github.com/pypa/pip/issues/9752&gt;</code>_)</li>
</ul>
<h2>Bug Fixes</h2>
<ul>
<li>Fix grammar by changing &quot;A new release of pip available:&quot; to &quot;A new release of pip is available:&quot; in the notice used for indicating that. (<code>[#11529](pypa/pip#11529) &lt;https://github.com/pypa/pip/issues/11529&gt;</code>_)</li>
<li>Normalize paths before checking if installed scripts are on PATH. (<code>[#11719](pypa/pip#11719) &lt;https://github.com/pypa/pip/issues/11719&gt;</code>_)</li>
<li>Correct the way to decide if keyring is available. (<code>[#11774](pypa/pip#11774) &lt;https://github.com/pypa/pip/issues/11774&gt;</code>_)</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/pypa/pip/commit/6424ac4600265490462015c2fc7f9a402dba9ed8"><code>6424ac4</code></a> Bump for release</li>
<li><a href="https://github.com/pypa/pip/commit/868338f9f79b58eff34dafb168aed65480d080d5"><code>868338f</code></a> Update AUTHORS.txt</li>
<li><a href="https://github.com/pypa/pip/commit/4f3a4f72697299da1a412cf10c919a989e0692f5"><code>4f3a4f7</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/11919">#11919</a> from sbidoul/deprecate-legacy-ignore-config-setting...</li>
<li><a href="https://github.com/pypa/pip/commit/dbf4e6842c9603792f6d3944a5c9cec17bd0a92a"><code>dbf4e68</code></a> Merge pull request <a href="https://redirect.github.com/pypa/pip/issues/11897">#11897</a> from sbidoul/cache-hash-checking-sbi</li>
<li><a href="https://github.com/pypa/pip/commit/efe2d27451d50b165df78093bf5885da713fbdf8"><code>efe2d27</code></a> Further refactor is_wheel_from_cache</li>
<li><a href="https://github.com/pypa/pip/commit/4beca6b4c9c510b19dbb6180e962425b89e8c839"><code>4beca6b</code></a> Improve test</li>
<li><a href="https://github.com/pypa/pip/commit/bd746e3136e5e1be2374a079bac66071dd967a8c"><code>bd746e3</code></a> Introduce ireq.cached_wheel_source_link</li>
<li><a href="https://github.com/pypa/pip/commit/caafe6e87d4f2998a77b194297e1c204cf6e10c2"><code>caafe6e</code></a> Add a couple of asserts</li>
<li><a href="https://github.com/pypa/pip/commit/a6ef6485be9512f18121298b058797c578f65d45"><code>a6ef648</code></a> Rename original_link_is_in_wheel_cache to is_wheel_from_cache</li>
<li><a href="https://github.com/pypa/pip/commit/ff8c8e38887880ad81ffd7cfc6a8373213c087b7"><code>ff8c8e3</code></a> Cosmetics</li>
<li>Additional commits viewable in <a href="https://github.com/pypa/pip/compare/23.0.1...23.1">compare view</a></li>
</ul>
</details>
<br />

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pip&package-manager=pip&previous-version=23.0.1&new-version=23.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

</details>
@dalebrydon
Copy link
Contributor

I think I tend to agree with @pfmoore, that this is a bit of a confusing change to me. I admit I never fully grasped why changes to other tools aren't the preferred method. Wouldn't something like a --copy-pip-config option in venv or virtualenv work too?

The big thing I don't like with the current design of pip config is that putting things in the Python install is treated as site config when not working in a virtualenv. To me things in the Python install are clearly global config but sort of get stuck as project-level config because we use sys.site for this. Note that even the comments in pip source refer to the site configuration as virtualenv config, so I think this is an accidental side effect more than intentional design. The base thing only extends this problem, which is why I would prefer some other solution.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C: configuration Configuration management and loading state: awaiting PR Feature discussed, PR is needed type: feature request Request for a new feature
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants