Isolated venv does not inherit PIP index

[LajosCseppento]

I am working in an development & build environment without internet (PyPI) access. I call python -m build --sdist . from a venv already configured with PIP_INDEX_URL and PIP_TRUSTED_HOST. However, these are not inherited by the isolated venv and I have to export them again.

To setup the venv, we use an enterprise wrapper for Python, and as general guideline, we should not specify manually the PIP_* parameters (but for now it is fine).

Tagging @pelson , dev of our wrapper.

(Suspected) related code: https://github.com/pypa/build/blob/main/src/build/__main__.py#L49-L59

[layday]

Thanks for reporting. It should not be the case that env vars are not inherited. Can you put together a minimal reproducible example for us to try?

[gaborbernat]

from a venv already configured with PIP_INDEX_URL and PIP_TRUSTED_HOST

How is this configuration set?

[LajosCseppento]

It is in the venv's pip.conf:

venv/pip.conf -> /some/nfs/location/pip.conf (symlink)

(venv) $ cat venv/pip.conf 
[global]
timeout = 20
index = https://repo.example.com/repository/vr-py-releases/pypi
index-url = https://repo.example.com/repository/vr-py-releases/simple
trusted-host=repo.example.com
disable-pip-version-check=1
no-python-version-warning=1
user=no

Does it make sense to inherit it?

[layday]

No, the parent venv is not visible to the isolated venv. That would defeat the purpose of isolation.

[gaborbernat]

Does it make sense to inherit it?

If it would, it would be a venv feature. You can make this happen though by using virtualenv extra, and implementing a virtualenv plugin https://virtualenv.pypa.io/en/latest/user_guide.html#seeders

[LajosCseppento]

Ok, thanks for the info - maybe pypa/pip#9752 will solve it

[pradyunsg]

IMO for use cases beyond "get dependencies from PyPI", you should use the Python API provided, and implement something with the specific behaviours that you want.

I quite like how much power is easily accessible from the API of this tool. :)

[LajosCseppento]

First of all, wow folks, picking up this so quickly!

@pradyunsg To clarify the description, exporting the variables works and does not try to reach the internet:

export PIP_INDEX_URL=...
export PIP_TRUSTED_HOST=...
python -m build --sdist . 

To be frank, for me this seems a much simpler workaround than direct API usage (less code ;- )). However, in the CI/CD workflow I need to re-define these variables only when using build. The point is that in our enterprise setup devs should rely on the enterprise tooling, on-premise repo and not specify "low-level parameters" (such as PIP_INDEX_URL) manually.

Thus, I opened this issue. I could imagine that a new flag like --inherit-pip-conf (or find a better name) could also solve the problem:

python -m build --sdist --inherit-pip-conf .

Or pypa/pip#9752 might also work - but I can imagine other solutions too. However, as far as I remember, the usage of pip.conf is not recommended in our environment.

Approching from another way:

• the parent venv is configured to use a custom repository, not PyPI on purpose
• when it comes to build, it uses PyPI in the isolated environment

For me, isolated environment means that isolated from the original venv and packages installed into that - maybe I am wrong.

All in all, a feature toggle would be great and would not break backward compat -- yet I understand if it is an undesired complexity. I am also curious how other companies tackle this problem (export again like the workaround above? always have internet connection?)

[gaborbernat]

I'd expect you to not have venv specific pip.conf but a global (per machine) pip.conf for such use cases.

[FFY00]

I´d like to note that pypa/build using pip is merely a implementation detail, it should not be relied on, and so this workaround can not be relied on either (though it will still probably work in the near-mid future). This issue needs to be fixed, IIRC there was other issue discussing it but I am too tired to find it right now. My suggestion if you want to make sure everything absolutely works, and will continue to work, is to build without isolation - you will be required to provide the dependencies yourself.

[LajosCseppento]

Thanks for the replies. @gaborbernat my colleague explained here per machine pip.conf is not suitable: pypa/pip#9752 (comment)

[gaborbernat]

In conclusion, I think the issue here isn't related to this tool. Is a limitation of virtualenv/venv/pip. And likely the fix would be either to allow pip to be smarter from where it pulls its configuration or extending virtualenv/venv to generate the appropriate configuration on virtual environment creation. Neither offers actionable items for this project, so I'll close this for now.