PEX_INHERIT_PATH (and maybe other vars) ignored when venv used from cache

[benjyw]

To reproduce:

python -m pex ansicolors --venv -o repro.pex 
./repro.pex

ctrl-D

PEX_INHERIT_PATH=invalid_val ./repro.pex

ctrl-D

expected behavior: Invalid value for PEX_INHERIT_PATH error.
actual behavior: pex runs from cached venv

Expected behavior can be triggered by:

rm -rf ~/.pex/venvs
PEX_INHERIT_PATH=invalid_val ./repro.pex

Obviously the invalid_val thing is just an easy way to show whether PEX_INHERIT_PATH is being inspected. My real goal is to use it with a valid value and have that value take effect...

[jsirois]

Where would you expect the PEX to inherit from? It's running in a venv after all.

[benjyw]

Good point from an implementation perspective.

But from the user perspective: I've been handed a .pex file (I neither know nor care that it was created with --venv). I see in the documentation that PEX_INHERIT_PATH should do a thing, and I see that in practice it not only does not do the thing, but it is ignored - I can give it a bad value without getting an error. Then I nuke the venv cache and suddenly do get that error.

I think that user would expect the PEX to inherit from the venv they think they're in (that is, the venv active in their current shell), and/or from their PYTHONPATH.

But if we can't make the feature work sensibly, and you make a good argument for that, then we should probably error out on any attempt to use it?

[jsirois]

Yeah, it's a tough feature generally. Venv aside it almost makes no sense since Pex will pick a compatible interpreter at runtime and maybe that one was not the one you were expecting / doesn't have the extra distributions you wanted. Afaict the feature is only for experimenting unless you either:

1. Have tightly controlled target hosts with just one compatible interpreter.
2. Like 1, except you use pexrc files to do the tightening.
3. You run PEX files via a script or other command line that is of the form /this/compatible/interpreter my.pex.

I'd prefer to fail fast for unsupported Pex env vars in venv mode until someone can come up with an easy to understand algorithm for picking which env to inherit the path from for a venv pex. I'd like even more to drop the feature!

[benjyw]

Remind me why we have the feature? AFAICT we don't use it in Pants.

[jsirois]

Pretty sure Twitter. You can imagine having a blessed interpreter on a host with puppet installed numerical computing distributions in its site-packages and avoiding shipping those dists over the network in N PEXes. I need to do some research though to confirm this.

[benjyw]

Oh right, that makes sense. Well we should probably keep the feature, I can see how it might be handy.