Pex fails to layout venvs properly.

[jsirois]

There is probably a bug here for unzip mode too.

For example, via Pip:

^jsirois@gill /tmp/test $ python -mvenv nbconvert.venv
^jsirois@gill /tmp/test $ source nbconvert.venv/bin/activate
(nbconvert.venv) ^jsirois@gill /tmp/test $ pip -q install -U pip
(nbconvert.venv) ^jsirois@gill /tmp/test $ pip -q install nbconvert
(nbconvert.venv) ^jsirois@gill /tmp/test $ find nbconvert.venv -type d -name share
nbconvert.venv/share
(nbconvert.venv) ^jsirois@gill /tmp/test $ grep share nbconvert.venv/lib/python3.10/site-packages/nbconvert-6.4.2.dist-info/RECORD | head
../../../share/jupyter/nbconvert/templates/asciidoc/conf.json,sha256=YtRONxl919kXwtomCBZ-CM4p-uOFDOCk9SnmFThA_Qg,87
../../../share/jupyter/nbconvert/templates/asciidoc/index.asciidoc.j2,sha256=ldvuvwVyz0b7bO2ZZ8NXcnI4cdyKJBfslWnd_xZLZKc,2298
../../../share/jupyter/nbconvert/templates/base/celltags.j2,sha256=_Gi6rqD8HQUQZgzuRRSVqDJ-J5rD0Z9lmPswq2kW6mc,213
../../../share/jupyter/nbconvert/templates/base/display_priority.j2,sha256=wEMM_359LWVzWQpz8GzYXeWkzQGA3Mqwp_pRiYCgUhw,1442
../../../share/jupyter/nbconvert/templates/base/jupyter_widgets.html.j2,sha256=3vEiL0FN_Nni6SCNlj-jLpukRyPGRY2NUzj-JgJhBFs,1256
../../../share/jupyter/nbconvert/templates/base/mathjax.html.j2,sha256=vmg4jv_ICDmQbQiN40X4V_GIfDlxNTusgfG2Qv6HpB0,1198
../../../share/jupyter/nbconvert/templates/base/null.j2,sha256=Fw8IIQZnAa6RcI30Vjy6YPDLkFX44glRQTTjEc-PfrI,6229
../../../share/jupyter/nbconvert/templates/basic/conf.json,sha256=74s0_-_38TnwNLy8xIjys_J_acbOEGRu8DKNUftxMHk,87
../../../share/jupyter/nbconvert/templates/basic/index.html.j2,sha256=ovChqOqeUNBJMBSvRCDPBvF6wJVhDPPF5ly3Y-qPs6g,39
../../../share/jupyter/nbconvert/templates/classic/base.html.j2,sha256=w01A7PQw2sLpKCl1tycqhAHl9NTTYir9W9Myi-hZZ6Y,7499

But via Pex:

^jsirois@gill /tmp/test $ pex nbconvert --venv -o nbconvert.pex
^jsirois@gill /tmp/test $ PEX_TOOLS=1 ./nbconvert.pex venv nbconvert.pex.venv
^jsirois@gill /tmp/test $ find nbconvert.pex.venv -type d -name share
nbconvert.pex.venv/lib/python3.10/site-packages/share

So Pex is failing to read the RECORD to determine final relative resting paths in the venv. It currently special-cases bin with this code instead of using the RECORD so gets away with the vast majority of use cases, but not all!:
https://github.com/pantsbuild/pex/blob/f931befd5ed96d343f2182f34c403bc8bfb188fe/pex/tools/commands/venv.py#L191-L196

[jsirois]

In the PEX case, the RECORD is different with - apparently - one too few ..:

$ grep share nbconvert.pex.venv/lib/python3.10/site-packages/nbconvert-6.4.2.dist-info/RECORD | head
../../share/jupyter/nbconvert/templates/asciidoc/conf.json,sha256=YtRONxl919kXwtomCBZ-CM4p-uOFDOCk9SnmFThA_Qg,87
../../share/jupyter/nbconvert/templates/asciidoc/index.asciidoc.j2,sha256=ldvuvwVyz0b7bO2ZZ8NXcnI4cdyKJBfslWnd_xZLZKc,2298
../../share/jupyter/nbconvert/templates/base/celltags.j2,sha256=_Gi6rqD8HQUQZgzuRRSVqDJ-J5rD0Z9lmPswq2kW6mc,213
../../share/jupyter/nbconvert/templates/base/display_priority.j2,sha256=wEMM_359LWVzWQpz8GzYXeWkzQGA3Mqwp_pRiYCgUhw,1442
../../share/jupyter/nbconvert/templates/base/jupyter_widgets.html.j2,sha256=3vEiL0FN_Nni6SCNlj-jLpukRyPGRY2NUzj-JgJhBFs,1256
../../share/jupyter/nbconvert/templates/base/mathjax.html.j2,sha256=vmg4jv_ICDmQbQiN40X4V_GIfDlxNTusgfG2Qv6HpB0,1198
../../share/jupyter/nbconvert/templates/base/null.j2,sha256=Fw8IIQZnAa6RcI30Vjy6YPDLkFX44glRQTTjEc-PfrI,6229
../../share/jupyter/nbconvert/templates/basic/conf.json,sha256=74s0_-_38TnwNLy8xIjys_J_acbOEGRu8DKNUftxMHk,87
../../share/jupyter/nbconvert/templates/basic/index.html.j2,sha256=ovChqOqeUNBJMBSvRCDPBvF6wJVhDPPF5ly3Y-qPs6g,39
../../share/jupyter/nbconvert/templates/classic/base.html.j2,sha256=w01A7PQw2sLpKCl1tycqhAHl9NTTYir9W9Myi-hZZ6Y,7499

Pex will need to adjust for that assuming that's a consistent result.

[jsirois]

The RECORD being different / wrong for --target mode, which Pex uses, is known: pypa/pip#7658

The --prefix mode records a valid RECORD so maybe Pex wheel install chroots should use this scheme and adjustments be made to the runtime.

[jsirois]

As usual, the variety present in the PyPA ecosystem made this interesting.