Multi-platform PEXes

[jsirois]

@dimaqq, re: https://discuss.python.org/t/multi-arch-venv-sound-or-bad-idea/80510, does this accomplish what you want?:

# Step 1: Define the remote platforms you want the venv PEX to support.
# This is a one time step per target platform.
# Here we define 2:
:; touch python3.13-linux-aarch64.complete-platform.json && \
docker run --rm -it --platform linux/aarch64 -v $PWD/python3.13-linux-aarch64.complete-platform.json:/complete-platform.json python:3.13 bash -c '
  python -mvenv .venv && \
  .venv/bin/pip install pex && \
  .venv/bin/pex3 interpreter inspect --markers --tags --indent 2 -o /complete-platform.json
'

:; touch python3.12-linux-x86_64.complete-platform.json && \
docker run --rm -it --platform linux/x86_64 -v $PWD/python3.12-linux-x86_64.complete-platform.json:/complete-platform.json python:3.12 bash -c '
  python -mvenv .venv && \
  .venv/bin/pip install pex && \
  .venv/bin/pex3 interpreter inspect --markers --tags --indent 2 -o /complete-platform.json
'

# Build a multiplatform PEX for the two foreign platforms defined above plus a local interpreter too:
:; cat exe.py 
import cowsay
import requests
from charset_normalizer import md


print(cowsay.tux(requests.get("https://echo.free.beeceptor.com").text))
print(cowsay.__file__)
print(md.__file__)
print(requests.__file__)

:; pex \
    --python python3.11 \
    --complete-platform python3.12-linux-x86_64.complete-platform.json \
    --complete-platform python3.13-linux-aarch64.complete-platform.json \
    --sh-boot \
    --venv prepend \
    cowsay \
    requests \
    --exe exe.py \
    -o cowsay.pex

# Inspect the PEX. Note it has 3 copies of the charset_normalizer wheel to satisfy each of the 3 platforms the PEX is built for optimally:
:; PEX_TOOLS=1 ./cowsay.pex info | jq .distributions
{
  "certifi-2025.1.31-py3-none-any.whl": "f9c221888099d904cd3e04587bcad01a8e99187478e5e8a00a6fd8d500c0a8d6",
  "charset_normalizer-3.4.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "526e6edd211ca84ecee0e77deada0e0c4d15623da0efc1ecfa2eff0ec3601132",
  "charset_normalizer-3.4.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl": "0baa50ca65f7aa3cdfd5b577d9f7a10f3af865869b385f05a895016dd6361641",
  "charset_normalizer-3.4.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl": "91a1e60d45cbc8bfe46d81def5168a670e9f48612e5c5a339cec43a5b5d22039",
  "cowsay-6.1-py3-none-any.whl": "fa6f4a84c8762cbb487fd86b7808a641f9909bb3d30eecdc8d63687eef5d1305",
  "idna-3.10-py3-none-any.whl": "7c60d05cda1bf40836124b2018f4bbbfcab10a8c761890952f6cb9ae3fe7a77a",
  "requests-2.32.3-py3-none-any.whl": "2f57f4ae5920a2f5f55a8f4971fe010b00cb2a30f8ab9c572826150fa309c450",
  "urllib3-2.3.0-py3-none-any.whl": "ea05fa117acceb3a6e7a9517434486c76ba02dba03e5770453c46204f79bbea9"
}

# Now exercise the PEX on each of the 3 target platforms, noting the venv it creates for each is tailored to that platform:
:; docker run --rm -it --platform linux/aarch64 -v $PWD/cowsay.pex:/cowsay.pex python:3.13 /cowsay.pex
  _______________________________________
 /                                       \
| {                                       |
| "method": "GET",                        |
| "protocol": "https",                    |
| "host": "echo.free.beeceptor.com",      |
| "path": "/",                            |
| "ip": "174.62.125.224:41802",           |
| "headers": {                            |
| "Host": "echo.free.beeceptor.com",      |
| "User-Agent": "python-requests/2.32.3", |
| "Accept": "*/*",                        |
| "Accept-Encoding": "gzip, deflate"      |
| },                                      |
| "parsedQueryParams": {}                 |
| }                                       |
 \                                       /
  =======================================
                                            \
                                             \
                                              \
                                               .--.
                                              |o_o |
                                              |:_/ |
                                             //   \ \
                                            (|     | )
                                           /'\_   _/`\
                                           \___)=(___/
None
/root/.cache/pex/venvs/1/7906d7606c8f6a80c434e31bfc262bcd92841304/bf3f69efadcf38174323cd831de7582de699efe9/lib/python3.13/site-packages/cowsay/__init__.py
/root/.cache/pex/venvs/1/7906d7606c8f6a80c434e31bfc262bcd92841304/bf3f69efadcf38174323cd831de7582de699efe9/lib/python3.13/site-packages/charset_normalizer/md.cpython-313-aarch64-linux-gnu.so
/root/.cache/pex/venvs/1/7906d7606c8f6a80c434e31bfc262bcd92841304/bf3f69efadcf38174323cd831de7582de699efe9/lib/python3.13/site-packages/requests/__init__.py

:; docker run --rm -it --platform linux/x86_64 -v $PWD/cowsay.pex:/cowsay.pex python:3.12 python /cowsay.pex
  _______________________________________
 /                                       \
| {                                       |
| "method": "GET",                        |
| "protocol": "https",                    |
| "host": "echo.free.beeceptor.com",      |
| "path": "/",                            |
| "ip": "174.62.125.224:51736",           |
| "headers": {                            |
| "Host": "echo.free.beeceptor.com",      |
| "User-Agent": "python-requests/2.32.3", |
| "Accept": "*/*",                        |
| "Accept-Encoding": "gzip, deflate"      |
| },                                      |
| "parsedQueryParams": {}                 |
| }                                       |
 \                                       /
  =======================================
                                            \
                                             \
                                              \
                                               .--.
                                              |o_o |
                                              |:_/ |
                                             //   \ \
                                            (|     | )
                                           /'\_   _/`\
                                           \___)=(___/
None
/root/.cache/pex/venvs/1/7906d7606c8f6a80c434e31bfc262bcd92841304/0bd42d8ec20fd6af7633d7916806c2a7c7b50de1/lib/python3.12/site-packages/cowsay/__init__.py
/root/.cache/pex/venvs/1/7906d7606c8f6a80c434e31bfc262bcd92841304/0bd42d8ec20fd6af7633d7916806c2a7c7b50de1/lib/python3.12/site-packages/charset_normalizer/md.cpython-312-x86_64-linux-gnu.so
/root/.cache/pex/venvs/1/7906d7606c8f6a80c434e31bfc262bcd92841304/0bd42d8ec20fd6af7633d7916806c2a7c7b50de1/lib/python3.12/site-packages/requests/__init__.py

:; ./cowsay.pex
  _______________________________________
 /                                       \
| {                                       |
| "method": "GET",                        |
| "protocol": "https",                    |
| "host": "echo.free.beeceptor.com",      |
| "path": "/",                            |
| "ip": "174.62.125.224:58862",           |
| "headers": {                            |
| "Host": "echo.free.beeceptor.com",      |
| "User-Agent": "python-requests/2.32.3", |
| "Accept": "*/*",                        |
| "Accept-Encoding": "gzip, deflate"      |
| },                                      |
| "parsedQueryParams": {}                 |
| }                                       |
 \                                       /
  =======================================
                                            \
                                             \
                                              \
                                               .--.
                                              |o_o |
                                              |:_/ |
                                             //   \ \
                                            (|     | )
                                           /'\_   _/`\
                                           \___)=(___/
None
/home/jsirois/.cache/pex/venvs/1/7906d7606c8f6a80c434e31bfc262bcd92841304/049002868f99e6ce3c0ba611c840e441c5f69c29/lib/python3.11/site-packages/cowsay/__init__.py
/home/jsirois/.cache/pex/venvs/1/7906d7606c8f6a80c434e31bfc262bcd92841304/049002868f99e6ce3c0ba611c840e441c5f69c29/lib/python3.11/site-packages/charset_normalizer/md.cpython-311-x86_64-linux-gnu.so
/home/jsirois/.cache/pex/venvs/1/7906d7606c8f6a80c434e31bfc262bcd92841304/049002868f99e6ce3c0ba611c840e441c5f69c29/lib/python3.11/site-packages/requests/__init__.py