Custom Python toolchain for building rules_docker targets

[mvgijssel]

🐞 bug report

Affected Rule

All rules using a binary created by the py_binary rule, like download_pkgs and container_layer.

Is this a regression?

Didn't verify this.

Description

I've installed https://github.com/asdf-vm/asdf on my local machine to handle multiple languages for multiple projects. When I try to build anything with rules_docker I get an error from asdf:

❯ bazel build //tools/python:python_pkgs_image
unknown command: python3. Perhaps you have to reshim?

This means that the HOME variable changed and asdf can't find the Python installation. This particular case is fixed by passing in additional action_env when running bazel:

bazel build --action_env="ASDF_DATA_DIR=/Users/user/.asdf" --action_env="ASDF_PYTHON_VERSION=3.9.7" //tools/python:python_pkgs_image

I don't want the build to rely on an externally installed version of Python, but I want the build system to be responsible for compiling Python itself. There are tons of articles on this topic, so far so good.

The problem I'm running into is to get rules_docker to use a Bazel toolchain registered Python interpreter for building docker images. I've tried https://github.com/digital-plumbers-union/rules_pyenv and manually setting up a Python toolchain like:

# tools/python/BUILD

load("@rules_python//python:defs.bzl", "py_runtime_pair")

py_runtime(
    name = "my_py3_runtime",
    interpreter_path = "/Users/user/.asdf/installs/python/3.9.7/bin/python",
    python_version = "PY3",
)

py_runtime_pair(
    name = "my_py_runtime_pair",
    py2_runtime = None,
    py3_runtime = ":my_py3_runtime",
)

toolchain(
    name = "my_toolchain",
    toolchain = ":my_py_runtime_pair",
    toolchain_type = "@rules_python//python:toolchain_type",
)

# WORKSPACE

register_toolchains("//tools/python:my_toolchain")

It doesn't seem to matter where or how I register the Python toolchain. The registered toolchain works as expected for locally built py_binary targets, but does not work for rules_docker. I know this because when I remove the --action_env flags from the build command when building a rules_docker target, I get the same error unknown command: python3. Perhaps you have to reshim?.

🔬 Minimal Reproduction

1. Install asdf
2. Install asdf bazel plugin asdf plugin-add bazel https://github.com/mrinalwadhwa/asdf-bazel.git
3. Install asdf python plugin asdf plugin-add python
4. Clone repository https://github.com/mvgijssel/test-rules_docker
5. Install bazel 5.0.0 version using asdf install
6. bazel build //:test_pkgs
7. Expect unknown command: python3. Perhaps you have to reshim? in the output

🔥 Exception or Error

❯ bazel build --define=VERBOSE_LOGS=1 //tools/python:python_pkgs_image
INFO: Build option --define has changed, discarding analysis cache.
INFO: Analyzed target //tools/python:python_pkgs_image (0 packages loaded, 441 targets configured).
INFO: Found 1 target...
ERROR: /Users/user/bazel-test/tools/python/BUILD:30:14: Action tools/python/python_pkgs.tar failed: (Exit 1): python_pkgs.sh failed: error executing command bazel-out/darwin-fastbuild/bin/tools/python/python_pkgs.sh

Use --sandbox_debug to see verbose messages from the sandbox
unknown command: python3. Perhaps you have to reshim?
Target //tools/python:python_pkgs_image failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.471s, Critical Path: 0.10s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully

🌍 Your Environment

Operating System:

  
macOS Monterey, 12.2
  

Output of bazel version:

  
❯ bazel --version
bazel 5.0.0
  

Rules_docker version:

  
http_archive(
    name = "io_bazel_rules_docker",
    sha256 = "85ffff62a4c22a74dbd98d05da6cf40f497344b3dbf1e1ab0a37ab2a1a6ca014",
    strip_prefix = "rules_docker-0.23.0",
    urls = ["https://github.com/bazelbuild/rules_docker/releases/download/v0.23.0/rules_docker-v0.23.0.tar.gz"],
)
  

Anything else relevant?

This issue seems to touch on similar topics #1680

[yibum]

This seems to be related to bazelbuild/rules_python#691 (comment)
bazel itself requires system install python for some bootstraping.

[github-actions]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days.
Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_docker!

[github-actions]

This issue was automatically closed because it went 30 days without a reply since it was labeled "Can Close?"