Windows, Python: --python_path default value is useless on Windows

[laszlocsomor]

Description of the problem / feature request / question:

The default value of --python_path is python. This only works on Linux/macOS, however not because on Windows the binary is python.exe but because on Windows it's not on the $PATH.

When Bazel runs a py_binary as an action's executable (e.g. as part of a genrule), the action's $PATH is not the same as client's but a sanitized one (for sake of reproducibility).
On Linux/macOS, this sanitized $PATH includes /usr/bin, which contains python.
On Windows, this $PATH contains the Windows and the MSYS directory, neither of which has python.

Therefore to run Python tools in build actions we must pass the --python_path flag to Bazel.
I get bitten by this every time I try to build the Android Hello World project.

Proposed solution

The Bazel client could detect the location of python.exe (by looking at the $PATH) and add the --python_path flag to the server's command line, like we do with --windows_unix_root. (Caveat: the --python_path flag is a command flag, not a startup flag, and the client should only pass it if the command is "build", "test", etc., i.e. commands where this flag is valid.)

Environment info

• Operating System: Windows 10

• Bazel version (output of bazel info release): HEAD

[laszlocsomor]

Symptom: I get a launcher error when I don't use --python_path:

C:\work\bazel>C:/users/laszlocsomor/appdata/local/temp/_bazel_laszlocsomor/o6hbs7n0/execroot/io_bazel/bazel-out/msvc_x64-fastbuild/bin/src/bazel.exe --output_user_root=c:\tmp2 build //examples/android/java/bazel:hello_world --verbose_failures
WARNING: The major revision of the Android NDK referenced by android_ndk_repository rule 'androidndk' is 15. The major revisions supported by Bazel are [10, 11, 12, 13, 14]. Defaulting to revision 14.
WARNING: API level 26 specified by android_ndk_repository 'androidndk' is not available. Using latest known API level 25.
INFO: Found 1 target...
ERROR: C:/work/bazel/examples/android/java/bazel/BUILD:12:1: Extracting Java resources from deploy jar for split Java resource apk failed (Exit 2): resource_extractor.exe failed: error executing command
  cd C:/tmp2/o6hbs7n0/execroot/io_bazel
bazel-out/host/bin/external/bazel_tools/tools/android/resource_extractor.exe bazel-out/msvc_x64-fastbuild/bin/examples/android/java/bazel/hello_world_deploy.jar bazel-out/msvc_x64-fastbuild/bin/examples/android/java/bazel/_dx/hello_world/extracted_hello_world_deploy.jar.
LAUNCHER ERROR: Cannot launch process:
(error: 2): The system cannot find the file specified.

[meteorcloudy]

I am trying to reproduce this on my machine, but got

ERROR: C:/tools/msys64/home/pcloudy/workspace/bazel/examples/android/java/bazel/BUILD:33:1: C++ compilation of rule '//examples/android/java/bazel:jni_dep' failed (Exit -1). Note: Remote connection/protocol failed with: execution failed: clang failed: error executing command
  cd C:/tmp/_bazel_pcloudy/7uxoax_v/execroot/io_bazel
  SET PATH=C:\tools\msys64\usr\bin;C:\tools\msys64\bin;C:\Program Files\CMake\bin;C:\Program Files\Anaconda3;C:\tools\msys64\home\pcloudy\bin;C:\Program Files\Java\jdk1.8.0_77\bin;C:\python_27_amd64\files;C:\Program Files\CMake\bin;C:\Program Files\Anaconda3;C:\tools\msys64\home\pcloudy\bin;C:\Program Files\Java\jdk1.8.0_77\bin;C:\tools\msys64\usr\local\bin;C:\tools\msys64\usr\bin;C:\tools\msys64\usr\bin;C:\tools\msys64\opt\bin;C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\tools\msys64\usr\bin\site_perl;C:\tools\msys64\usr\bin\vendor_perl;C:\tools\msys64\usr\bin\core_perl
    SET PWD=/proc/self/cwd
    SET TEMP=C:/tmp
    SET TMPDIR=C:\tmp
  external/androidndk/ndk/toolchains/llvm/prebuilt/windows-x86_64/bin/clang -gcc-toolchain external/androidndk/ndk/toolchains/arm-linux-androideabi-4.9/prebuilt/windows-x86_64 -fpic -ffunction-sections -funwind-tables -fstack-protector-strong -Wno-invalid-command-line-argument -Wno-unused-command-line-argument -no-canonical-prefixes -fno-integrated-as -target armv7-none-linux-androideabi -march=armv7-a -mfloat-abi=softfp -mfpu=vfpv3-d16 -w -MD -MF bazel-out/android-arm-linux-androideabi-clang3.8-v7a-gnu-libstdcpp-fastbuild/bin/examples/android/java/bazel/_objs/jni_dep/examples/android/java/bazel/jni_dep.d -frandom-seed=bazel-out/android-arm-linux-androideabi-clang3.8-v7a-gnu-libstdcpp-fastbuild/bin/examples/android/java/bazel/_objs/jni_dep/examples/android/java/bazel/jni_dep.o -iquote . -iquote bazel-out/android-arm-linux-androideabi-clang3.8-v7a-gnu-libstdcpp-fastbuild/genfiles -iquote external/bazel_tools -iquote bazel-out/android-arm-linux-androideabi-clang3.8-v7a-gnu-libstdcpp-fastbuild/genfiles/external/bazel_tools -isystem external/bazel_tools/tools/cpp/gcc3 --sysroot=external/androidndk/ndk/platforms/android-24/arch-arm -isystem external/androidndk/ndk/sources/cxx-stl/gnu-libstdc++/4.9/include -isystem external/androidndk/ndk/sources/cxx-stl/gnu-libstdc++/4.9/libs/armeabi-v7a/include -isystem external/androidndk/ndk/sources/cxx-stl/gnu-libstdc++/4.9/include/backward -c examples/android/java/bazel/jni_dep.cc -o bazel-out/android-arm-linux-androideabi-clang3.8-v7a-gnu-libstdcpp-fastbuild/bin/examples/android/java/bazel/_objs/jni_dep/examples/android/java/bazel/jni_dep.o
Action failed to execute: java.io.IOException: CreateProcess(): The system cannot find the file specified.

I'm using Bazel@HEAD, but it doesn't seem to use MSVC as the C++ compiler, how should I resolve this?

[dslomov]

It does not use MSVC as C++ compiler because it builds for Android

[laszlocsomor]

Maybe you have a bazelrc file that Bazel is picking up? Try using --bazelrc=NUL.

Here's what I did:

1. Checked out 98c831ec11d8bea47bcafce69a22df716381da5a
2. Added this to my WORKSPACE:

   android_sdk_repository(
     name = "androidsdk",
     path = "c:\\Users\\laszlocsomor\\AppData\\Local\\Android\\sdk",
   )
   
   android_ndk_repository(
     name = "androidndk",
     path = "c:\\Users\\laszlocsomor\\AppData\\Local\\Android\\sdk\\ndk-bundle",
   )
   

3. built Bazel
4. used the binary to build the Android project:

   C:\work\bazel>C:/users/laszlocsomor/appdata/local/temp/_bazel_laszlocsomor/o6hbs7n0/execroot/io_bazel/bazel-out/msvc_x64-fastbuild/bin/src/bazel.exe --bazelrc=NUL --output_user_root=c:\tmp2 build //examples/android/java/bazel:hello_world --verbose_failures
   (...)
   ERROR: C:/work/bazel/examples/android/java/bazel/BUILD:12:1: Extracting Java resources from deploy jar for split Java resource apk failed (Exit 2): resource_extractor.exe failed: error executing command
     cd C:/tmp2/o6hbs7n0/execroot/io_bazel
   bazel-out/host/bin/external/bazel_tools/tools/android/resource_extractor.exe bazel-out/msvc_x64-fastbuild/bin/examples/android/java/bazel/hello_world_deploy.jar bazel-out/msvc_x64-fastbuild/bin/examples/android/java/bazel/_dx/hello_world/extracted_hello_world_deploy.jar.
   LAUNCHER ERROR: Cannot launch process:
   (error: 2): The system cannot find the file specified.
   
   Target //examples/android/java/bazel:hello_world failed to build
   

[meteorcloudy]

Got it, my NDK repo had some problem, now it's working! Thanks!

[meteorcloudy]

Hmm,, now I have some doubt on the proposed solution, in order to know when to pass --python_path from client, we not only need to know which type of Bazel command is running (should only be build, test, run), but also needs to know whether --python_path is already specified by user. The requires parsing all command line arguments AND arguments read from bazelrc files, which appears in a strange format --default_override=1:build=--python_path=C:/a/b/c/python.exe. It feels a bit too hacky to me.

Instead, how about having a @local_python_config rule? We can use skylark repository rule to configure a py_runtime rule. (py_runtime is introduced in a5c3862)

@erain Do we have a plan to add a default py_runtime in Bazel?

[laszlocsomor]

That's a good point.

The @local_python_config rule sounds promising. Can you say more specifically how you'd use it?

Alternatively, is it possible to pass the --python_path flag before the user- and bazelrc-specified ones?

[meteorcloudy]

Just like the @local_config_cc repo, we configure a python runtime rule using skylark, then make --python_top points to @local_config_py by default. Users can override it by passing --python_path or having --python_top point to another location.

Do you mean adding --python_path before the user- and bazelrc-specified ones will causing it being override?

[meteorcloudy]

Ah, I just realized we can pretend we are adding --python_path in the least important rc file by adding --default_override=0:build=--python_path=<python/path>. Just like https://github.com/bazelbuild/bazel/blob/master/src/main/cpp/option_processor.cc#L504

This way it will be overridden by any --python_path option from both command line and rc files. And we don't even need to add it before them.

[dslomov]

I do prefer very strongly a principled solution with @local_config_py

(We can add command line hackery in the meanwhile if we need to for some clients)

[laszlocsomor]

I also prefer a principled solution, and @local_config_py sounds like it while adding the flag doesn't. But as a short term solution I'm also OK with it, hopefully it won't make it into a release :)

[Warchant]

In release 4.1.0 python_path is counted in configuration - which leads to different configurations in projects which are otherwise exactly same, but have different absolute path to python.exe on Windows. This impacts Bazel Remote Cache hit ratio - 100% cache hit turns into 0% cache hit, due to different python installations on Windows...

Should python_path be counted in configuration?