devcontainer tensorflow build problem when running container on ARM64

[juliays]

"cargo build --workspace" failed after starting devcontainer on Macbook with M1 chip.

Expected Behavior

Apps defined in cargo.toml [workspace] section should build

Current Behavior

Error when building tensorflow-sys. Error message is copied below -

vscode ➜ /workspaces/chariott (main ✗) $ cargo build --workspace
Blocking waiting for file lock on package cache
Blocking waiting for file lock on package cache
Blocking waiting for file lock on package cache
Blocking waiting for file lock on build directory
Compiling tensorflow-sys v0.22.1
Compiling mock-vas v0.1.0 (/workspaces/chariott/examples/applications/mock-vas)
Compiling kv-app v0.1.0 (/workspaces/chariott/examples/applications/kv-app)
Compiling cloud-object-detection-app v0.1.0 (/workspaces/chariott/examples/applications/cloud-object-detection)
Compiling chariott v0.1.0 (/workspaces/chariott)
Compiling lt-provider-app v0.1.0 (/workspaces/chariott/examples/applications/lt-provider)
error: failed to run custom build command for tensorflow-sys v0.22.1

Caused by:
process didn't exit successfully: /workspaces/chariott/target/debug/build/tensorflow-sys-6b5908438712dc75/build-script-build (exit status: 1)
--- stdout
cargo:rerun-if-env-changed=TENSORFLOW_NO_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG_aarch64-unknown-linux-gnu
cargo:rerun-if-env-changed=PKG_CONFIG_aarch64_unknown_linux_gnu
cargo:rerun-if-env-changed=HOST_PKG_CONFIG
cargo:rerun-if-env-changed=PKG_CONFIG
cargo:rerun-if-env-changed=TENSORFLOW_STATIC
cargo:rerun-if-env-changed=TENSORFLOW_DYNAMIC
cargo:rerun-if-env-changed=PKG_CONFIG_ALL_STATIC
cargo:rerun-if-env-changed=PKG_CONFIG_ALL_DYNAMIC
cargo:rerun-if-env-changed=PKG_CONFIG_PATH_aarch64-unknown-linux-gnu
cargo:rerun-if-env-changed=PKG_CONFIG_PATH_aarch64_unknown_linux_gnu
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_PATH
cargo:rerun-if-env-changed=PKG_CONFIG_PATH
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_aarch64-unknown-linux-gnu
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR_aarch64_unknown_linux_gnu
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_LIBDIR
cargo:rerun-if-env-changed=PKG_CONFIG_LIBDIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_aarch64-unknown-linux-gnu
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR_aarch64_unknown_linux_gnu
cargo:rerun-if-env-changed=HOST_PKG_CONFIG_SYSROOT_DIR
cargo:rerun-if-env-changed=PKG_CONFIG_SYSROOT_DIR
tensorflow-sys/build.rs:309: output = "/workspaces/chariott/target/debug/build/tensorflow-sys-090d29f4eb498d4e/out"
tensorflow-sys/build.rs:311: source = "/usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tensorflow-sys-0.22.1/target/source-v2.9.1"
tensorflow-sys/build.rs:313: lib_dir = "/workspaces/chariott/target/debug/build/tensorflow-sys-090d29f4eb498d4e/out/lib-v2.9.1"
tensorflow-sys/build.rs:315: Directory "/workspaces/chariott/target/debug/build/tensorflow-sys-090d29f4eb498d4e/out/lib-v2.9.1" already exists
tensorflow-sys/build.rs:322: framework_library_path = "/workspaces/chariott/target/debug/build/tensorflow-sys-090d29f4eb498d4e/out/lib-v2.9.1/libtensorflow_framework.so.2"
tensorflow-sys/build.rs:325: library_path = "/workspaces/chariott/target/debug/build/tensorflow-sys-090d29f4eb498d4e/out/lib-v2.9.1/libtensorflow.so.2"
tensorflow-sys/build.rs:442: Executing "bazel" "version"
cargo:error=Bazel must be installed at version 3.7.2 or greater. (Error: No such file or directory (os error 2))
warning: build failed, waiting for other jobs to finish...
error: linking with cc failed: exit status: 1

Steps to Reproduce

1. Get an ARM based laptop
2. Comment out line 48 in ./.devcontainer/Dockerfile. This gets you around issue
3. Start devcontainer
4. In bash window, run "cargo build --workspace"

Context (Environment)

tensorflow crate needs to be rebuilt on non x86-64 linux or Mac

Logs

Additional Information

[juliays]

I followed tensorflow instruction and created a script to install bazel on arm machines. However, I'm getting an error when doing the install. Filed a bug with bazel.

Here is the script that does the bazel install in case it could be helpful.

#!/usr/bin/env bash
#-------------------------------------------------------------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# Licensed under the MIT License. See https://go.microsoft.com/fwlink/?linkid=2090316 for license information.
#-------------------------------------------------------------------------------------------------------------
#
architecture=""
case $(uname -m) in
    i386 | i686)   architecture="386" ;;
    x86_64) architecture="amd64" ;;
    arm | arm64 | aarch64)    dpkg --print-architecture | grep -q "arm64" && architecture="arm64" || architecture="arm" ;;
esac

echo "chip is " $architecture

if [[ $architecture == *"arm"* ]]; then
    # install prerequisite to build tensorflow dependency for cloud-object-detection
    sudo apt install python3-pip python3-dev
    pip install -U pip numpy wheel packaging requests opt_einsum
    pip install -U keras_preprocessing --no-deps

    sudo apt install g++ unzip zip -y
    sudo apt-get install openjdk-11-jdk -y
    wget https://github.com/bazelbuild/bazel/releases/download/5.3.2/bazel-5.3.2-installer-darwin-arm64.sh
    chmod +x bazel-5.3.2-installer-darwin-arm64.sh
    ./bazel-5.3.2-installer-darwin-arm64.sh --user
    #wget https://github.com/bazelbuild/bazel/releases/download/4.2.3/bazel-4.2.3-installer-darwin-arm64.sh
    #chmod +x bazel-4.2.3-installer-darwin-arm64.sh
    #./bazel-4.2.3-installer-darwin-arm64.sh --user
    export PATH="$PATH:$HOME/bin"
fi 

[danigian]

@juliays I am not sure whether you already had a look at it but, in this issue, someone figured out how to make this work on M1 and posted a gist

[stepknees]

Not core chariott functionality but needed to run dogmode app. Needs more investigation into dependencies. Needs MacOS community user to help. #helpwanted #backlog

[Prem-Kumar16]

I got the exact same error in my ec2 instance with arm64 architecture.

I am not sure whether the solution that I have used will work for Mac with M1 chip, but it worked for me (I am using ec2 instance of arm64 architecture running Debian 12)

Install bazelisk (A user friendly launcher for bazel)
wget https://github.com/bazelbuild/bazelisk/releases/download/v1.18.0/bazelisk-linux-arm64

Change the downloaded file as an executable
chmod +x bazelisk-linux-arm64
sudo mv bazelisk-linux-arm64 /usr/local/bin/bazel

Please note that the commands given above are used by me in my machine. I haven't used Mac before, so please change it accordingly for your machines and architectures.

The reference links are:
https://stackoverflow.com/questions/65656165/how-do-you-install-bazel-using-bazelisk
https://github.com/bazelbuild/bazelisk
https://github.com/bazelbuild/bazelisk/releases

I hope this helps.