Build problem [cause: missing require('MarkupSafe') on Jinja2-using targets]

[renau]

I run in a fresh ubuntu:20.04 docker, and it fails to build.

To get the fresh docker ubuntu 20.04

docker pull ubuntu:20.04
docker run -it ubuntu:20.04

My steps inside the docker:

apt update
apt install g++
apt install python3-dev python3-distutils python3-dev libtinfo5

After installing the bazel (using the link from xls website)

bazel --version
bazel 3.4.1
gcc --version
gcc (Ubuntu 9.3.0-10ubuntu2) 9.3.0

In addition to the python packages, I also had to install this one (build fails earlier otherwise)

apt install python3-markupsafe

I run bazel build -c dbg -s ... (-s is to show failing command)

I get this error:

SUBCOMMAND: # @com_google_protobuf//:protoc_lib [action 'Compiling external/com_google_protobuf/src/google/protobuf/compiler/java/java_field.cc [for host]', configuration: d4adb9d4b74a212f04db0ab495d908331bdd7b5383b77684a9910d74094a0e62, execution platform: @local_config_platform//:host]
(cd /home/user/.cache/bazel/_bazel_user/6ce1e51cd667cc5c934d2234feeac00b/execroot/com_google_xls && \
  exec env - \
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/pstill_dist:/home/user/bin:/usr/local/bin:/usr/NX/bin:/opt/CodeAnalyst/bin:/opt/CodeAnalyst/sbin:/home/user/projs/esesc/conf/scripts:/home/renau/.cabal/bin:/home/user/.rvm/bin:/home/renau/projs/mips/install-mips-img-linux-gnu/bin:/home/renau/projs/mips/install-mips-img-elf/bin:/home/renau/projs/mips/depot_tools:/soe/renau/projs/pyrope/parser/bin \
    PWD=/proc/self/cwd \
  /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++17' -MD -MF bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_field.d '-frandom-seed=bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_field.o' -iquote external/com_google_protobuf -iquote bazel-out/host/bin/external/com_google_protobuf -iquote external/zlib -iquote bazel-out/host/bin/external/zlib -isystem external/com_google_protobuf/src -isystem bazel-out/host/bin/external/com_google_protobuf/src -isystem external/zlib/zlib/include -isystem bazel-out/host/bin/external/zlib/zlib/include -g0 -g0 -DHAVE_PTHREAD -DHAVE_ZLIB -Woverloaded-virtual -Wno-sign-compare -Wno-unused-function -Wno-write-strings -Wno-deprecated-declarations -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/com_google_protobuf/src/google/protobuf/compiler/java/java_field.cc -o bazel-out/host/bin/external/com_google_protobuf/_objs/protoc_lib/java_field.o)
ERROR: /home/user/projs/synth/xls/xls/ir/BUILD:642:8: Executing genrule //xls/ir:gen_op_proto failed (Exit 127) bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)

[cdleary]

Thanks for reporting Jose, here is a Dockerfile somebody told me worked successfully for them:

#Download base image ubuntu 20.04
FROM ubuntu:20.04

# LABEL about the custom image
LABEL version="0.1"
LABEL description="Docker Image for Building and Executing XLS."

# Update Ubuntu Software repository
RUN apt update -y

# Install Bazel
RUN apt install -y curl gnupg && \
curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor > bazel.gpg && \
mv bazel.gpg /etc/apt/trusted.gpg.d/ && \
echo "deb [arch=amd64] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list && \
apt update -y && apt install -y bazel

# Install dependencies
RUN apt -y install python3-dev python3-distutils python3-dev libtinfo5

# Setup
RUN mkdir -p $HOME/opt/bin/ && \
ln -s $(which python3) $HOME/opt/bin/python && \
echo 'export PATH=$HOME/opt/bin:$PATH' >> ~/.bashrc

# Install Other
RUN apt install -y python3-pip git && \
pip3 install jinja2 && \
apt install -y vim

Should be able to check out the particular issue you're seeing more in the afternoon, but LMK if the dockerfile works! We'll be sure to add some canonical instructions to the documentation as well.

[cdleary]

Just guessing, it may have been the python3 path issue, you have to set it up in the way described here: https://google.github.io/xls/#building-from-source because Bazel currently makes some assumptions about the Python environment, I believe this issue is the latest status: bazelbuild/bazel#8685

[cdleary]

Thanks Jose, that commit should have fixed the issue, and there's a Dockerfile in the root directory now so you should be able to docker build . in the project checkout. LMK if you run into any issues!