filtering out -mcpu=native when building with Bazel on Arm 64-bit (aarch64)

[boegel]

I'm hitting problems with building XNNPACK during the build of TensorFlow on Arm 64-bit, when using bazel build --copt="-mcpu=native" ...:

  /mnt/shared/home/boegel/easybuild/graviton2/software/GCCcore/12.2.0/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections -MD -MF bazel-out/aarch64-opt/bin/external/XNNPACK/_objs/asm_microkernels/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.pic.d -fPIC '-DBAZEL_CURRENT_REPOSITORY="XNNPACK"' -iquote external/XNNPACK -iquote bazel-out/aarch64-opt/bin/external/XNNPACK -isystem external/XNNPACK/include -isystem bazel-out/aarch64-opt/bin/external/XNNPACK/include -isystem external/XNNPACK/src -isystem bazel-out/aarch64-opt/bin/external/XNNPACK/src -Wno-all -Wno-extra -Wno-deprecated -Wno-deprecated-declarations -Wno-ignored-attributes -Wno-array-bounds -Wunused-result '-Werror=unused-result' -Wswitch '-Werror=switch' '-Wno-error=unused-but-set-variable' -DAUTOLOAD_DYNAMIC_KERNELS -O2 -ftree-vectorize '-mcpu=native' -fno-math-errno -fPIC -fPIC -Iinclude -Isrc '-march=armv8.2-a+fp16+dotprod' -O2 -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S -o bazel-out/aarch64-opt/bin/external/XNNPACK/_objs/asm_microkernels/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.pic.o)
# Configuration: 0d4560b813c94b91f71e59ae9e182e4612b959f02afc2e8b9422054dd2bd321f
# Execution platform: @local_execution_config_platform//:platform
cc1: warning: switch '-mcpu=zeus+crypto+sha3+sm4+nodotprod+noprofile+nopauth' conflicts with '-march=armv8.2-a+fp16+dotprod' switch
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S: Assembler messages:
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:54: Error: selected processor does not support `sdot v28.4s,v16.16b,v0.4b[0]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:56: Error: selected processor does not support `sdot v29.4s,v17.16b,v0.4b[0]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:58: Error: selected processor does not support `sdot v30.4s,v18.16b,v0.4b[0]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:60: Error: selected processor does not support `sdot v31.4s,v19.16b,v0.4b[0]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:62: Error: selected processor does not support `sdot v28.4s,v4.16b,v0.4b[1]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:64: Error: selected processor does not support `sdot v29.4s,v5.16b,v0.4b[1]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:66: Error: selected processor does not support `sdot v30.4s,v6.16b,v0.4b[1]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:68: Error: selected processor does not support `sdot v31.4s,v7.16b,v0.4b[1]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:125: Error: selected processor does not support `sdot v28.4s,v16.16b,v0.4b[0]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:127: Error: selected processor does not support `sdot v29.4s,v17.16b,v0.4b[0]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:129: Error: selected processor does not support `sdot v30.4s,v18.16b,v0.4b[0]'
external/XNNPACK/src/qc8-gemm/gen/qc8-gemm-1x16c4-minmax-fp32-asm-aarch64-neondot-ld64.S:131: Error: selected processor does not support `sdot v31.4s,v19.16b,v0.4b[0]'
SUBCOMMAND: # @XNNPACK//:asm_microkernels [action 'Compiling src/qc8-gemm/gen/qc8-gemm-4x16-minmax-fp32-asm-aarch64-neon-mlal-lane-cortex-a53.S', configuration: 0d4560b813c94b91f71e59ae9e182e4612b959f02afc2e8b9422054dd2bd321f, execution platform: @local_execution_config_platform//:platform]

(see also easybuilders/easybuild-easyconfigs#18899)

The problem here is clear: -mcpu=native is expanded by GCC to -mcpu=zeus+crypto+sha3+sm4+nodotprod+noprofile+nopauth, which clashes with the -march=armv8.2-a+fp16+dotprod that is used when building one of the XNNPACK kernels.

As mentioned in #1884 this can be fixed by simply not using -mcpu when building XNNPACK.
This makes sense, but I do want to use -mcpu=native when building TensorFlow (and the other external libraries it needs)...

Is there a way that -mcpu=native can be filtered out by XNNPACK itself when it got passed in via bazel build --copt, for example by modifying xnnpack_cc_library in build_defs.bzl somehow?

I'm not very familiar with Bazel myself, but I couldn't figure out a way after searching around for a couple of hours.
The only option I saw was using nocopts, but apparently this is no longer supported even though the Bazel documentation still mentions it (see bazelbuild/bazel#8706)...

[ngzhian]

Maybe https://bazel.build/docs/user-manual#per-file-copt ?
use the regex to exclude XNNPack files from -mcpu=native.

[boegel]

@ngzhian Thanks a lot for the suggestion, that's working like a charm: by using --per_file_copt='-.*XNNPACK/.*@-mcpu=native' instead of --copt='-mcpu=native' I can avoid that -mcpu=native is used when building XNNPACK, while it's still being used for everything else 👍