Generating system library: unknown type name 'atomic_flag'

[cajus]

Version of emscripten/emsdk:

emcc (Emscripten gcc/clang-like replacement + linker emulating GNU ld) 3.1.27 (afa75f342eef4d925172479afa8e6233eb0ae5a9)
clang version 16.0.0 (https://github.com/llvm/llvm-project dee009d3b5c71a340eb4ebb3cfb49f6015a789cc)
Target: wasm32-unknown-emscripten
Thread model: posix
InstalledDir: /home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/bin

When building a library from our toolchain (bzip2 in this case), it starts to generate system libraries when linking. Starting with 3.1.25, I'm getting this:

cache:INFO: generating system library: sysroot/lib/wasm32-emscripten/libcompiler_rt.a... (this will be cached in "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/lib/wasm32-emscripten/libcompiler_rt.a" for subsequent builds)
/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear.c:21:33: error: unknown type name 'atomic_flag'
void atomic_flag_clear(volatile atomic_flag *object) {
                                ^
1 error generated.
emcc: error: '/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/bin/clang -target wasm32-unknown-emscripten -fignore-exceptions -fvisibility=default -mllvm -combiner-global-alias-analysis=false -mllvm -enable-emscripten-sjlj -mllvm -disable-lsr -Werror=implicit-function-declaration -I/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/include/SDL --sysroot=/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot -Xclang -iwithsysroot/include/c++/v1 -iwithsysroot/include/compat -O2 -Werror -fno-unroll-loops -fno-builtin -g3 -c /home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear.c -o /home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/build/libcompiler_rt-tmp/atomic_flag_clear.o' failed (returned 1)

Maybe I need to change something in the build flags, but I can't find any hints for that in the changelog. Any ideas?

[sbc100]

Most likely this is because updates libc++ in #18113 which was released in 3.1.25.

Are you sure the version libc++ in your .emscripten_cache/sysroot is not somehow stale? Can you trying completely removing the sysroot and building from scratch?

[cajus]

Hmm. I'm relative sure that the build is always from scratch (conan recipe), as I set EM_CACHE to be in the package directory. This is created and therefore empty for each build. So this should be exactly what comes in for 3.1.27. I've cleared out every .emscripten_cache/sysroot I can find on my system, and it still runs into the error mentioned above.

[sbc100]

I guess you need to take a look at that failing command and dive deeper. On my machine (and in all our test environments) this doesn't show up so it must be something specific to your environment.

Lets try to dive a bit deeper. The first thing you can see is that the problem is completely unrelated to your code, its simply trying to build one of the standard system libraries and failing. Can you confirm this by running ./embuilder build libcompiler_rt and confirming that it fails in the same way? (You can pass --verbose to that command to see each of the subcommands if that helps. You can also pass --force for force it to build even if it already exists).

Next you want to look at why atomic_flag might be undefined that point in the file. That file is only very short and expects atomic_flag to have been defined by #include <stdatomic.h>. I can confirm that this is that case on my machine.

Can you trying running the failing emcc command with -E to get the preprocessed output? Can you attach the full failing command along with that output?

[cajus]

The failing command is:

"/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/emscripten/emcc" -O2 -Werror -fno-unroll-loops -fno-builtin -g -sSTRICT -ffile-prefix-map=/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/emscripten=/emsdk/emscripten -fdebug-compilation-dir=/emsdk/emscripten -c /home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear.c -o /home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/build/libcompiler_rt-tmp/atomic_flag_clear.o

If I add a -E as a first parameter (in front of -O2), there's no visible output at all. The .o contains

# 1 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear.c"
# 1 "<built-in>" 1
# 1 "<built-in>" 3
# 332 "<built-in>" 3
# 1 "<command line>" 1
# 1 "<built-in>" 2
# 1 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear.c" 2
# 19 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear.c"
# 1 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/include/c++/v1/stdatomic.h" 1 3
# 118 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/include/c++/v1/stdatomic.h" 3
# 1 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/include/c++/v1/__config" 1 3
# 13 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/include/c++/v1/__config" 3
# 1 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/include/c++/v1/__config_site" 1 3
# 14 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/include/c++/v1/__config" 2 3
# 23 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/include/c++/v1/__config" 3
# 119 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/include/c++/v1/stdatomic.h" 2 3
# 122 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/.emscripten_cache/sysroot/include/c++/v1/stdatomic.h" 3
# 20 "/home/prcs1076/.conan/data/emsdk_installer/3.1.27/bje/stable/package/7f1fc53e4e5d5c02850574cdf1f32f27d3a68b54/upstream/emscripten/system/lib/compiler-rt/lib/builtins/atomic_flag_clear.c" 2

void atomic_flag_clear(volatile atomic_flag *object) {
  __c11_atomic_store(&(object)->_Value, 0, 5);
}

in this case.

[cajus]

When I do it manually (clone, emsdk install/activate, embuild --force) it works. So I'll try to find out more about the differences on monday. And what exactly changed from .24 to .25 to cause this. Thanks for the input so far @sbc100

[sbc100]

As I say I suspect the libc++ upgrade: #18113. That change contains a lot of upstream llvm changes. You can try to understand them, but it might not be worth you time. The imporant thing I believe is trying to figure out the difference between your two environments where one is working and one isn't. We haven't had any other reports like this so all signs point to something with you setup.

[cajus]

Ok. It looks like this is caused by an additional include I needed to add in the past, in order to make compilation from conan work. Fact is that it stopped working for whatever reason and I'm going to find a workaround here.

Closing it, as it's out of scope - and thanks for the input.