cmake `check_c_source_compiles` fails while finding deps when `c` binary is a list of strings #13049

dg0yt · 2024-04-06T15:46:33Z

Describe the bug
Vcpkg port mesa fails to build in configurations which transitively depend on libterminfo (Terminfo::terminfo) via LLVMSupport.
I suspect the issue is in meson.

Vcpkg passes c via a machine file as:

c = ['/usr/bin/cc', '-fPIC', '-O3', '-DNDEBUG']

LLVMConfig.cmake calls

find_package(Terminfo)

as it does during llvm build. And the find module is properly found as part of the llvm's CMake config files.
This module has a

check_c_source_compiles("..." Terminfo_LINKABLE)

as a prerequisite for Terminfo_FOUND and target setup.

However, this check_c_source_compiles fails when meson tries to find llvm via CMake. In buildtrees/mesa/x64-linux-rel/meson-private/cmake_LLVM/CMakeFiles/CMakeConfigureLog.yaml, I found:

---
events:
  -
    kind: "message-v1"
    backtrace:
      - "/vcpkg/vcpkg/downloads/tools/cmake-3.27.1-linux/cmake-3.27.1-linux-x86_64/share/cmake-3.27/Modules/CMakeDetermineSystem.cmake:206 (message)"
      - "CMakeLists.txt:3 (project)"
    message: |
      The target system is: Linux -  - 
      The host system is: Linux - 5.4.0-174-generic - x86_64
  -
    kind: "try_compile-v1"
    backtrace:
      - "/vcpkg/vcpkg/downloads/tools/cmake-3.27.1-linux/cmake-3.27.1-linux-x86_64/share/cmake-3.27/Modules/Internal/CheckSourceCompiles.cmake:101 (try_compile)"
      - "/vcpkg/vcpkg/downloads/tools/cmake-3.27.1-linux/cmake-3.27.1-linux-x86_64/share/cmake-3.27/Modules/CheckCSourceCompiles.cmake:52 (cmake_check_source_compiles)"
      - "/vcpkg/vcpkg/installed/x64-linux/share/llvm/FindTerminfo.cmake:21 (check_c_source_compiles)"
      - "/vcpkg/vcpkg/scripts/buildsystems/vcpkg.cmake:859 (_find_package)"
      - "/vcpkg/vcpkg/installed/x64-linux/share/llvm/LLVMConfig.cmake:165 (find_package)"
      - "/vcpkg/vcpkg/scripts/buildsystems/vcpkg.cmake:859 (_find_package)"
      - "CMakeLists.txt:17 (find_package)"
    checks:
      - "Performing Test Terminfo_LINKABLE"
    directories:
      source: "/vcpkg/vcpkg/buildtrees/mesa/x64-linux-rel/meson-private/cmake_LLVM/CMakeFiles/CMakeScratch/TryCompile-PU34xV"
      binary: "/vcpkg/vcpkg/buildtrees/mesa/x64-linux-rel/meson-private/cmake_LLVM/CMakeFiles/CMakeScratch/TryCompile-PU34xV"
    cmakeVariables:
      CMAKE_C_FLAGS: "-fPIC"
      CMAKE_C_FLAGS_DEBUG: "-g"
      CMAKE_EXE_LINKER_FLAGS: ""
      CMAKE_MODULE_PATH: "/vcpkg/vcpkg/installed/x64-linux/share/llvm"
      VCPKG_APPLOCAL_DEPS: "ON"
      VCPKG_CHAINLOAD_TOOLCHAIN_FILE: "/vcpkg/vcpkg/scripts/toolchains/linux.cmake"
      VCPKG_TARGET_TRIPLET: "x64-linux"
      Z_VCPKG_ROOT_DIR: "/vcpkg/vcpkg"
    buildResult:
      variable: "Terminfo_LINKABLE"
      cached: true
      stdout: |
        Change Dir: '/vcpkg/vcpkg/buildtrees/mesa/x64-linux-rel/meson-private/cmake_LLVM/CMakeFiles/CMakeScratch/TryCompile-PU34xV'
        
        Run Build Command(s): /vcpkg/vcpkg/downloads/tools/cmake-3.27.1-linux/cmake-3.27.1-linux-x86_64/bin/cmake -E env VERBOSE=1 /usr/bin/make -f Makefile cmTC_92605/fast
        /usr/bin/make  -f CMakeFiles/cmTC_92605.dir/build.make CMakeFiles/cmTC_92605.dir/build
        make[1]: Entering directory '/vcpkg/vcpkg/buildtrees/mesa/x64-linux-rel/meson-private/cmake_LLVM/CMakeFiles/CMakeScratch/TryCompile-PU34xV'
        Building C object CMakeFiles/cmTC_92605.dir/src.c.o
        "/usr/bin/cc;-fPIC;-O3;-DNDEBUG" -fPIC -O3 -DNDEBUG -DTerminfo_LINKABLE  -fPIC  -o CMakeFiles/cmTC_92605.dir/src.c.o -c /vcpkg/vcpkg/buildtrees/mesa/x64-linux-rel/meson-private/cmake_LLVM/CMakeFiles/CMakeScratch/TryCompile-PU34xV/src.c
        /bin/sh: 1: /usr/bin/cc;-fPIC;-O3;-DNDEBUG: not found
        make[1]: *** [CMakeFiles/cmTC_92605.dir/build.make:78: CMakeFiles/cmTC_92605.dir/src.c.o] Error 127
        make[1]: Leaving directory '/vcpkg/vcpkg/buildtrees/mesa/x64-linux-rel/meson-private/cmake_LLVM/CMakeFiles/CMakeScratch/TryCompile-PU34xV'
        make: *** [Makefile:127: cmTC_92605/fast] Error 2
        
      exitCode: 2
...

Note the "/usr/bin/cc;-fPIC;-O3;-DNDEBUG" which points at the c list of strings being incorrectly fed as a single item into CMAKE_C_COMPILER

To Reproduce
All I can offer now is

./vcpkg install mesa llvm[core,enable-terminfo] --cmake-args=-DVCPKG_BUILD_TYPE=release

on x64-linux which takes while due to llvm, even in release-only build.

In vcpkg CI, mesa on linux was disabled for a number of problems, but I'm trying to fix that in microsoft/vcpkg#36081.

It is probaly possible to create a more trivial reproducer for the given input (multi-string c, cmake dependency, check_c_source_compiles).

Expected behavior
It should run /usr/bin/cc -fPIC -O3 -DNDEBUG ..., letting the mesa link steps succeed.

system parameters

native with machine file from vcpkg, x64-linux
Ubuntu 20.04
Python version '3.8.10 at /usr/bin/python3'
meson version: 1.3.0 (vcpkg port)
ninja version: 1.10.0

The text was updated successfully, but these errors were encountered:

Neumann-A · 2024-04-06T16:40:44Z

Looks like a meson upstream bug to me. Passing it this way is the blessed way e.g. see

meson/cross/armclang-linux.txt

Line 20 in e455cb0

    
           c = ['/opt/arm/developmentstudio-2019.0/sw/ARMCompiler6.12/bin/armclang', '--target=aarch64-arm-none-eabi']

(with upstream I meant meson problem. Thought I was in the vcpkg repo; how the hell did I end up here?)

eli-schwartz · 2024-04-07T17:26:38Z

Experimented with "test cases/frameworks/15 llvm/".

diff --git a/test cases/frameworks/15 llvm/meson.build b/test cases/frameworks/15 llvm/meson.build
index bb945cc44..c1c15721e 100644
--- a/test cases/frameworks/15 llvm/meson.build	
+++ b/test cases/frameworks/15 llvm/meson.build	
@@ -15,7 +15,7 @@ endif
 
 modules_to_find = [
   'bitwriter', 'asmprinter', 'executionengine', 'mcjit', 'target',
-  'nativecodegen', 'amdgpu', 'engine'
+  'nativecodegen', 'amdgpu', 'engine', 'LLVMSupport',
 ]
 
 if method == 'combination'

meson logs:

C compiler for the host machine: /usr/bin/cc -fPIC -O3 -DNDEBUG (gcc 13.2.1 "cc (Gentoo 13.2.1_p20240210 p14) 13.2.1 20240210")
C linker for the host machine: /usr/bin/cc -fPIC -O3 -DNDEBUG ld.bfd 2.41

[...]

llvm-config found: YES (/usr/lib/llvm/17/bin/llvm-config) 17.0.6
Run-time dependency LLVM (modules: LLVMSupport(missing), amdgpu, asmprinter, bitwriter, engine, executionengine, mcjit, nativecodegen, target) found: NO (tried config-tool)
Run-time dependency LLVM found: NO 

meson.build:51:2: ERROR: Assert failed: config-tool and cmake both need to be found

So this is reproducible using the meson unittests as a guideline.

eli-schwartz · 2024-04-07T17:28:15Z

In builddir/meson-private/cmake_LLVM/CMakeMesonToolchainFile.cmake

# Variables from meson
set(CMAKE_SIZEOF_VOID_P "8")
set(CMAKE_C_COMPILER "/usr/bin/cc" "-fPIC" "-O3" "-DNDEBUG")
set(CMAKE_CXX_COMPILER_LAUNCHER "/usr/bin/ccache")
set(CMAKE_CXX_COMPILER "/usr/bin/c++")

I'm guessing maybe CMAKE_C_COMPILER isn't meant to work this way...

eli-schwartz · 2024-04-07T17:36:21Z

meson/mesonbuild/cmake/toolchain.py

Lines 71 to 79 in 24a96a3

    
           @staticmethod 
        
           def _print_vars(vars: T.Dict[str, T.List[str]]) -> str: 
        
               res = '' 
        
               for key, value in vars.items(): 
        
                   res += 'set(' + key 
        
                   for i in value: 
        
                       res += f' "{i}"' 
        
                   res += ')\n' 
        
               return res

meson/mesonbuild/cmake/toolchain.py

Lines 111 to 114 in 24a96a3

    
           # Set variables from the current machine config 
        
           res += '# Variables from meson\n' 
        
           res += self._print_vars(self.variables) 
        
           res += '\n'

Defaults for compilers are defined here based on meson's own compiler environment info:

meson/mesonbuild/cmake/toolchain.py

Lines 175 to 192 in 24a96a3

    
           # Set the compiler variables 
        
           for lang, comp_obj in self.compilers.items(): 
        
               prefix = 'CMAKE_{}_'.format(language_map.get(lang, lang.upper())) 
        
               exe_list = comp_obj.get_exelist() 
        
               if not exe_list: 
        
                   continue 
        
               if len(exe_list) >= 2 and not self.is_cmdline_option(comp_obj, exe_list[1]): 
        
                   defaults[prefix + 'COMPILER_LAUNCHER'] = [make_abs(exe_list[0])] 
        
                   exe_list = exe_list[1:] 
        
               exe_list[0] = make_abs(exe_list[0]) 
        
               defaults[prefix + 'COMPILER'] = exe_list 
        
               if comp_obj.get_id() == 'clang-cl': 
        
                   defaults['CMAKE_LINKER'] = comp_obj.get_linker_exelist() 
        
           return defaults

So with this machine file:

[binaries]
c = ['/usr/bin/cc', '-fPIC', '-O3', '-DNDEBUG']

variables has (python):

self.variables = {}
self.variables['CMAKE_C_COMPILER'] = ['/usr/bin/cc', '-fPIC', '-O3', '-DNDEBUG']

dg0yt · 2024-04-08T06:14:40Z

https://cmake.org/cmake/help/latest/variable/CMAKE_LANG_COMPILER.html: "The full path to the compiler for LANG." This doesn't allow for extra flags.

OTOH https://mesonbuild.com/Machine-files.html#binariesallows lists of strings for binaries. It even uses a list for the "Cross example". Now the question is how to translate this for CMake, with out intruding too much into the CMake toolchain's setup of flags.

eli-schwartz · 2024-04-08T12:12:58Z

Meson expects compile definitions and optimization flags and similar to be in CFLAGS, not in the CC binary definition itself. However it's a common use case to do something like c = ['gcc', '-m32'] or similarly the --target flag.

The distinction is whether the flags in question are intended to select the right compiler output profile. Questions such as "does this influence what cross compile platform the compiler masquerades at" or "will it affect --print-search-dirs reporting" are key.

Since this issue would affect that usage as well, I figure the answer isn't just "move it to meson's c_args list / $CFLAGS".

Now the question is how to translate this for CMake, with out intruding too much into the CMake toolchain's setup of flags.

I seem to recall that cmake has a separate setting for ccache, as opposed to meson which defines the use of ccache as "the compiler is ccache cc". Could this be related?

dg0yt changed the title ~~cmake check_c_source_compiles fails while finding deps when c binaries is a list of strings~~ cmake check_c_source_compiles fails while finding deps when c binary is a list of strings Apr 6, 2024

dg0yt mentioned this issue Apr 6, 2024

[mesa] Features, dependencies and CI microsoft/vcpkg#36081

Draft

eli-schwartz added the dependency:cmake Issues related to `dependency` with the `cmake` method label Apr 7, 2024

eli-schwartz assigned mensinda Apr 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmake `check_c_source_compiles` fails while finding deps when `c` binary is a list of strings #13049

cmake `check_c_source_compiles` fails while finding deps when `c` binary is a list of strings #13049

dg0yt commented Apr 6, 2024 •

edited

Loading

Neumann-A commented Apr 6, 2024 •

edited

Loading

eli-schwartz commented Apr 7, 2024

eli-schwartz commented Apr 7, 2024

eli-schwartz commented Apr 7, 2024

dg0yt commented Apr 8, 2024

eli-schwartz commented Apr 8, 2024

cmake check_c_source_compiles fails while finding deps when c binary is a list of strings #13049

cmake check_c_source_compiles fails while finding deps when c binary is a list of strings #13049

Comments

dg0yt commented Apr 6, 2024 • edited Loading

Neumann-A commented Apr 6, 2024 • edited Loading

eli-schwartz commented Apr 7, 2024

eli-schwartz commented Apr 7, 2024

eli-schwartz commented Apr 7, 2024

dg0yt commented Apr 8, 2024

eli-schwartz commented Apr 8, 2024

cmake `check_c_source_compiles` fails while finding deps when `c` binary is a list of strings #13049

cmake `check_c_source_compiles` fails while finding deps when `c` binary is a list of strings #13049

dg0yt commented Apr 6, 2024 •

edited

Loading

Neumann-A commented Apr 6, 2024 •

edited

Loading