Build fails with -DCLR_CMAKE_USE_SYSTEM_BROTLI=true

[omajid]

Description

Building the runtime with system brotli fails in the main branch

Reproduction Steps

$ git rev-parse HEAD
efb8a8fe6263cb66c154576c859523e885dc5a24
$ git branch -lv
* main efb8a8fe626 [browser] Speed up `WasmBuildTests` by using the compression switch (#110645)
$ ./build.sh  --cmakeargs  -DCLR_CMAKE_USE_SYSTEM_BROTLI=true /p:FullAssemblySigningSupported=false
...
  [100%] Linking CXX executable singlefilehost                                                                                                                                                                                                                                                                               
  ld.lld: error: undefined symbol: BrotliDecoderCreateInstance                                                                                                                                                                                                                                                               
  >>> referenced by entrypoints.c                                                                                                                                                                                                                                                                                            
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a                                                                                                                                                                           
                                                                                                                                                                                                                                                                                                                             
  ld.lld: error: undefined symbol: BrotliDecoderDecompress                                                                                                    
  >>> referenced by entrypoints.c                                                                                                                             
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a       
                                                                                                                                                                                                                                                                                                                             
  ld.lld: error: undefined symbol: BrotliDecoderDecompressStream                                                                                              
  >>> referenced by entrypoints.c                                                                                                                             
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a          
                                                                                                                                                                                                                                                                                                                             
  ld.lld: error: undefined symbol: BrotliDecoderDestroyInstance                                                                                                                                                                                                                                                              
  >>> referenced by entrypoints.c                                                                                                                                                                                                                                                                                            
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a                                                                                                                                                                           
                                                                                                                                                                                                                                                                                                                             
  ld.lld: error: undefined symbol: BrotliDecoderIsFinished                                                                                                                                                                                                                                                                   
  >>> referenced by entrypoints.c                                                                                                                                                                                                                                                                                            
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a                                                                                                                                                                           
                                                                                                                                                                                                                                                                                                                             
  ld.lld: error: undefined symbol: BrotliEncoderCompress                                                                                                      
  >>> referenced by entrypoints.c                                                                                                                             
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a                                                                                                                                                                           
                                                                                                                                                              
  ld.lld: error: undefined symbol: BrotliEncoderCompressStream                                                                                                
  >>> referenced by entrypoints.c                                                                                                                             
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a                                                                                                                                                                           
                                                                                                                                                                                                                                                                                                                             
  ld.lld: error: undefined symbol: BrotliEncoderCreateInstance                                                                                                
  >>> referenced by entrypoints.c                                                                                                                             
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a                                                                                                                                                                           
                                                                                                                                                              
  ld.lld: error: undefined symbol: BrotliEncoderDestroyInstance                                                                                               
  >>> referenced by entrypoints.c                                                                                                                             
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a        
                                              
  ld.lld: error: undefined symbol: BrotliEncoderHasMoreOutput 
  >>> referenced by entrypoints.c
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a
   
  ld.lld: error: undefined symbol: BrotliEncoderMaxCompressedSize
  >>> referenced by entrypoints.c
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a
   
  ld.lld: error: undefined symbol: BrotliEncoderSetParameter    
  >>> referenced by entrypoints.c
  >>>               entrypoints.c.o:(s_compressionNative) in archive ../libs-native/System.IO.Compression.Native/libSystem.IO.Compression.Native.a
  clang++-19: error: linker command failed with exit code 1 (use -v to see invocation)                                                                        
  make[2]: *** [Corehost.Static/CMakeFiles/singlefilehost.dir/build.make:1303: Corehost.Static/singlefilehost] Error 1                                        
  make[1]: *** [CMakeFiles/Makefile2:2191: Corehost.Static/CMakeFiles/singlefilehost.dir/all] Error 2      

Expected behavior

Build works

Actual behavior

Linker fails to assemble an executable

Regression?

Yes. This was working some time ago.

Known Workarounds

No response

Configuration

• This is main branch of dotnet/runtime
• Fedora 41 on x64 (aka x86_64)

Other information

No response

[omajid]

cc @tmds

[tmds]

#109105 got closed when #109707 was merged. That makes #109707 a likely suspect for this regression.

[jkotas]

cc @jkoritzinsky

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-io-compression
See info in area-owners.md if you want to be subscribed.

[omajid]

#109105 got closed when #109707 was merged. That makes #109707 a likely suspect for this regression.

A git bisect with git bisect run bash -c 'git clean -xdf && ./build.sh --cmakeargs -DCLR_CMAKE_USE_SYSTEM_BROTLI=true /p:FullAssemblySigningSupported=false' seems to confirm this:

5fe72c12cd557197824c2485748c9e9a4af36b46 is the first bad commit                                                                                                 
commit 5fe72c12cd557197824c2485748c9e9a4af36b46                                                                                                                  
Author: Jeremy Koritzinsky <jekoritz@microsoft.com>                                                                                                              
Date:   Thu Dec 5 16:31:13 2024 -0800                                                                                                                            
                                                                                
    Put brotli on the FetchContent plan (Try 2) (#109707)                                                                                                        
                                                                                
    * Reapply "Put brotli on the FetchContent plan (#107166)" (#109108)         
                                                                                
    This reverts commit b780069abf78978112cb860f9c636f767d0b4437.                                                                                                
                                                                                
    * Don't double-install the brotli libs                                      
                                                                                                                                                                 
    * Don't try looking up locations for system brotli. Rely on the distro maintainer to set the paths correctly.                                                
                                                                                
    Fixes https://github.com/dotnet/runtime/issues/109105                       
                                                                                                                                                                 
    * Install brotli where we install the compression native lib so they're always side-by-side (and we don't need to add it to the Mono build)                  
                                                                                
    * Brotli should get pulled from the same place as Compression.Native        
                                                                                
    * Fix looking for libbrotlicommon.a on disk to determine if we should use system brotli                                                                      
                                                                                

[jkoritzinsky]

I think we need to fix the CMake logic to link against the system brotli libs. Its possible that those steps got lost at some point.