Feature: build_target for objects build_target(target_type:'objects')

[piotrrak]

At this moment there is no straightforward way to create library/executable with <lang>_args defined per compilation unit.

The idea to do that is to provide functionality that is close to cmake per source properties, but bit more powerful and convenient.

Example usage could be:

avx_objects = build_target(
  target_type:'objects',
  sources: ['my_super_fast_kernel.cc'],
  cpp_args: my_fast_libflags + avx_flags)`,
  name_prefix='avx_'
)

my_fast_lib = library('fast', ['fast.cc'], 
  cpp_args: my_fastlib_flags,
  objects: avx_objects.extract_all_objects()
)

I am aware that can be achieved by using build_target(target_type:'static_library') but considering it simply might be single object creating ar archive to achieve that is bit suboptimal.

That also could be helpful to improve/fix cmake.subproject() support which currently gets it wrong.
During compilation of OpenCV for entry from cmake_trace.txt:

{"args":["SOURCE","/home/prak/DALI_deps/builddir/third_party/opencv/__CMake_build/modules/imgproc/sumpixels.avx2.cpp","PROPERTY","COMPILE_DEFINITIONS","CV_CPU_DISPATCH_MODE=AVX2"],"cmd":"set_property","file":"/home/prak/DALI_deps/builddir/meson-private/data/preload.cmake","frame":6,"line":69,"time":1637193286.2726841}

Library with union of all source files COMPILE_DEFINITIONS / flags is created in meson.build
This results in:

build third_party/opencv/libopencv_imgproc.a.p/_home_prak_DALI_deps_builddir_third_party_opencv___CMake_build_modules_imgproc_sumpixels.avx2.cpp.o: cpp_COMPILER /home/prak/DALI_deps/builddir/third_party/opencv/__CMake_build/modules/imgproc/sumpixels.avx2.cpp || third_party/opencv/opencl_kernels_imgproc.hpp
 DEPFILE = third_party/opencv/libopencv_imgproc.a.p/_home_prak_DALI_deps_builddir_third_party_opencv___CMake_build_modules_imgproc_sumpixels.avx2.cpp.o.d
 DEPFILE_UNQUOTED = third_party/opencv/libopencv_imgproc.a.p/_home_prak_DALI_deps_builddir_third_party_opencv___CMake_build_modules_imgproc_sumpixels.avx2.cpp.o.d
 ARGS = -Ithird_party/opencv/libopencv_imgproc.a.p -Ithird_party/opencv -I../third_party/opencv -I../third_party/opencv/modules/imgproc/include -Ithird_party/opencv/__CMake_build/modules/imgproc -I../third_party/opencv/__CMake_build/modules/imgproc -I../third_party/opencv/modules/core/include -Ithird_party/opencv/__CMake_build -I../third_party/opencv/__CMake_build -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wnon-virtual-dtor -g -fPIC -isystem../third_party/opencv/__CMake_build -isystemthird_party/opencv/__CMake_build -fsigned-char -W -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden \
 -O3 -DNDEBUG -D_USE_MATH_DEFINES -D__OPENCV_BUILD=1 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS \
-mssse3 -msse4.1 -DCV_CPU_COMPILE_SSE4_1=1 -DCV_CPU_COMPILE_SSSE3=1 \
 -DCV_CPU_DISPATCH_MODE=SSE4_1 -mpopcnt -msse4.2 -mavx \
 -DCV_CPU_COMPILE_AVX=1 -DCV_CPU_COMPILE_POPCNT=1 \
 -DCV_CPU_COMPILE_SSE4_2=1 \ 
-DCV_CPU_DISPATCH_MODE=AVX -mf16c -mfma -mavx2 \
 -DCV_CPU_COMPILE_AVX2=1 -DCV_CPU_COMPILE_FMA3=1 -DCV_CPU_COMPILE_FP16=1 \
 -DCV_CPU_DISPATCH_MODE=AVX2 \
 -mavx512f -mavx512f -mavx512cd -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq \ 
-DCV_CPU_COMPILE_AVX512_COMMON=1 -DCV_CPU_COMPILE_AVX512_SKX=1 \
 -DCV_CPU_COMPILE_AVX_512F=1 -DCV_CPU_DISPATCH_MODE=AVX512_SKX

Which is not cool.

vs correct in case of cmake -GNinja

build modules/imgproc/CMakeFiles/opencv_imgproc.dir/sumpixels.avx2.cpp.o: CXX_COMPILER__opencv_imgproc_RELEASE modules/imgproc/sumpixels.avx2.cpp || cmake_object_order_depends_target_opencv_imgproc
  DEFINES = -D_USE_MATH_DEFINES -D__OPENCV_BUILD=1 -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DCV_CPU_COMPILE_AVX2=1 -DCV_CPU_COMPILE_AVX=1 -DCV_CPU_COMPILE_FMA3=1 -DCV_CPU_COMPILE_FP16=1 -DCV_CPU_COMPILE_POPCNT=1 -DCV_CPU_COMPILE_SSE4_1=1 -DCV_CPU_COMPILE_SSE4_2=1 -DCV_CPU_COMPILE_SSSE3=1 -DCV_CPU_DISPATCH_MODE=AVX2
  DEP_FILE = modules/imgproc/CMakeFiles/opencv_imgproc.dir/sumpixels.avx2.cpp.o.d
  FLAGS = -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Wsuggest-override -Wno-delete-non-virtual-dtor -Wno-comment -Wimplicit-fallthrough=3 -Wno-strict-overflow -fdiagnostics-show-option -Wno-long-long -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections  -msse -msse2 -msse3 -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG -fPIC -std=c++11  -mssse3 -msse4.1 -mpopcnt -msse4.2 -mf16c -mfma -mavx -mavx2
  INCLUDES = -I. -I../modules/imgproc/include -Imodules/imgproc -I../modules/core/include
  OBJECT_DIR = modules/imgproc/CMakeFiles/opencv_imgproc.dir
  OBJECT_FILE_DIR = modules/imgproc/CMakeFiles/opencv_imgproc.dir
  TARGET_COMPILE_PDB = lib/libopencv_imgproc.pdb
  TARGET_PDB = lib/libopencv_imgproc.pdb

Wanted to ask first if such functionality is welcome/good idea, before I give a shot build_target(target_type:'static_library') workaround.
Please note that I might not be not best person to implement it, but can try if no one else is willing to.

[dcbaker]

One thing that I think we likely want to do is provide an ExtractObjects directly, so that the end result would be:

objs = compiled_objects(...)
library(
  ...,
  objects : objs,  # no need for extract_all_objects(), as this is already the type returned by library.extract_all_objects()
)

Also, we generally have a free form name for targets, so you don't have to use build_target, compiled_objects seems like the obvious one in this case.

[piotrrak]

Sure, compiled_objects(...) sounds fine to me. Right now I have simply objects()and BuildObjects BuildObjectsHolder in my attempt it.

Had similar feelings it should be possible to pass it to objects directly.

Thanks for feedback!

[piotrrak]

For starters #9600

[bonzini]

but considering it simply might be single object creating ar archive to achieve that is bit suboptimal.

I was thinking (but never finished implementing it) that we could simply make link_whole of a static library include the objects directly instead of going through the .a file. Then compiled_objects would simply be a static library that can only be used with link_whole or extract_objects.

[piotrrak]

@bonzini
Do you suggest to forgo adding compiled_objects? From the user point of view, it'd be really nice to have that as first class citizen in meson. This is pretty intuitive API in my opinion.

Also it wouldn't solve one (though pretty niche) use-case of installing objects to ${libdir} or some other specific directory (think compilers, sanitizer deps, libc). Or, would it?

I find the idea of such optimization for static_library, very appealing.
Will think about that route, to at least share more code with static_library, thanks. What I came up so far went pretty problem free, but hard part I am still tinkering with is that install thing, but it slowly comes together.
What I am trying to solve would also be needed to make what you've described possible, I guess.

[bonzini]

Do you suggest to forgo adding compiled_objects?

Maybe not, but it might be easier to first do all backend changes, and then introduce compiled_objects just for the install usecase.

[dcbaker]

We tried to add thin archives, but ran into some issues with them. I still think it's possible to make them work, and that would basically be what you're suggesting. I still think that the object only option is still worth while, even if just for libc.

[piotrrak]

@bonzini I think you might be right, that it would be best attack plan.

What I am trying to achieve now is to add better support for getting list of objects target.outputs of BuildTarget, which I see as prerequisite of what you're suggesting. For compiled languages its not that hard, but as I've mentioned vala and cython get in my way. That logic is burried in Ninja backend right now.
I've started factoring it out to see what it will take, making slow but steady progress there.

Did not look at link_whole logic yet, at all.

@dcbaker Do you have any pointers to disscussions/PRs that could shed some light on what were those issues?

[eli-schwartz]

#9453

"I have a custom_target that tries to use a .a file but doesn't use the system linker and doesn't understand what a thin archive is".

[bonzini]

Thin archives are a hack that Meson doesn't need. link_whole also is hackish; static libraries force Meson to add --start-group/--end-group arguments and who knows which arguments end up in the group. Instead, all .o files are implicitly "grouped" ahead of static libraries.