[question] Using custom cross-compilation toolchains with requirements and CMakeDeps

[rollebolle]

What is your question?

Updating to conan 2.3.2

I'm building packages via a Yocto based x-compilation toolchain, and the SDK generates this toolchain for us. It is vital to use it as it sets CMAKE_FIND_ROOT_PATH among other things, to find packages that are in the SDK and does not come from Conan. Therefore I override the generated toolchain with the tools.cmake.cmaketoolchain:toolchain_file option.

However this poses a problem when we also have requirements statements for our Conan requirements. Conan generates the correct <dep>-config.cmake files for each requirement, but they end up under .../build/Release/generators. As we are using a custom toolchain, we have no way of knowing this path in beforehand, and therefore we dont know how to set the CMAKE_MODULE_PATH.

If I instead do NOT use the custom toolchain file, but rely on the generated one from Conan with CMakeToolchain, I find the requirement packages as the module path is calculated and added to the toolchain file. But then I get problems for packages in our cross-compilation SDK that we cannot find.

The solution we had in Conan 1.63 was to use the old syntax without toolchain etc, and using cmake_find_package. This worked, even with a custom toolchain file, because the Find<dep>.cmake files ended up in the build folder. As cmake_find_package is deprecated, I want to move away from this.

What is the best practice here?

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @rollebolle

Thanks for your question.

The self.generators_folder is known in the recipe, and it will point to the folder that you want.
What is the mechanism that you need to have this folder communicated to your CMake? You can add any arbitrary variables to the toolchain and presets like:

def generate(self):
     tc = CMakeToolchain(self)
     gen_folder = self.generators_folder.replace("\\", "/")  # Windows-CMake not happy
     tc.variables["MYPATH_GEN"] = gen_folder   # IT will be a variable in conan_toolchain.cmake
     tc.cache_variables["MYPATH_GEN"] = gen_folder  # It will be a cache variables in CMakePresets (and passed in CLI arg)
     tc.generate()

Does this make sense for your use case?

[rollebolle]

Thanks. This does work and I I'll use it. However it means my recipe now has adapations that is only needed in the special case of supplying a toolchain from outside. Ideally I would like the recipe to be agnostic to this detail. As I wrote above, this was not needed with the old generators as the FindXXX files happened to all end up in the build folder. Let me know if I misunderstood.

[memsharded]

Quick question:

it sets CMAKE_FIND_ROOT_PATH among other things, to find packages that are in the SDK and does not come from Conan. Therefore I override the generated toolchain with the tools.cmake.cmaketoolchain:toolchain_file option.

Why don't you use the generated conan_toolchain.cmake and also your own one via the tools.cmake.cmaketoolchain:user_toolchain? That appends your own toolchain, but also uses the Conan one, so that should work?

[rollebolle]

I tried that but get some nasty conflicts - I think in many advanced cases you want to have complete control of the toolchain file, as in our case.

[memsharded]

I have been thinking a bit about this problem, and honestly I don't know yet what would be the best approach.

I think it might be possible to try to find the generated files from your toolchain. This might be done for example by looking for the location of the conandeps_legacy.cmake that will be in the vast majority of cases in a subfolder of the current "build" folder, containing all the other generated files. It might be a bit fragile, because the conandeps_legacy.cmake might be removed in the future, as its name indicates, but it might work. The CMakePresets.json is also generated inside the generators_folder with the other files, and if there is a CMakeLists.txt in the root, the CMakeUserPreset.json file that is generated will have an include to the CMakePresets.json file. Using this, it seems quite feasible to obtain the generators_folder from your own toolchain file.

What would be the best mechanism to communicate the generators_folder, which might be different in different packages (if all use the same cmake_layout(), then it would be direct to deduce the location), to a user toolchain_file otherwise?

Please let me know if this helps.

[rollebolle]

Thanks for taking the time to think about this. Would it make sense for Conan to always append a valid CMAKE_MODULE_PATH to CMakeUserPresets.json, so that even if we use a custom toolchain we can find the dependencies?

I don't really know the details around this, but in Conan 2, a dependency on using the generated toolchains to be able to find any requirement packages was introduced. This in combination with the common cross-compile scenario where we want to use a custom toolchain file certainly makes it a bit more complicated to "get it right".

[memsharded]

Thanks for taking the time to think about this. Would it make sense for Conan to always append a valid CMAKE_MODULE_PATH to CMakeUserPresets.json, so that even if we use a custom toolchain we can find the dependencies?

Yes, this might be something worth exploring.

I don't really know the details around this, but in Conan 2, a dependency on using the generated toolchains to be able to find any requirement packages was introduced. This in combination with the common cross-compile scenario where we want to use a custom toolchain file certainly makes it a bit more complicated to "get it right".

yes, it is true that the toolchain_file conf, fully replacing the toolchain, hasn't been that used so far, it seems the injection of toolchains via include() with user_toolchain conf is way more popular, because there is some important configuration that Conan knows already how to map, that would require an extra effort from users. That is, the mapping from Conan settings to CMake variables, for example to set the VS runtime, to select BUILD_SHARED_LIBS or not, enable/disable fPIC, etc, is a variability that otherwise the user toolchain_file would have to implement, most likely requiring different toolchain_file files, and that is more difficult to maintain that using the include() mechanism of user_toolchain and letting Conan complete the information to adjust to Conan settings.

But in any case, I understand the use case, let me discuss with the team, it sounds that to minimize risks we could add those variables when the toolchain_file is defined. I am assigning this to have a look for Conan 2.5, thanks for the feedback!

[memsharded]

I am exploring the idea in #16455, but it is a draft, I am not very convinced, I might explore other approaches too.

[memsharded]

I have been syncing with the team about #16455 we are still not fully convinced about the approach. We are going to explore the possibility of fully controlling the CMakeToolchain blocks from the outside or inject the toolchain at the end so it has higher priority.

[rollebolle]

Ok interesting!

[memsharded]

This would be the follow up PR: #16563

Full control over which blocks are enabled or not, with just a conf. That would allow to keep the find_paths and include blocks only, which is the one you need, then using user_toolchain you would get the both things: full control over all toolchain, as Conan will not inject anything more, except the necessary paths to locate dependencies provided by find_paths module.

If you want to give it a try from sources, that would be very useful feedback.

[memsharded]

#16563 has been merged for next 2.5, and it will be the recommended approach for this issue, feedback welcome!

[rollebolle]

I tried the feature and it works. Only small unexpected behavior was that the orders of the enabled blocks is not the same as in the "untouched" toolchain file. For example, the user toolchain inclusion came after the find_path block for me

[memsharded]

Thanks for the feedback @rollebolle

The feature is intended to respect the order defined by users in tools.cmake.cmaketoolchain:enabled_blocks or in the conanfile, precisely to allow full control over the generated file. If you define it in the right order, it should respect it, have you tried changing the order?

[rollebolle]

Sorry - you are right. I accidentally changed the order

[rollebolle]

Sorry for opening a discussion here again - and please say if I should move it somewhere else as this issue is resolved. So in short I was a bit fast to celebrate. There are certain cases where the approach even with enable_block is not enough.

Consider the following example scenario. We have a cross-compilation SDK. In this SDK we have fmt 9.1 installed (and we don't have full control to just remove it, because of corp reasons). Then we have our own package mypack that we want to build with Conan 2. mypack is dependent on fmt 10.2.1 and cannot link to 9.1. So, we add fmt 10.2.1 as a requirement. The fmt-config.cmake file with friends ends up in the generators folder as it should.

We add our custom user_toolchain, and we enable the block find_paths. Our own custom toolchain contains a bunch of lines, but the problematic and interesting ones are:

set( CMAKE_SYSROOT $ENV{OECORE_TARGET_SYSROOT} CACHE STRING "" )
set( CMAKE_FIND_ROOT_PATH $ENV{OECORE_TARGET_SYSROOT} CACHE STRING "")
set( CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER )
set( CMAKE_FIND_ROOT_PATH_MODE_LIBRARY BOTH )
set( CMAKE_FIND_ROOT_PATH_MODE_INCLUDE BOTH )
set( CMAKE_FIND_ROOT_PATH_MODE_PACKAGE BOTH )

This means, when looking for packages, the sysroot (our SDK root) will always be prefered. EVEN though the find_path block actually defines:

list(PREPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_LIST_DIR})
list(PREPEND CMAKE_PREFIX_PATH ${CMAKE_CURRENT_LIST_DIR} )

This does not matter - Conan 2 will always prefer the fmt-config.cmake from the SDK, if it finds it. This making the find_paths block unusable. If I remove fmt from the SDK, it immediately works (but this is not an option, as stated).

The reason it worked in Conan 1.X is that the folder where CMAKE_MODULE_PATH points to always seemed to contain a FindFmt.cmake file - in this case, the Find file actually takes precedence over the fmt-config.cmake files from the SDK.

I am not sure how to work around this. FindXXX is kinda deprecated I guess, but the config interface seems to not have considered this scenario. Any feedback is welcome. I have looked through all the CMake variables I can find that controls the find behaviour but cannot find any solution.

UPDATE: Editing the generated toolchain from conan so the find_path block also contains the following line seems to fix the issue:

list(PREPEND CMAKE_FIND_ROOT_PATH ${CMAKE_CURRENT_LIST_DIR})

I have no idea what other side effects this could give - but as you do the same thing with CMAKE_MODULE_PATH and CMAKE_PREFIX_PATH, maybe this should also get added?

[jcar87]

Thanks @rollebolle. I have troubleshooting this recently (so that we can fix this), and I believe what you are experiencing is the same as "example 3" here: https://github.com/jcar87/conan-crossbuild-with-sysroot?tab=readme-ov-file#example-3-cmake-finds-a-dependency-from-the-sysroot-when-it-should-pick-it-up-from-conan

A workaround in this case - is to define fmt_DIR to the directory that contains fmt-config.cmake - then it will look in the right place.

CMAKE_FIND_ROOT_PATH should also work - currently trying to work out if that should be the default approach - the only downside "so far" is that it will cause all searches to be rooted there, so all find commands may take a presumedly negligible amount of time longer

[rollebolle]

Thanks for the reply. In specific cases - yes this could work, when we own the recpie. Consider though when we want to take a package from conancenter "as is" - we have no way to use this workaround. I got this exact problem for the nlohmann-json-schema-validator package from conancenter, that picked up an old version of the nlohman-json library.

[jcar87]

Thanks for the reply. In specific cases - yes this could work, when we own the recpie. Consider though when we want to take a package from conancenter "as is" - we have no way to use this workaround. I got this exact problem for the nlohmann-json-schema-validator package from conancenter, that picked up an old version of the nlohman-json library.

could you print the paths in which CMake is finding the wrong nlohmann-json and fmt? there could variables to define that would work without modifying any recipes

[rollebolle]

Sure. We use Yocto (openembedded) and this is the paths in the sysroot it finds:

/usr/lib/cmake/fmt/fmt-config.cmake

and similarily for other libs

[jcar87]

Thanks @rollebolle -

Perhaps this can be useful:
https://cmake.org/cmake/help/latest/variable/CMAKE_IGNORE_PREFIX_PATH.html#variable:CMAKE_IGNORE_PREFIX_PATH

you could add /usr/lib/cmake to CMAKE_IGNORE_PREFIX_PATH and hopefully that takes care of the problem in a more generic way.

This could be defined in a user toolchain (chain-loaded by the one that Conan generates), or defined as a variable in the tools.cmake.cmaketoolchain:extra_variables conf.

[rollebolle]

Maybe, but it kinda breaks my intention. The logic I would want to achieve is the following (and should be pretty common in any cross compilation):

1. Is there any explicit package reqs via Conan. Choose them for find_package etc.
2. No explicit reqs? Check the sysroot for deps.

This is an usual usecase for us, as not all packages are managed via Conan. This was how it worked before (even though it may not have been intentional).

[rollebolle]

We have been able to work around this issue now, but I think still this is a problem that needs to be solved in some way. To summarize the problem with an example:

• We have a cross-compilation SDK that happens to contain libfmt v9 (needed for some packages in the SDK itself)
• We want to use spdlog that depends on libfmt v10 and prefer to get it through Conan.
• We cannot modify the recipe as we want to take it "as-is" from Conan Center to not create conflicts in the dep graph, so we cannot explicitly point to the Conan version of fmt with fmt_DIR
• Setting CMAKE_IGNORE_PREFIX_PATH or stuff like that can break other stuff for us
• Now we want spdlog to use the Conan 2 version of libfmt (v10) and not the default in the SDK /usr/lib folder. Basically, any Conan package with explicit dependencies should FIRST select the libraries from Conan, and the system libraries secondly. Note that the problem (that it picks up system path libs first) is a changed behvior from Conan 1, probably due to the FindXXX vs XXX-config way of searching for packages.