question: how to same package name, different version, different settings combinations

[Johnnyxy]

hi there,
I have two versions of an third party dependency. This dependency supports serval os/archs/compilers/etc. but in different versions of the dependency itself.

I have a version of the dependency dependency/1.0.1@test/test that supports serval combinations of the settings. Like supporting Windows x86/x86_64 and Neutrino x86, but not Neutrino x86_64.

But I have another version of the dependency dependency/1.0.2@test/test that supports Windows x86/x86_64 and Neutrino x86_64, but not Neutrino x86.

Both recipes have the following settings:

settings = {
    "os"            : ["Windows", "Neutrino"],
    "arch"          : ["x86", "x86_64"],
}

Now when I specify the dependency in a consuming recipe with:

def build_requirements(self):
    self.build_requires(f"dependency/[~1.0]@test/test")

the version detection can choose dependency/1.0.1@test/test or dependency/1.0.2@test/test.
This would be fine if both dependencies are supporting all settings combinations. But they do not.

Is there a way that can I instruct Conan to take into account that a certain settings combination are valid only for version dependency/1.0.1@test/test?

Thx for any advice in advance :)

[memsharded]

Hi @Johnnyxy !

The dependencies resolution is done at 2 steps: First, the "recipes" are resolved, and a dependency graph of recipes with exact versions is built. Then, the binary IDs of the graph are computed. Note that this is necessary, as the version of upstream dependencies can change the binary ID of packages.

This means, that there is no possible automatic way to do this, as it would be a chicken-and-egg problem.

However, you could add conditional logic to your methods, something like:

def build_requirements(self):
      if self.settings.os == "Windows" and self.settings.arch="x86":
             self.build_requires(f"dependency/1.0.2@test/test")
      else:
             self.build_requires(f"dependency/1.0.1@test/test")

However, I would very, very, very strongly recommend to be consistent in the binaries. Both versions 1.0.1 and 1.0.2 should probably build the same configurations of OS and architecture. Not doing that and trying to workaround it, will generate more headaches in the future for sure.

[Johnnyxy]

@memsharded Thx for the input. It already gives me headaches :).

My Problem is that the depedency of one specific version has serval binaries (lib, dll, so, a) for specific compile targets (a target consists of one possible settings combination). I am not able to build the dependency myself as it is externally provided which destroys any desired package consistency :(.

So I have to circumvent it by any means. I will think about it.

Edit:
Noone else uses the dependency and my code completely uses the dependency without making any traces of it public, I could insert a specific target identifier to the reipe name to identify a target.

Edit:
Ok, I will do it like you proposed and make a huge if in the recipe of my code that resolves the appropriate dependency version. This is inconvinient and redundant but the only way to solve this without any ugly "hacks".

Thx

[memsharded]

Are the binaries overlapping between versions? If not, maybe it makes sense to have a "1.0" version that aggregates the different binaries of both 1.0.1 and 1.0.2? Are you using "export-pkg" to create the package binaries from the pre-compiled artifacts?

If there is private information you don't want to disclose here, please write us to info@conan.io.

Thanks!

[Johnnyxy]

@memsharded in short:
The binaries are all separate. Each target has its complete standalone binaries package (a ZIP file in this case). Have a look at my explanation later.

For maintaining the recipes a git repository has been set up. This repository only contains the conanfile.py. Any sources or binaries have to be pulled from another server like FTP.

For local testing I use the conan-packaging-tools. That automatically builds a conan package in my local conan repository. sometimes I just use conan export, but never conan export-pkg the design should be

Conan look inside your local repository for recipes and binary packages; otherwise pull the recipe from a central server and build it.

No binary packages are distributed.

dependency structure

The third party SDK has a installer (exe) with tools to generate code.
Additionally one can extend the SDK with "targets". Those targets contain specific a/so/dll/lib for the (cross-)compile target. The generated code is "generic" and not target-specific.

I have already packed all "targets" as single ZIP files that are pulled from a server and just unzipped, as the tools do not need to be installed to work. I like portability.

So every ZIP file contains everything to generate code and link against for a target.

Now as I have available all Windows x86_64/x86 VS 10-15 for 1.0.1 and 1.02 it is no problem (for now).
But for QNX Neutrino for which I have to build for specific architectures like armv7 or x86_64 are not available for both versions (1.0.1 / 1.0.2).

I was thinking about creating a metapackage like "dependency_installer" that does nothing else than picking the right dependency version. This makes maintaining the "list" of available targets and the correct dependency version easier. The caveat is that the "conan version" of the "dependency_installer" would not reflect any meaningful version of the dependency or would be needed at all.

[memsharded]

I was thinking about creating a metapackage like "dependency_installer" that does nothing else than picking the right dependency version. This makes maintaining the "list" of available targets and the correct dependency version easier.

Yes, that could be a good idea. Like every SW problem is solved with a level of indirection. So having a recipe that abstract away the package patch version (so you would call the metapackage like "sdk-meta/1.0" which in practice will resolve to "sdk/1.0.1" or to "sdk/1.0.2".

The caveat is that the "conan version" of the "dependency_installer" would not reflect any meaningful version of the dependency or would be needed at all.

Not really, as commented, you can use the MAJOR.MINOR version of the SDK, and drop the patch. The result is still meaningful.

The conditional logic in that meta package shouldn't be difficult, if for Windows you have both. Mostly a conditional in requirements() for QNX, to select 1.0.1 or 1.0.2 based on the architecture, instead of resolving it via version ranges or whatever.

Also, you might be interested in following: #4294. Please have a look and report there, give feedback if those settings would be useful also for your use case or not. UPDATE: sorry, I just saw that you are already involved in that thread.

Thanks!

[Johnnyxy]

Yea, you are right with the version selection "major.minor". That should be sufficient.

Yes, I am already involved in #4294 cause despite Windows is the main target for development here, QNX is a absolut requirement that has to be supported by me. The initial start came from https://github.com/conan-community/community/issues/78 I think, where @SSE4 took the initiative to bring it over to Conan :).

[memsharded]

Yea, you are right with the version selection "major.minor". That should be sufficient.

Ok, great, thanks for the feedback. As it seems there is no further action needed at the moment, I am closing this, but please comment, re-open or open a new issue for any further question. Cheers!

[gallencalade]

Hi, I am a little confused. Could you tell me the design principle of that version is prefer priority? Why does conan not consider the setting first? Thank you. @memsharded