[question] How to handle different versions of the same requirement

[bog-dan-ro]

Hi,

I'm facing this problem especially with Qt (but the same problem applies to other libs as well e.g. openssl).
Let's take an example to be easier to understand:
I want to create a package for kuesa. The problem is that the same kuesa version can be built with multiple Qt versions (e.g. 5.12.x, 5.13.x, 5.14.x), but depending on what qt version you build it, it will enable different features (e.g. kuesa built with qt 5.14 will have more features than kuesa built with 5.12).

The problem comes when application A requires Qt 5.12 and kuesa 1.0 and application B requires Qt 5.14 and kuesa 1.0. How to tell conan to download kuesa 1.0 built with Qt 5.12 for application A and kuesa 1.0 built with qt 5.14 for application B?

What is the best solution to handle this situation?

I explored a few solutions, but all of them have some disadvantages:

1. add qt version to the package naming /version@user/channel:
   1.a. add qt version to the package version: kuesa/0.10.0-qt_5.12@user/channel
   1.b use different channels for each qt version: kuesa/0.10.0@user/qt_5.12

Pro:

• the qt version is explicit in the package name

Con:

• will be challenging to create and maintain conanfile.txt (and conanfile.py) as we need to change these files to specify which qt version we like to use for each package (unless there is a way to use placeholders in both conanfile.txt and conanfile.py).

2. add qt version to conan settings and to profiles

Pro:

• no changes are need to be done to conanfile.txt nor to conanfile.py !

Con:

• local changes are needed to settings.yml for each problematic package and we need a profile for each combination (can be easily handled using conan config install).

3. add qt version to package options

Pro:

• no changes are needed to settings.yml nor to profiles
• no changes are needed to conanfile.py besides the followings:

class KuesaConan(ConanFile):
    name = "Kuesa"
# ...
    options = {"shared": [True, False], "qt":"ANY"}
    default_options = {"shared": True, "qt": "5.12"} # instead of "5.12" we can use an env var ;-)

Con:

• same as option 1, we need to specify for each package the qt version in the conanfile.txt (in the options section)

[requires]
kuesa/0.10.0@user/channel
[options]
kuesa:qt="5.12" # here it will be great if we can use a placeholder instead of "5.12"

• Even worst, I didn't find a way to specify the required packages options in the conanfile.py.

class OtherPackageConan(ConanFile):
    name = "OtherPackage"
# ...
   requires = "kuesa/0.10.0"    # How do I specify options for the requires packages?

Is there a better solution that I missed?

• I've read the CONTRIBUTING guide.

[fulara]

Hello @bog-dan-ro
Why you dont use minor_mode in kuesa for qt dependency

and in the consuming package you would have in requires both kuesa and qt.

Wouldnt this fit all your needs?

So something like this.
in kuesa conanfile you would do:

... 
   requires = "qt/5.12" # << doesnt really matter we will override it.
...
   def package_id()
         self.info.requires["qt"].minor_mode()  # this will force to recompile kuesa when you want to use it with different minor more so for example 5.13.

Now in Application A you would do:
Conanfile:

  requires="kuesa/1.0.0", "qt/5.12" # ok this matches the kuesa requires - no need to rebuild.

In application B you would do:
Conanfile:

  requires="kuesa/1.0.0", "qt/5.14"

Now when you would try to build the Application B it would tell you that you didnt prebuilt the kuesa with qt/5.14 << you can do --build kuesa or build --missing and it will build kuesa that is aware of different qt version.

I may be missing something and I am not that knowledgeable in conan but above should work?

[bog-dan-ro]

Thanks a lot for your reply!

Building the dependencies for package B is what we try to avoid by using conan. It's needed to decrease the time needed by our CI to check every patch.

I think we'll go with solution 2: we'll add a qt: [None,ANY] in settings.yml, set it in our profiles and, if needed, set it in conanfile.py. This way we'll have the same package version prebuilt for multiple qt version.

I hope in conan 2.0 to have an easier way to do it.

[fulara]

Hello @bog-dan-ro

After you did --build missing you can can upload that kuesa binary with 'conan upload '*' for example.

After that you wouldn't need to recompile kuesa or qt in that particular configuration again.

[bog-dan-ro]

Hi @fulara,

I did a quick test with your suggestions, but, sadly, conan (1.21.1) doesn't download the right kuesa package, the one built with a different Qt version.

These are my test files:

# qt/conanfile.py
import os

from conans import ConanFile, tools

class QtConan(ConanFile):
    name = "Qt"
    version = "5.12" #build it twice, once with 5.12 and once with 5.14
    no_copy_source = True

    def source(self):
        tools.save("qt.h", "#pragma once\n#define QT_VERSION %s" % self.version);

    def package(self):
        self.copy("*.h", "include")

# kuesa/connanfile.py
import os

from conans import ConanFile, tools


class KuesaConan(ConanFile):
    name = "kuesa"
    version = "1.0"
    no_copy_source = True
    requires = "Qt/5.12@test/stable" # << doesnt really matter we will override it.

    def source(self):
        tools.save("kuesa.h", "#pragma once\n// NEEDED QT_VERSION %s" % self.info.requires["Qt"].dumps());

    def package(self):
        self.info.requires["Qt"].minor_mode()
        self.copy("*.h", "include")

# test conanfile.txt
[requires]
Qt/5.14@test/stable # <-- downloads Qt 5.14
kuesa/0.10.0@test/stable # <-- downloads kuesa built with Qt 5.12 ;-(

$ conan install . -g cmake
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=None
compiler=gcc
compiler.libcxx=libstdc++11
compiler.version=9
os=Linux
os_build=Linux
[options]
[build_requires]
[env]

WARN: kuesa/1.0@test/stable: requirement Qt/5.12@test/stable overridden by your conanfile to Qt/5.14@test/stable 
conanfile.txt: Installing package
Requirements
    Qt/5.14@test/stable from local cache - Cache
    kuesa/1.0@test/stable from local cache - Cache
Packages
    Qt/5.14@test/stable:5ab84d6acfe1f23c4fae0ab88f26e3a396351ac9 - Cache
    kuesa/1.0@test/stable:0211f0c48061208a6b5a44b20f266f049ad559e7 - Cache

Qt/5.14@test/stable: Already installed!
kuesa/1.0@test/stable: Already installed!

It warns me about the requirements override, but it still using the version built with 5.12.

[fulara]

Hello @bog-dan-ro
It seems you are victim of common problem :) I had this same issue many times.

def package(self):
        self.info.requires["Qt"].minor_mode()
        self.copy("*.h", "include")

minor_mode() switch does not go into def package i wrote in my original text that it goes into
def package_id
see here: https://docs.conan.io/en/latest/creating_packages/define_abi_compatibility.html#versioning-schema

Scroll down a little bit and you will see:

def package_id(self):
  self.info.requires["MyOtherLib"].semver_mode()

After that be sure to check whats the conanmanifest of your package! you will see there that it will probably have qt/5.12.Z and for other qt/5.14.Z

Let me know if it works for you now :)

[bog-dan-ro]

Hi @fulara

Many thanks, it works perfectly!

Now I need to figure out how to pre-build these packages for all qt versions ;-) .

[memsharded]

Hi!

Thanks very much @fulara for the great explanations.

Indeed the default semver_mode is probably not the best default, and we might change that in Conan 2.0. This is the issue to track: #3762

At the moment it sounds difficult to do it, but I think it might be worth in 2.0 too to try to define a more comprehensive scope checking, and raise an error if self.info is accessed outside of the package_id() method. I have created an issue for it too: #6409

So if the above is working, and with these 2 things the UX will improve, maybe we can close this issue?

Thanks for all the feedback!

[bog-dan-ro]

@memsharded yes, please close it, though without #6384 it will be hard to package Qt 5.14.x for Android :) .

[memsharded]

Closing this as outdated/solved in Conan 2