Official mechanism to backfill/pin older library versions [was: multiple versions]

[YehezkelShB]

Sometimes there is a reason to use a specific older version of a library. Usually, it happens due to API changes the project can't absorb right now or 3rd-party libraries that aren't compatible with the newer version (yet).

Seems like a good idea to allow installation of an older version and maybe even having a few versions installed side-by-side.

[Telokis]

This is indeed a very nice feature but the actual state of the project doesn't really allow something like that.
Like said in the FAQ, it's like a dev version of the library that is used.

[operativeF]

Preferred libraries are the most up-to-date, stable libraries though?

[Telokis]

Yes. By "dev version" they mean "built from the sources" version.

[corporateshark]

This can be partially solved by adding a major version number to the package name. Here is a relevant example #26

[equalsraf]

This can be partially solved by adding a major version number to the package name. Here is a relevant example #26

Is there a way to mark two ports as being mutually exclusive?

[corporateshark]

As of right now, I don't think so. But should be relatively easy to add it.

[Telokis]

Supporting a major version number is only one step away from supporting a "real" version number.
We could easily add another level of directories used only to distinct the version number.

    buildtrees
        |--- boost
            |--- src

would become

    buildtrees
        |--- boost
            |--- 1.61.1
                |--- src

And the CMake file responsible for finding the library required would also have to be able to look for a precise version. (Maybe defaulting to the most recent one if no specific version is asked.)

[ras0219-msft]

The criteria for defining whether it's appropriate to split package names on major version is "Can I include and link multiple major versions of this library at the same time?" If the answer is yes, then the package should certainly be split on major version (eigen3, libpng16, etc).

However, if the answer is no (i.e. if you would need to mark the libraries as mutually exclusive), then they should be the same package.

If you (locally, for a specific project) need to change the library version, you can replace the portfile. If it's an older version in the repository, you can git checkout <refspec> -- ports\<library>. Essentially, the "version you want" is recorded in the portfile itself instead of in the command history of vcpkg install <x>.

[Telokis]

That is true for, as you stated, a local specific project.
But if someone wants to use vcpkg as a global package manager, wouldn't it be nice to be able to choose a specific version ?
That could be handled on generation time by a include (vcpkg) line in a CMake file.
And this would install the libraries needed (to the proper version) by reading a config file placed at the project root.
Of course, this would induce a lot of changed to the current system.

[ras0219-msft]

The issue is what happens when you have two projects which claim to require boost 1.55 and boost 1.60. Do you install both? What if they're libraries and you want to link both of them into the same process? Now you have multiple copies of boost in your process; I hope you aren't passing boost objects over any boundaries or relying on any global state to be shared.

Instead, the approach vcpkg takes is that you specify the versions you want for your system. Every library you build will build against the same set of libraries, guaranteeing compatibility with each other. If you really like boost 1.55, you can certainly set that as your boost version by pinning the port\boost directory back to the old version in your repo that you've run vcpkg integrate install on.

[YehezkelShB]

This is a great example. How can I get Boost 1.55? I can't find it in the project history...

The point is, I understand the reasons behind not adding a direct support for library versioning, but then it means that it supports only library versions since vcpkg creation, because no one bothered to add older versions and now it's too late (assuming you will not accept a series of PRs to e.g. change Boost version to 1.53, then to 1.55, then 1.58 and then back to 1.61, assuming those are the "major" versions people use).

[Telokis]

I understand what you mean but we could just change the meaning install saying it wouldn't install globally but per-project.
Requiring a library would

1. Require the latest available version if no version is specified or
2. Require the specified version if it is available.

By the way, I only think about CMake-based projects while thinking about solutions. I never really create projects based solely on Visual Studio. Maybe the problem is more easily spotted when you are used to Visual Studio-based projects.

[YehezkelShB]

Just to make it clear, we have two topics here:

1. Having vcpkg support for specifying library version,
2. Allowing side-by-side installation.

The second one can be resolved by having pinned ports folder with the project as suggested above and mentioned in the FAQ and there are good arguments why to leave it like this.
Still, without a way to add older versions to ports in vcpkg repo, I don't see how it's going to be used.
Implementation of the first point without the second point seems possible with making the packages mutually exclusive to install, as suggested above.

[ras0219-msft]

Yes, we definitely need a way to backfill older ports and make them discoverable.

Perhaps an alternate branch/tree in the git repository that can be automatically searched, either via git grep or even a top level command like vcpkg pin <library> <version>?

[mikebmcl]

What if they're libraries and you want to link both of them into the same process? Now you have multiple copies of boost in your process; I hope you aren't passing boost objects over any boundaries or relying on any global state to be shared.

I believe this would be a problem even without using vcpkg, though I understand the desire to make vcpkg as robust as possible. I admit that I have only a cursory understanding of vcpkg's internal mechanisms. Right now it seems like vcpkg just puts all headers from all installed packages into one dir, all LIB files into one dir, etc. If it hasn't already, this is almost definitely going to create collisions in the future.

I'm not sure how hard it would be to implement (assuming it's even possible), but I can imagine a system where there is one port that is "packagename" and potentially multiple other ports that are "packagename-version". If someone just includes the headers for "packagename, they will get the most current version (vcpkg would redirect to the subdir that has the most current version; see below). Alternatively, if they need a specific version (that is available through vcpkg), they would add a preprocessor define to their project properties that would be something like VCPKG_packagename_version which in turn would result in vcpkg setting the appropriate environment variables so that the requested version would be where the includes, libs, dlls, etc., would be drawn from.

This, if possible, would both eliminate the potential problems that could result from the current shared directories design while also making side-by-side library versions available and usable by separate projects. As described, it wouldn't address any attempts to link the same libraries or libraries with the same exports. But to me that's a separate issue and one that people trying to do that without vcpkg would also face.

Just an idea. Maybe it's workable. If not, maybe it will help spark something that is.

[fcooper8472]

Any new thoughts on this? The vcpkg pin <library> <version> suggestion (@ras0219-msft) seems excellent to me.

Another potential issue would be propagating fixes to ports back to any older versions that may need the same fix.

[soroshsabz]

I think PR #11758 can resolve this requirement

[JonLiu1993]

Hi @YehezkelShB
PR #11758 can resolve this requirement
Do you think it can meet your expectations

[YehezkelShB]

@JonLiu1993 Sounds promising! Thanks.

[soroshsabz]

@vicroms I think we can close this issue, if link this issue with some documentation to show how to doing this.

[PhoebeHui]

Hi All, Installing an old or specific version in vcpkg is supported now, see versioning for details, please file a new issue if you encounter a problem.