[question] create a shippable deployment

[thorsten-klein]

Hello,
we store our conan recipes in our own JFrog artifactory, so that they can be consumed by our customers.
Now we need to ship these packages to some colleages, which do not have access to our artifactory.
Is there any deployable file format for this purpose?

I would expect something similar to docker save, which stores an container image as file. This file can then be shipped and imported on another PC via docker load.

Would it make sense to introduce some mechanism for this in conan 2.0?

conan save mylib/1.0.0@user/channel -o myfile.tar.gz # export the recipe only (export + exports_sources)
conan save mylib/1.0.0@user/channel -p <packageid1> -p <packageid2> -o myfile.tar.gz # + precompiled packageids
conan load myfile.tar.gz # load the conan package into local cache
conan load myfile.tar.gz user1/channel1 # combines "load" + "copy"

Background:
Currently we do not know, which packages or old RREVs of a package we can remove. A customer has maybe used it (with an explicit #RREV). If we remove the package now, we will break his (old) builds. Therefore we want to have a possibility to provide packages without "releasing" them into an artifactory. Even having a "development remote", this is not a solution for us. people may use the explicit package with #RREV.

This feature would also be helpful for CI system / pipelines:

The CI does not need to upload any conan package to artifactory anymore, but instead it can create tar.gz artifacts.
Only in a last step (if all previous steps succeeded), the CI may upload the conan package to artifactory (in case of a release).
During development the package is not consumed via artifactory but via CI artifact (which the developer can download and load locally). As a result there is no need to upload all packages to artifactory anymore and waste a lot of space.

Further Refs: #4316

• I've read the CONTRIBUTING guide.

[memsharded]

Hi @thorsten-klein

The answer might be different, also depending if you want to release to the final user as Conan packages or not.

Conan 2.0 includes a new feature for the later, using new --deploy=mydeployer it is possible to get copies from artifacts from the Conan cache, and to operations over them, including running later generators like CMakeDeps that point to the copy, and not to the Conan cache.

If you want to distribute it as Conan package, get it from one cache and copy it in the other, no, it is not possible at the moment. We have done some first steps towards it in Conan 2.0, but this is not easy, as the Conan cache structure is more complicated than the docker one. We have some progress towards extracting the artifacts layout and being able to do a copy and zip them, but recovering them is more complicated, because the DB, etc. If something, we could talk about it in Conan 2.X (after 2.0 is released)

[thorsten-klein]

We build a lot of packages in our CI (parallel jobs).
If one job fails, the other jobs maybe already have uploaded some packages to the remote.
It is very hard for us to do housekeeping and remove all the obsolete recipes + packages.

It would be nice if there would be some possibility, so that we can deploy artifact in our CI instead of pushing to remotes.
Only if all jobs were successful, the next stage can load the artifacts into the conan cache and upload them

Some proposal to export from local cache:

conan cache export mylib/1.0.0@user/channel#RREV -o myfile.tar.gz --only-recipe # only recipe and source code
conan cache export mylib/1.0.0@user/channel#RREV -o myfile.tar.gz # recipe and source code + all available PKGREVs
conan cache export mylib/1.0.0@user/channel#RREV:PKGREV -o myfile.tar.gz # recipe and source code + one specific PKGREV
conan cache export '*' -o mycache.tar.gz --only-recipe # recipe and source code of all local packages

Some proposal to import into local cache:

conan cache import myfile.tar.gz
conan cache import myfile.tar.gz --only-recipe

This would also be benefitial if users want to hand over packages to people who do not have access to the same conan remotes. This is the case very often in big companies (different teams have different artifactories), or across companies (company A develops software for company B).

[memsharded]

Implemented in #14923, for next 2.0.14 release.