How to automatically expire a package?

[AliAskari]

In our CI and workflow we are using Conan channels just like git branches. Meaning when a developer starts working on a feature branch and pushes that branch to our servers in the CI we release the Conan package under a channel with the same name as the git feature branch. Once the package is stable enough and has passed all tests we release it on a ReleaseCandidate (RC) channel. Then the package might stay on that channel for a few weeks and then it is merged to master. Now the problem is that all of the consumers of the package have the RC package in their local cache and they have to manually clean up their cache.

It would be very nice if a package could also have an expiry date and when the package is expired it is automatically removed from local cache, and even from the server.

• The expiry date is calculated from the last time a package was updated (Either the recipe or the package itself).
• The expiry date should not work if the package is released under a list of predefined channels (stable/master/...).
• Preferably the expiry option takes two dates, one for local cache and one for server.
• The package should not be deleted if it is being used as a dependency in the stable/master channel of another library.

We are doing the server part automatically on our server in a cron job but we cannot automate this on the package consumers machines.

[memsharded]

Regarding automatic expiration, I think it is a very risky and dangerous feature. I am not opposed to some feature that allows to search (and also remove those found) by date, or something like that. So users willing to clean, can directly clean old packages with the pattern they want. Same for the server. Wiping machines caches is not a big issue, because they are just that: caches. But removing packages from server automatically, being the server the ultimate and "true" location for the binaries, it is something that conan packages should never do.

So I think we can have a look to a potential feature that allows searching by date. It still has some challenges, because which packages will you remove? Imagine that you have an old binary for VS12, and a new binary for VS11: will you just remove the VS12 one? Do you require all binaries to be outdated?

[claasd]

How about adding a optional field to the conanfile, like supported_until or best_before. This informs user if the component should not be used anymore. If the the search would support searching by these fields, you can put your own removal logic into place.
I have the connected request by our users, they want to tag their packages with a support date that tells consumers that the package is not supported anymore and should not be used. (but it's a minor feature).

[memsharded]

That could make more sense. As it is something really, really explicit, defined by the package creator, and the clean would be also explicit... lets look into it.

[lasote]

I don't think the field in the conanfile is a good idea either. These kinds of "meta-processes" or maintenance processes should not be mixed with the recipes logic at all.
Maybe related with #1233
We shouldn't perform complex tasks to detect if some other package in the local cache is using that package either. So I'm sorry I can't provide yet a better approach to solve this, we definitely should think about it a little more.

The local cache is, at the end, a cache. Do you really need to keep it and not just remove it sometimes?

[skizzay]

I feel that retention policies like this are beyond the scope of conan (except for local caches) unless we want to be opinionated on that topic.

Some companies may feel like keeping a package around until they start running out of disk space. Another company may feel like removing a package after N-many version increments. Another company may base their retention on date, SCM strategy, clearance (for those of us who have worked in classified environments), etc...

I've also worked with Artifactory in the past and know that their API can be particularly useful in the use-case stated. I think that once a package is put into the repository (conan server or Artifactory), it's easy enough to have an external process resolve which packages should be retained and which packages can be discarded depending on the firm's individual retention policay.

[memsharded]

Closing this question as outdated/wont' fix. For related follow ups:

• [feature] Track artifacts in Conan cache, expire least recently used #7727 will handle the case of expiring packages in the Conan cache (with a explicit command)
• The repo https://github.com/conan-io/conan-extensions will contain extensions like Artifactory specific commands that can set properties and other metadata on packages that can be later used for expiring packages.