WinGet Cache Management

[denelon]

We've seen a few issues related to where WinGet downloads installers, and their removal after installation.
*#343
*#2740 (comment)

We've also gotten the explicit ask for a "download" only command:
*#658

Let's have some discussion about what might make sense from an explicit cache management perspective so we can understand the scenarios and requirements a bit better.

[Trenly]

I think one or two settings make a lot of sense here, at least from the perspective of hitting the most pain points. The first would be a CacheRetentionPeriod, which would be the number of days to retain packages in the cache folder, defaulting to 0. -1 would be for retain infinitely, 0 for delete after command completion, and then 1 to 14 days for a set duration. I would expect that a setting like this could also be set through group policy. Winget would first try to see if the required file was in the cache, skipping the download if the file was present, downloading if not present. This would solve the problem of slow downloads for many users, since they could enable the cache and avoid downloading large packages multiple times.

The accompanying setting would be the PackageCacheRoot which would be the directory used for the cache. Right now packages are downloaded to the temp folder, so a cache implementation should allow for the user to specify the cache directory. This should also allow for mapped network drives to be used, so that a shared cache could be set up for all users on a local intranet.

---

The second portion of the cache, which would be the download only, could be accomplished through a new cache command. If the settings above are implemented, it would only make sense to have commands to manage the cache.

winget cache - Show details about the cache; Path to cache, number of items in cache, total size of cache. If retention period is 0, show a message that the cache is not enabled, with information on how to enable the cache.
winget cache list [[-q] query] - Show a list of the package versions present in the cache. If a retention period is specified, also show how many days until the package is removed from the cache
winget cache remove [[-q] query - Remove a package version from the cache
winget cache [add] [[-q] query] [--force] [--expire <-1..14> - Search sources for single, if there is a match, download the package to the cache. Able to force the overwrite of a package currently in the cache. Users would be able to override the CacheRetentionPeriod
winget cache reset --force - Clear the cache of all packages

It would be imperative that multiple versions of the same package could be in the cache. It would also be required that file names be unique such that even if the download for two versions would normally result in the same file name, files aren't overwritten unless they are for the same package identifier and version. This would probably involve creating a cache.db file that can correlate file path to PackageIdentifier, PackageVersion, Source, and Retention Period, but specific implementation details can be addressed later.

This would address the use case of download only scenarios, and would provide control over any packages automatically cached.

This would also necessitate an admin setting (and probably a group policy) for PackageCache - enabled by default. Interactions between the retention period and enablement would be as described below.

CacheRetentionPeriod	CacheEnabled	Expected Behavior
0 -or- Unset	Enabled	Packages are removed by default when downloaded through install. Users can still manually add and remove packages from the cache. Retention period for manually added packages would default to -1, unless the --expire argument is used
Not 0	Enabled	Packages are added to the cache by default when downloaded through install. Users can still manually add and remove packages from the cache. Retention period for manually added packages would default to the CacheRetentionPeriod, unless the --expire argument is used
-1	Disabled	Packages are removed by default when downloaded through install. Users cannot manually add or remove packages from the cache. Packages currently in the cache will not be removed. *Note* - This covers the use case of a read-only static cache
Unset -or- Not -1	Disabled	Packages are removed by default when downloaded through install. Users cannot manually add packages to the cache. Users can manually remove packages from the cache. Packages currently in the cache will be removed according to when they would have originally been removed.

[DemGiran]

wow this is still not a thing? :( Any workarounds in the meantime??

[denelon]

What specific scenario(s) is/(are) causing trouble that you're looking for a workaround?

[tadghh]

Stealing my kilobytes >:( plus just good maintenance. Sure you dont need to clean the dryers lint trap every time but eventually it could cause issues

[denelon]

Just FYI: WinGet starts purging old logs after 100 are present in the directory.

[tadghh]

Thats true but if im in a cleaning mood and theres 3 socks on the floor im not gonna wait until the floor is covered in clothes. This is still good to know thanks

[Paciupa]

Also related question

[Paciupa]

Issues I have faced with:

• Installer is already being existed locally.
  Why should I download it again?
• Machine is offline.
  A lot cases I can see: Internet is weak or expensive or just no access to Internet.
• Site is unavailable.
  For example FarManager was down recently.
  TorProject is banned in many countries, and even GitHub.
• Downloading from specific site is slow.
  LinuxReader.
• Version in Windows Package Manager repository is stale.
  Sublime Merge has two new versions (2083 and 2079) after 2077.

I'd like to have two (or maybe even more sometimes) different stages: downloading and installation.

Possible scenarios except solving issues above:

• Installation or upgrading several machines.
  I'd like to download and prepare all software once and then just execute one script on every PC / laptop.
• Always available pack of installers.
  If something goes wrong (list above) I will use the locale source.

And thanks a lot for WinGet. It is a huge step for Windows ecosystem.

[Dragon1573]

I am a contributor of microsoft/winget-pkgs.

When creating brand new or updating stale manifests using wingetcreate.exe, I SHOULD ALWAYS test the manifest locally using winget install --manifest or Tools/SandboxTest.ps1 -Manifest. I can't achieve making manifests perfect with only 1 step, so I may run (un)installation process during debugging again and again.

For well-known reasons in Mainland China, I'm behind a proxy. The proxy is billing monthly and the cost (counting in CNY per Gigabytes) is extremely expensive. I don't have enough package size to download the exactly same installer again and again during debugging.

It's nice for me to cache the installer after first download. It can save costs on my proxy billing.

[Comprox]

We currently use the paid version of Ninite. The pro version has some switches (https://ninite.com/help/features/switches.html) like /cachepath which we can specify where installers are located (and it updates them if needed, so other computers can use them) and also /cleancache which removes old versions of installers.
The big benefit of letting it use a network cache is not having hundreds of clients all downloading the same installer/update from the internet. We can keep the traffic just internal.
Thanks!

[MarcoBuess]

@denelon We have a usecase with Intune EPM where we wan't to be able to automatically elevate certain installers that require it. At this time it looks like the setup files are removed before EPM is able to validate the file, which crashes the EPM process. An option to configure a time period before the cache is cleared or some sort of --keep option would therefore be nice.

[nhatkhai]

Hi, I'm new here...
I like this feature... However, In my design I actually have 3 level caches to improve the robust run under company situation. So may be we should prove a hook for download call to a batch script or so to handling the download as if it calling the curl.exe but with addition customize caches handling.
The 3 level cache usually are:

1. Level1 Local folder in the machine
2. Level2 On a local share network
3. Level3 On a global company server like Artifactory

[tadghh]

would you still need to check the latest checksums? ex. Machine on local network is infected + scoop cache