Automatically evict zig-cache to stop it growing too large

[silversquirl]

I was recently freeing up some space on my laptop and came across a zig-cache 18GiB in size. This is far larger than reasonable, and it was caused by having ~45 cached copies of a large C library (Binaryen, in this case) each about 340MiB.

Zig could stop this happening by automatically removing old cache entries, either keeping cached data for the most recent n builds, or trying to keep the cache under a certain size. This being configurable per-project would be nice.

[IntegratedQuantum]

I have the same problem.
Since about a month ago, when I last deleted it, my zig-cache has already grown to 30 GB, containing ~750 versions of my project executable.

[tauoverpi]

I have the same problem with it reaching ~60GiB as I tend to run find . -name '*.zig' | entr zig build. Have dealt with it thus far with an occational find ~/ -name zig-cache -exec rm -r {} \;

[alexrp]

It seems like any limit would necessarily be arbitrary and not satisfy everyone. Maybe a zig gc command could make sense?

[silversquirl]

Then you have to remember to run zig gc. Better to have it automatically evict unless configured not to.

[kcbanner]

I like the idea of zig gc, as having it happen occasionally would imply some overhead for checking modified times on the cache during normal operation.

[silversquirl]

The overhead is minimal, and it can even be done on a separate thread since it shouldn't affect the rest of the build process. There's no real need to check modified times - probably better to keep a "journal" file that keeps track of when/how often cache entries are used, especially since modified times aren't hugely reliable to begin with.

[nektro]

this is a complexity that does not feel like it would be worth the maintenance burden when Zig can instead communicate the policy that its okay to rm -r zig-cache every once in a while, and let the users decide the right frequency

[tauoverpi]

@nektro this is an issue caused by zig where rm -r zig-cache doesn't seem like a great option as it would also mean a complete clean rebuild of everything where zig generally knows which items are required by the latest build of the project. Having zig gc just remove entries which weren't touched in the last build would be enough.

[andrewrk]

Note that zig's caching system is designed explicitly so that garbage collection could happen in one process simultaneously while the cache is being used by another process.

[bigidea-kz]

Hi guys! I have the same problem. I use raylib in one project and zig-cache folder size = 97G 😱

[andrewrk]

That's too big, delete it!

[ni-vzavalishin]

There seems to be a new subfolder created under zig-cache/o for each new state of the source code (in a single-file zig project). These subfolders simply keep adding up.

[plajjan]

I'd love to see this. I'm primarily using zig build as the low level compilation in another language called Acton. An Acton project is compiled by the acton compiler into C source code which is then compiled by the zig build system. Any Acton project can thus also be seen as a Zig project. We currently just use a single global (well, per user) cache at ~/.cache/acton, which works rather well. I would love to have that be like a LRU cache so the oldest unused stuff is removed when we hit some threshold of the total cache size. Right now we just wipe the whole cache once we hit a threshold and then have to start over. Since we compile everything from scratch, that first compilation after clearing the cache can stretch into a few minutes. If we could avoid rebuilding our base system (RTS, stdlib, builtins) as well as libc and external C libraries, that would give a much nicer and more predictable experience.

[daurnimator]

I note that this is not only the .zig-cache directory in my projects, but the global zig cache directory which is caching various dependencies: I'm finding each week I have to clear both caches to prevent run-away disk space

[a-day-old-bagel]

Yeah, I'm also excited about the possibility of automatic cleanup. I just ran WizTree to find out why my disk was full, and the zig cache for one project alone was like 140 GB. And I have more than one zig project.