Garbage collect untagged manifests

[Vanuan]

As suggested in #1600, there should be a garbage collect option to delete all manifests that are not references by any tags (dangling).

Use case 1:

• push a new tag
• delete an old tag (by deleting a manifest)
• reclaim disk space by running garbage collection

Use case 2:

• push a latest tag (creating "manifest 1")
• push a latest tag (creating "manifest 2")
• reclaim disk space by running garbage collection (delete "manifest 1" automatically)

[Vanuan]

#1813 is a pre-requisite to this feature

[Vanuan]

I assume that the first use case should already be working:

1. fetch a list of tags: GET /v2/<name>/tags/list (sorted by create/update date)
2. for each tag, fetch a manifest digest: GET /v2/<name>/manifests/<reference> (use tag for reference)
3. delete all tags except of the last created/updated: DELETE /v2/<name>/manifests/<reference> (use digest for reference)

But there's no way to implement the second one.

[mhornbech]

In our company we use tags that match the branch name og the code, which corresponds to your use case 2. I was actually quite surprised that this wasn't the standard behaviour, since as far as I can see there is no way to retrieve the old manifest digests from the API. Our current workaround is to delete the current manifest before pushing a new one.

[bwb]

Use case 2 is extremely common.

Blobs that would be eligible for garbage collection were it not for untagged manifests consume the majority of storage space allocated to the private registries I'm responsible for.

Is anyone working on this?

[fp-pszczekutowicz]

Correct me if I'm wrong but second use case scenario is:

For each repository:

1. get list of all manifests
2. for each tag remove tag's manifest from list of all manifests
3. remove remaining manifests from storage

After above steps garbage collector will reclaim disk space.

[Vanuan]

Yeah that would be how to solve it. Though there are 3 more steps:

4. Stop registry
5. Run gc
6. Start registry

[Vanuan]

This should solve point 1: #2199
So that we can implement 1-3 through HTTP API.
4-6 still impossible without some docker commands.

[rdalverny]

Fixed by #2302 ?

[Vanuan]

Is it released yet?