Introduce v1alpha3 and make the garbage collection TTL configurable

README.md

@@ -9,6 +9,7 @@ Gardener extension controller which deploys pull-through caches for container re
 
 - [Configuring the Registry Cache Extension](docs/usage/registry-cache/configuration.md) - learn what is the use-case for a pull-through cache, how to enable it and configure it
 - [How to provide credentials for upstream repository?](docs/usage/registry-cache/upstream-credentials.md)
+- [Migration from `v1alpha2` to `v1alpha3`](docs/usage/registry-cache/migration-from-v1alpha2-to-v1alpha3.md) - learn how to migrate from the `v1alpha2` API version of the `RegistryConfig` to `v1alpha3`
 - [Configuring the Registry Mirror Extension](docs/usage/registry-mirror/configuration.md) - learn what is the use-case for a registry mirror, how to enable and configure it
 
 ## Local setup and development

docs/usage/registry-cache/configuration.md

@@ -40,7 +40,7 @@ spec:
   extensions:
   - type: registry-cache
     providerConfig:
-      apiVersion: registry.extensions.gardener.cloud/v1alpha2
+      apiVersion: registry.extensions.gardener.cloud/v1alpha3
       kind: RegistryConfig
       caches:
       - upstream: docker.io
@@ -50,7 +50,7 @@ spec:
       - upstream: ghcr.io
       - upstream: quay.io
         garbageCollection:
-          enabled: false
+          ttl: 0s
         secretReferenceName: quay-credentials
   # ...
   resources:
@@ -77,15 +77,15 @@ This field is immutable. See [Increase the cache disk size](#increase-the-cache-
 The `providerConfig.caches[].volume.storageClassName` field is the name of the StorageClass used by the registry cache volume.
 This field is immutable. If the field is not specified, then the [default StorageClass](https://kubernetes.io/docs/concepts/storage/storage-classes/#default-storageclass) will be used.
 
-The `providerConfig.caches[].garbageCollection.enabled` field enables/disables the cache's garbage collection. Defaults to `true`. The time to live (ttl) for an image is `7d`. See the [garbage collection section](#garbage-collection) for more details.
+The `providerConfig.caches[].garbageCollection.ttl` field is the time to live of a blob in the cache. If the field is set to `0s`, the garbage collection is disabled. Defaults to `168h` (7 days). See the [garbage collection section](#garbage-collection) for more details.
 
 The `providerConfig.caches[].secretReferenceName` is the name of the reference for the Secret containing the upstream registry credentials. To cache images from a private registry, credentials to the upstream registry should be supplied. For details see [How to provide credentials for upstream registry](upstream-credentials.md#how-to-provide-credentials-for-upstream-registry).
 
 > **Note**: It's only possible to provide one set of credentials for one private upstream registry.
 
 ## Garbage Collection
 
-When the registry cache receives a request for an image that is not present in its local store, it fetches the image from the upstream, returns it to the client and stores the image in the local store. The registry cache runs a scheduler that deletes images when their time to live (ttl) expires. When adding an image to the local store, the registry cache also adds a time to live for the image. The ttl value is `7d`. Requesting an image from the registry cache does not extend the time to live of the image. Hence, an image is always garbage collected from the registry cache store after `7d`.
+When the registry cache receives a request for an image that is not present in its local store, it fetches the image from the upstream, returns it to the client and stores the image in the local store. The registry cache runs a scheduler that deletes images when their time to live (ttl) expires. When adding an image to the local store, the registry cache also adds a time to live for the image. The ttl defaults to `168h` (7 days) and is configurable. The garbage collection can be disabled by setting the ttl to `0s`. Requesting an image from the registry cache does not extend the time to live of the image. Hence, an image is always garbage collected from the registry cache store when its ttl expires.
 At the time of writing this document, there is no functionality for garbage collection based on disk size - e.g. garbage collecting images when a certain disk usage threshold is passed.
 The garbage collection cannot be enabled once it is disabled. This constraint is added to mitigate [distribution/distribution#4249](https://github.com/distribution/distribution/issues/4249).
 

docs/usage/registry-cache/migration-from-v1alpha2-to-v1alpha3.md

@@ -0,0 +1,43 @@
+# Migration from `v1alpha2` to `v1alpha3`
+
+This document descibres how to migrate from API version `registry.extensions.gardener.cloud/v1alpha2` of the `RegistryConfig` to `registry.extensions.gardener.cloud/v1alpha3`.
+
+The `registry.extensions.gardener.cloud/v1alpha2` is deprecated and will be removed in a future version. Use `registry.extensions.gardener.cloud/v1alpha3` instead.
+
+Let's first inspect how the `RegistryConfig` looks like in API version `registry.extensions.gardener.cloud/v1alpha2`:
+
+```yaml
+apiVersion: registry.extensions.gardener.cloud/v1alpha2
+kind: RegistryConfig
+caches:
+- upstream: docker.io
+  volume:
+    size: 10Gi
+    storageClassName: default
+  garbageCollection:
+    enabled: true
+  secretReferenceName: docker-credentials
+```
+
+The translation of the above `RegistryConfig` in API version `registry.extensions.gardener.cloud/v1alpha3` is:
+
+```yaml
+apiVersion: registry.extensions.gardener.cloud/v1alpha3
+kind: RegistryConfig
+caches:
+- upstream: docker.io
+  volume:
+    size: 10Gi
+    storageClassName: default
+  garbageCollection:
+    ttl: 168h
+  secretReferenceName: docker-credentials
+```
+
+As you can notice, there is one breaking change in API version `registry.extensions.gardener.cloud/v1alpha3` - the `caches[].garbageCollection.enabled` field is replaced by `caches[].garbageCollection.ttl`.
+
+`v1alpha2` was an API created against `distribution/distribution@2.8`. The registry-cache evolves and upgraded recently to `distribution/distribution@3.0`. In `distribution/distribution@2.8` the ttl is not configurable and it is hard-coded to `168h` (7 days). That's why in versions prior to `v1alpha3` the `storage.delete.enabled` config field in the `distribution/distribution` configuration was used to control deletion of blobs. `distribution/distribution@3.0` exposes the `proxy.ttl` config field. It is now possible to natively disable the garbage collection (expiration of blobs) by setting `proxy.ttl=0`.
+
+Conversions are as follows:
+- `garbageCollection.enable=true` gets converted to `garbageCollection.ttl=168h` and vice versa (a positive `garbageCollection.ttl` duration is considered `garbageCollection.enable=true`)
+- `garbageCollection.enable=false` gets converted to `garbageCollection.ttl=0` vice versa

docs/usage/registry-cache/upstream-credentials.md

@@ -38,7 +38,7 @@ In order to pull private images through registry cache, it is required to supply
      extensions:
      - type: registry-cache
        providerConfig:
-         apiVersion: registry.extensions.gardener.cloud/v1alpha2
+         apiVersion: registry.extensions.gardener.cloud/v1alpha3
          kind: RegistryConfig
          caches:
          - upstream: docker.io

example/shoot-registry-cache.yaml

@@ -15,7 +15,7 @@ spec:
   extensions:
   - type: registry-cache
     providerConfig:
-      apiVersion: registry.extensions.gardener.cloud/v1alpha2
+      apiVersion: registry.extensions.gardener.cloud/v1alpha3
       kind: RegistryConfig
       caches:
       - upstream: docker.io
@@ -24,7 +24,7 @@ spec:
       - upstream: ghcr.io
       - upstream: quay.io
         garbageCollection:
-          enabled: false
+          ttl: 0s
   networking:
     type: calico
   provider:

hack/api-reference/registry.md

@@ -1,20 +1,20 @@
 <p>Packages:</p>
 <ul>
 <li>
-<a href="#registry.extensions.gardener.cloud%2fv1alpha2">registry.extensions.gardener.cloud/v1alpha2</a>
+<a href="#registry.extensions.gardener.cloud%2fv1alpha3">registry.extensions.gardener.cloud/v1alpha3</a>
 </li>
 </ul>
-<h2 id="registry.extensions.gardener.cloud/v1alpha2">registry.extensions.gardener.cloud/v1alpha2</h2>
+<h2 id="registry.extensions.gardener.cloud/v1alpha3">registry.extensions.gardener.cloud/v1alpha3</h2>
 <p>
-<p>Package v1alpha2 is a version of the API.</p>
+<p>Package v1alpha3 is a version of the API.</p>
 </p>
 Resource Types:
 <ul></ul>
-<h3 id="registry.extensions.gardener.cloud/v1alpha2.GarbageCollection">GarbageCollection
+<h3 id="registry.extensions.gardener.cloud/v1alpha3.GarbageCollection">GarbageCollection
 </h3>
 <p>
 (<em>Appears on:</em>
-<a href="#registry.extensions.gardener.cloud/v1alpha2.RegistryCache">RegistryCache</a>)
+<a href="#registry.extensions.gardener.cloud/v1alpha3.RegistryCache">RegistryCache</a>)
 </p>
 <p>
 <p>GarbageCollection contains settings for the garbage collection of content from the cache.</p>
@@ -29,22 +29,26 @@ Resource Types:
 <tbody>
 <tr>
 <td>
-<code>enabled</code></br>
+<code>ttl</code></br>
 <em>
-bool
+<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.25/#duration-v1-meta">
+Kubernetes meta/v1.Duration
+</a>
 </em>
 </td>
 <td>
-<p>Enabled indicates whether the garbage collection is enabled.</p>
+<p>TTL is the time to live of a blob in the cache.
+Set to 0s to disable the garbage collection.
+Defaults to 168h (7 days).</p>
 </td>
 </tr>
 </tbody>
 </table>
-<h3 id="registry.extensions.gardener.cloud/v1alpha2.RegistryCache">RegistryCache
+<h3 id="registry.extensions.gardener.cloud/v1alpha3.RegistryCache">RegistryCache
 </h3>
 <p>
 (<em>Appears on:</em>
-<a href="#registry.extensions.gardener.cloud/v1alpha2.RegistryConfig">RegistryConfig</a>)
+<a href="#registry.extensions.gardener.cloud/v1alpha3.RegistryConfig">RegistryConfig</a>)
 </p>
 <p>
 <p>RegistryCache represents a registry cache to deploy.</p>
@@ -73,7 +77,7 @@ The value must be a valid DNS subdomain (RFC 1123).</p>
 <td>
 <code>volume</code></br>
 <em>
-<a href="#registry.extensions.gardener.cloud/v1alpha2.Volume">
+<a href="#registry.extensions.gardener.cloud/v1alpha3.Volume">
 Volume
 </a>
 </em>
@@ -87,7 +91,7 @@ Volume
 <td>
 <code>garbageCollection</code></br>
 <em>
-<a href="#registry.extensions.gardener.cloud/v1alpha2.GarbageCollection">
+<a href="#registry.extensions.gardener.cloud/v1alpha3.GarbageCollection">
 GarbageCollection
 </a>
 </em>
@@ -112,11 +116,11 @@ string
 </tr>
 </tbody>
 </table>
-<h3 id="registry.extensions.gardener.cloud/v1alpha2.RegistryCacheStatus">RegistryCacheStatus
+<h3 id="registry.extensions.gardener.cloud/v1alpha3.RegistryCacheStatus">RegistryCacheStatus
 </h3>
 <p>
 (<em>Appears on:</em>
-<a href="#registry.extensions.gardener.cloud/v1alpha2.RegistryStatus">RegistryStatus</a>)
+<a href="#registry.extensions.gardener.cloud/v1alpha3.RegistryStatus">RegistryStatus</a>)
 </p>
 <p>
 <p>RegistryCacheStatus represents a deployed registry cache.</p>
@@ -154,7 +158,7 @@ Example: &ldquo;<a href="http://10.4.246.205:5000&quot;">http://10.4.246.205:500
 </tr>
 </tbody>
 </table>
-<h3 id="registry.extensions.gardener.cloud/v1alpha2.RegistryConfig">RegistryConfig
+<h3 id="registry.extensions.gardener.cloud/v1alpha3.RegistryConfig">RegistryConfig
 </h3>
 <p>
 <p>RegistryConfig contains information about registry caches to deploy.</p>
@@ -171,7 +175,7 @@ Example: &ldquo;<a href="http://10.4.246.205:5000&quot;">http://10.4.246.205:500
 <td>
 <code>caches</code></br>
 <em>
-<a href="#registry.extensions.gardener.cloud/v1alpha2.RegistryCache">
+<a href="#registry.extensions.gardener.cloud/v1alpha3.RegistryCache">
 []RegistryCache
 </a>
 </em>
@@ -182,7 +186,7 @@ Example: &ldquo;<a href="http://10.4.246.205:5000&quot;">http://10.4.246.205:500
 </tr>
 </tbody>
 </table>
-<h3 id="registry.extensions.gardener.cloud/v1alpha2.RegistryStatus">RegistryStatus
+<h3 id="registry.extensions.gardener.cloud/v1alpha3.RegistryStatus">RegistryStatus
 </h3>
 <p>
 <p>RegistryStatus contains information about deployed registry caches.</p>
@@ -199,7 +203,7 @@ Example: &ldquo;<a href="http://10.4.246.205:5000&quot;">http://10.4.246.205:500
 <td>
 <code>caches</code></br>
 <em>
-<a href="#registry.extensions.gardener.cloud/v1alpha2.RegistryCacheStatus">
+<a href="#registry.extensions.gardener.cloud/v1alpha3.RegistryCacheStatus">
 []RegistryCacheStatus
 </a>
 </em>
@@ -210,11 +214,11 @@ Example: &ldquo;<a href="http://10.4.246.205:5000&quot;">http://10.4.246.205:500
 </tr>
 </tbody>
 </table>
-<h3 id="registry.extensions.gardener.cloud/v1alpha2.Volume">Volume
+<h3 id="registry.extensions.gardener.cloud/v1alpha3.Volume">Volume
 </h3>
 <p>
 (<em>Appears on:</em>
-<a href="#registry.extensions.gardener.cloud/v1alpha2.RegistryCache">RegistryCache</a>)
+<a href="#registry.extensions.gardener.cloud/v1alpha3.RegistryCache">RegistryCache</a>)
 </p>
 <p>
 <p>Volume contains settings for the registry cache volume.</p>

hack/update-codegen.sh

@@ -39,7 +39,7 @@ bash "${PROJECT_ROOT}"/vendor/k8s.io/code-generator/generate-internal-groups.sh
   github.com/gardener/gardener-extension-registry-cache/pkg/client \
   github.com/gardener/gardener-extension-registry-cache/pkg/apis \
   github.com/gardener/gardener-extension-registry-cache/pkg/apis \
-  "registry:v1alpha2" \
+  "registry:v1alpha2,v1alpha3" \
   --go-header-file "${PROJECT_ROOT}/vendor/github.com/gardener/gardener/hack/LICENSE_BOILERPLATE.txt"
 
 bash "${PROJECT_ROOT}"/vendor/k8s.io/code-generator/generate-internal-groups.sh \

pkg/apis/config/v1alpha1/register.go

@@ -25,16 +25,6 @@ const GroupName = "config.registry.extensions.gardener.cloud"
 // SchemeGroupVersion is group version used to register these objects
 var SchemeGroupVersion = schema.GroupVersion{Group: GroupName, Version: "v1alpha1"}
 
-// Resource takes an unqualified resource and returns a Group qualified GroupResource
-func Resource(resource string) schema.GroupResource {
-	return SchemeGroupVersion.WithResource(resource).GroupResource()
-}
-
-// Kind takes an unqualified kind and returns a Group qualified GroupKind
-func Kind(kind string) schema.GroupKind {
-	return SchemeGroupVersion.WithKind(kind).GroupKind()
-}
-
 var (
 	// SchemeBuilder used to register the Configuration resource.
 	localSchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)

pkg/apis/mirror/v1alpha1/register.go

@@ -25,16 +25,6 @@ const GroupName = "mirror.extensions.gardener.cloud"
 // SchemeGroupVersion is group version used to register these objects
 var SchemeGroupVersion = schema.GroupVersion{Group: GroupName, Version: "v1alpha1"}
 
-// Resource takes an unqualified resource and returns a Group qualified GroupResource
-func Resource(resource string) schema.GroupResource {
-	return SchemeGroupVersion.WithResource(resource).GroupResource()
-}
-
-// Kind takes an unqualified kind and returns a Group qualified GroupKind
-func Kind(kind string) schema.GroupKind {
-	return SchemeGroupVersion.WithKind(kind).GroupKind()
-}
-
 var (
 	// SchemeBuilder used to register the Configuration resource.
 	localSchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)

pkg/apis/registry/helper/helper.go

@@ -16,13 +16,23 @@ package helper
 
 import (
 	"k8s.io/apimachinery/pkg/api/resource"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 
 	"github.com/gardener/gardener-extension-registry-cache/pkg/apis/registry"
 )
 
-// GarbageCollectionEnabled returns whether the garbage collection is enabled for the given cache.
+// GarbageCollectionEnabled returns whether the garbage collection is enabled (ttl > 0) for the given cache.
 func GarbageCollectionEnabled(cache *registry.RegistryCache) bool {
-	return cache.GarbageCollection == nil || cache.GarbageCollection.Enabled
+	return GarbageCollectionTTL(cache).Duration > 0
+}
+
+// GarbageCollectionTTL returns the time to live of a blob in the given cache.
+func GarbageCollectionTTL(cache *registry.RegistryCache) metav1.Duration {
+	if cache.GarbageCollection == nil {
+		return registry.DefaultTTL
+	}
+
+	return cache.GarbageCollection.TTL
 }
 
 // FindCacheByUpstream finds a cache by upstream.

pkg/apis/registry/helper/helper_test.go

@@ -16,10 +16,12 @@ package helper_test
 
 import (
 	"testing"
+	"time"
 
 	. "github.com/onsi/ginkgo/v2"
 	. "github.com/onsi/gomega"
 	"k8s.io/apimachinery/pkg/api/resource"
+	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/utils/ptr"
 
 	"github.com/gardener/gardener-extension-registry-cache/pkg/apis/registry"
@@ -38,9 +40,36 @@ var _ = Describe("Helpers", func() {
 		func(cache *registry.RegistryCache, expected bool) {
 			Expect(helper.GarbageCollectionEnabled(cache)).To(Equal(expected))
 		},
-		Entry("garbageCollection is nil", &registry.RegistryCache{GarbageCollection: nil}, true),
-		Entry("garbageCollection.enabled is false", &registry.RegistryCache{GarbageCollection: &registry.GarbageCollection{Enabled: false}}, false),
-		Entry("garbageCollection.enabled is true", &registry.RegistryCache{GarbageCollection: &registry.GarbageCollection{Enabled: true}}, true),
+		Entry("garbageCollection is nil",
+			&registry.RegistryCache{GarbageCollection: nil},
+			true,
+		),
+		Entry("garbageCollection.ttl is zero",
+			&registry.RegistryCache{GarbageCollection: &registry.GarbageCollection{TTL: metav1.Duration{Duration: 0}}},
+			false,
+		),
+		Entry("garbageCollection.ttl is a positive duration",
+			&registry.RegistryCache{GarbageCollection: &registry.GarbageCollection{TTL: metav1.Duration{Duration: 30 * 24 * time.Hour}}},
+			true,
+		),
+	)
+
+	DescribeTable("#GarbageCollectionTTL",
+		func(cache *registry.RegistryCache, expected metav1.Duration) {
+			Expect(helper.GarbageCollectionTTL(cache)).To(Equal(expected))
+		},
+		Entry("garbageCollection is nil",
+			&registry.RegistryCache{GarbageCollection: nil},
+			metav1.Duration{Duration: 7 * 24 * time.Hour},
+		),
+		Entry("garbageCollection.ttl is zero",
+			&registry.RegistryCache{GarbageCollection: &registry.GarbageCollection{TTL: metav1.Duration{Duration: 0}}},
+			metav1.Duration{Duration: 0},
+		),
+		Entry("garbageCollection.ttl is a positive duration",
+			&registry.RegistryCache{GarbageCollection: &registry.GarbageCollection{TTL: metav1.Duration{Duration: 30 * 24 * time.Hour}}},
+			metav1.Duration{Duration: 30 * 24 * time.Hour},
+		),
 	)
 
 	DescribeTable("#FindCacheByUpstream",

pkg/apis/registry/install/install.go

@@ -20,11 +20,13 @@ import (
 
 	"github.com/gardener/gardener-extension-registry-cache/pkg/apis/registry"
 	"github.com/gardener/gardener-extension-registry-cache/pkg/apis/registry/v1alpha2"
+	"github.com/gardener/gardener-extension-registry-cache/pkg/apis/registry/v1alpha3"
 )
 
 var (
 	schemeBuilder = runtime.NewSchemeBuilder(
 		v1alpha2.AddToScheme,
+		v1alpha3.AddToScheme,
 		registry.AddToScheme,
 		setVersionPriority,
 	)
@@ -34,7 +36,7 @@ var (
 )
 
 func setVersionPriority(scheme *runtime.Scheme) error {
-	return scheme.SetVersionPriority(v1alpha2.SchemeGroupVersion)
+	return scheme.SetVersionPriority(v1alpha3.SchemeGroupVersion, v1alpha2.SchemeGroupVersion)
 }
 
 // Install installs all APIs in the scheme.