sig-api-machinery: add "Defaulting for CustomResources"

keps/sig-api-machinery/20190426-crd-defaulting.md

@@ -0,0 +1,336 @@
+---
+title: Defaulting for Custom Resources
+authors:
+  - "@sttts"
+owning-sig: sig-api-machinery
+participating-sigs:
+  - sig-api-machinery
+reviewers:
+  - "@deads2k"
+  - "@lavalamp"
+  - "@liggitt"
+  - "@mbohlool"
+  - "@apelisse"
+approvers:
+  - "@deads2k"
+  - "@lavalamp"
+editor:
+  name: "@sttts"
+creation-date: 2019-04-26
+last-updated: 2019-05-01
+status: implementable
+see-also:
+ - "/keps/sig-api-machinery/20180731-crd-pruning.md"
+ - "/keps/sig-api-machinery/20190425-structural-openapi.md"
+---
+
+# Defaulting for Custom Resources
+
+## Table of Contents
+
+   * [Defaulting for Custom Resources](#defaulting-for-custom-resources)
+      * [Table of Contents](#table-of-contents)
+      * [Summary](#summary)
+      * [Motivation](#motivation)
+         * [Goals](#goals)
+         * [Non-Goals](#non-goals)
+      * [Proposal](#proposal)
+         * [Examples](#examples)
+      * [References](#references)
+         * [Test Plan](#test-plan)
+         * [Graduation Criteria](#graduation-criteria)
+         * [Upgrade / Downgrade Strategy](#upgrade--downgrade-strategy)
+         * [Version Skew Strategy](#version-skew-strategy)
+      * [Implementation History](#implementation-history)
+
+## Summary
+
+Defaulting is a fundamental step in the processing of API objects in the request pipeline of the kube-apiserver. Defaulting happens during deserialization, i.e. after decoding of a versioned object, **but before** conversion to a hub type.
+
+Defaulting is implemented for most native Kubernetes API types and plays a crucial role for API compatibility when adding new fields. CustomResources do not support this natively. 
+
+Mutating admission webhooks can be used to partially replicate defaulting for incoming request payloads. But mutating admission webhooks do not run when reading from etcd.
+
+This KEP is about adding support for specifying default values for fields via OpenAPI v3 validation schemas in the CRD manifest. OpenAPI v3 has native support for a `default` field with arbitrary JSON values, for example: 
+
+```yaml
+properties:
+  foo:
+    type: string
+    default: "abc" 
+```
+
+This KEP proposes to apply these default values during deserialization, in the same way as native resources do. We assume _structural schemas_ as defined in [KEP Vanilla OpenAPI Subset: Structural Schema](/keps/sig-api-machinery/20190425-structural-openapi.md).
+
+This feature starts behind a feature gate `CustomResourceDefaulting`, disabled by default as alpha.
+
+## Motivation
+
+* By far most native Golang based resources implement defaulting. CRDs do not allow that, leading to unnatural, not Kubernetes-like APIs. This is bad for the ecosystem.
+* Mutating Admission Webhooks can be used for defaulting, but:
+  * they are not adequate as it is not possible to set default values of newly added fields on GET because admission is not run in the storage layer when reading from etcd.
+  * webhooks are complex, both from the development/maintenance point of view and due to their non-trivial operational overhead. This makes them a no-go for many "not so ambitiously, professionally developed CRDs", e.g. in in-house enterprise environments.
+* _Structural schemas_ as defined in [KEP Vanilla OpenAPI Subset: Structural Schema](/keps/sig-api-machinery/20190425-structural-openapi.md) make defaulting a low hanging fruit.
+
+### Goals
+
+* add CustomResource defaulting at the correct position in the request pipeline 
+* allow definition of defaults via the OpenAPI v3 `default` field.
+
+### Non-Goals
+
+* allow non-constant defaults: native Golang code can of course set defaults which depends on other fields of a JSON object. This is out of scope here and would need some kind of defaulting webhook.
+* native-type declarative defaulting: this KEP is about CRDs. Though it might be desirable to support the same `// +default=<some-json-value>` tag and a mechanism in `k8s.io/apiserver` to evaluate defaults in native, generic registries, this is out-of-scope of the KEP though.
+
+## Proposal
+
+We assume the CRD has _structural schemas_ (as defined in [KEP Vanilla OpenAPI Subset: Structural Schema](/keps/sig-api-machinery/20190425-structural-openapi.md)).
+
+We propose to
+
+1. derive the value-validation-less variant of the structural schema (trivial by definition of _structural schema_) and 
+2. recursively follow the given CustomResource instance and the structural schema, applying defaults where an object field is 
+   * undefined (`_, ok := obj[field]; !ok`)
+   * `nil` if the field not nullable
+   * empty in case of lists and maps, and if nullable is not set.
+
+   This means in detail for primitive types and no types (i.e. IntOrString or arbitrary JSON) in the structural schema:
+
+   * `if v, ok := obj[fld]; !ok` => default
+   * `else if !nullable && v == nil` => default
+
+   and for `array` type in the schema one of these:
+
+   * `if v, ok := obj[fld]; !ok` => default
+   * `else if !nullable && v == nil` => default
+   * `else if array, ok := v.([]interface{}); !ok` => return deserialization error
+   * `else if len(array) == 0` => default
+
+   and for `object` type in the schema:
+
+   * `if v, ok := obj[fld]; !ok` => default
+   * `else if !nullable && v == nil` => default
+   * `else if object, ok := v.(map[string]interface{}); !ok` => return deserialization error
+   * `else if len(object) == 0` => default.
+
+Hence, we never default zero numbers, integers or empty string if they are specified explicitly in the input.
+
+Note: the behaviour for empty lists and maps is that of native resources in by far the most cases. If we need to preserve empty lists and maps, we can add a vendor extensions `x-kubernetes-default-preserve-empty: true` without breaking the existing behaviour.
+
+We do this in the serializer by passing a real defaulter to [`versioningserializer.NewCodec`](https://github.com/kubernetes/apimachinery/blob/master/pkg/runtime/serializer/versioning/versioning.go#L49) such that defaulting is done natively just after the binary payload has been unmarshalled into an `map[string]interface{}` and pruning of [KEP: Pruning for CustomResources](/keps/sig-api-machinery/20180731-crd-pruning.md) was done.
+
+Like for native resources, we do defaulting
+
+* during request payload deserialization
+* after mutating webhook admission
+* during read from storage.
+
+Note: like for native resources, we do not default after webhook conversions. Hence, webhook conversions should be complete in the sense that they return defaulted objects in order for the API user to see defaulted objects. Technically we could do additional defaulting, but to match native resources, we do not.
+
+Compare the yellow boxes in the following figure:
+
+![Decoding steps which must apply defaults](20190426-crd-defaulting-pipeline.png)
+
+We rely on the validation steps in the request pipeline to verify that the default value validates value validation. We will check the types in default values using the _structural schema_ during CRD creation and update though. We will also reject defaults which contain values which will be pruned.
+
+The `default` field in the CRD types is considered alpha quality. We will add a `CustomResourceDefaulting` feature gate. Values for `default` will be rejected if the gate is not enabled and there have not been `default` values set before (ratcheting validation). 
+
+[Kubebuilder's crd-gen](https://github.com/kubernetes-sigs/controller-tools/tree/master/cmd/crd) can make use of this feature by adding another tag, e.g. `// +default=<arbitrary-json-value>`. Defaults are arbitrary JSON values, which must also validate (types are checked during CRD creation and update, value validation is checked for requests, but not for etcd reads) and are not subject to pruning (defaulting happens after pruning).
+
+### Examples
+
+1. default in the undefined case
+
+   ```yaml
+   type: object
+   properties:
+     foo:
+       type: string
+       default: "abc"
+   ```
+
+   Then
+
+   ```json
+   {}
+   ```
+
+   is defaulted to:
+
+   ```json
+   {
+     "foo": "abc"
+   }
+   ```
+
+2. no defaulting
+
+   ```yaml
+   type: object
+   properties:
+     foo:
+       type: string
+       default: "abc"
+   ```
+
+   Then
+
+   ```json
+   {
+     "foo": "def"
+   }
+   ```
+
+   is defaulted to:
+
+   ```json
+   {
+     "foo": "def"
+   }
+   ```
+
+3. default in the undefined case
+
+   ```yaml
+   type: object
+   properties:
+     foo:
+       type: array
+       items:
+         type: integer
+       default: [1]
+   ```
+
+   Then
+
+   ```json
+   {}
+   ```
+
+   is defaulted to:
+
+   ```json
+   {
+     "foo": [1]
+   }
+   ```
+
+   Because `nullable` is not set, also
+
+   ```json
+   {
+     "foo": null  
+   }
+   ```
+
+   is defaulted to:
+
+   ```json
+   {
+      "foo": [1]
+   }
+   ```
+
+   In contrast, empty list stays empty list:
+
+    ```json
+    {
+      "foo": [] 
+    }
+    ```
+
+    is defaulted to:
+
+    ```json
+    {
+       "foo": []
+    }
+    ```
+
+4. nullable
+
+   ```yaml
+   type: object
+   properties:
+     foo:
+       type: array
+       items:
+         type: integer
+       nullable: true
+       default: [1]
+   ```
+
+   Then
+
+   ```json
+   {}
+   ```
+
+   is defaulted to:
+
+   ```json
+   {
+     "foo": [1]
+   }
+   ```
+
+   Because `nullable` is `true`
+
+   ```json
+   {
+     "foo": null  
+   }
+   ```
+
+   is defaulted to:
+
+   ```json
+   {
+      "foo": null
+   }
+   ```
+
+## References
+
+* Old pruning implementation PR https://github.com/kubernetes/kubernetes/pull/64558, to be adapted. With structural schemas it will become much simpler.
+* [OpenAPI v3 specification](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md)
+* [JSON Schema](http://json-schema.org/)
+
+### Test Plan
+
+**blockers for alpha:**
+
+* we add unit tests for the general defaulting algorithm
+* we add apiextensions-apiserver integration tests to
+  * verify that CRDs with default, but without structural schemas are rejected by validation.
+  * verify that CRDs without defaults keep working (probably nothing new needed)
+  * verify that CRDs with defaults are defaulted if the feature gate is enabled:
+    * during request payload deserialization
+    * during mutating webhook admission
+    * during read from storage
+  * verify with feature gate closed that CRDs with defaults are rejected on create and on updating for the first default value, but accepted if defaults existed before (ratcheting validation).
+
+**blockers for beta:**
+
+* no new tests
+* we are happy with the API and its behaviour
+
+**blockers for GA:**
+
+* we verified that performance of defaulting is adequate and not considerably reducing throughput. As a rule of thumb, we expect defaulting to be considerably faster than a deep copy.
+
+### Graduation Criteria
+
+* the test plan is fully implemented for the respective quality level
+
+### Upgrade / Downgrade Strategy
+
+* defaults cannot be set in 1.14 CRDs.
+* CRDs created in 1.15 will keep defaults when downgrading to 1.14 (because we have them in our `v1beta1` types already). They won't be effective and the CRD will not validate anymore. This is acceptable for an alpha feature.
+* CRDs created in 1.15 with the feature gate enabled will keep working the same way when upgrading to 1.16, or conversely during downgrade from 1.16 to 1.15 as we do ratcheting validation.
+
+### Version Skew Strategy
+
+* kubectl is not aware of defaulting in any relevant way. 
+
+## Implementation History