design 10574: transformations

design/10574.md

@@ -0,0 +1,160 @@
+<!--
+**Note:** When your Enhancement Proposal (EP) is complete, all of these comment blocks should be removed.
+
+This template is inspired by the Kubernetes Enhancement Proposal (KEP) template: https://github.com/kubernetes/enhancements/blob/master/keps/sig-architecture/0000-kep-process/README.md
+
+To get started with this template:
+
+- [ ] **Create an issue in kgateway-dev/kgateway**
+- [ ] **Make a copy of this template.**
+  `EP-[ID]: [Feature/Enhancement Name]`, where `ID` is the issue number (with no
+  leading-zero padding) assigned to your enhancement above.
+- [ ] **Fill out this file as best you can.**
+  At minimum, you should fill in the "Summary" and "Motivation" sections.
+- [ ] **Create a PR for this EP.**
+  Assign it to maintainers with relevant context.
+- [ ] **Merge early and iterate.**
+  Avoid getting hung up on specific details and instead aim to get the goals of
+  the EP clarified and merged quickly. The best way to do this is to just
+  start with the high-level sections and fill out details incrementally in
+  subsequent PRs.
+
+Just because a EP is merged does not mean it is complete or approved. Any EP
+marked as `provisional` is a working document and subject to change. You can
+denote sections that are under active debate as follows:
+
+```
+<<[UNRESOLVED optional short context or usernames ]>>
+Stuff that is being argued.
+<<[/UNRESOLVED]>>
+```
+
+When editing EPS, aim for tightly-scoped, single-topic PRs to keep discussions
+focused. If you disagree with what is already in a document, open a new PR
+with suggested changes.
+
+One EP corresponds to one "feature" or "enhancement" for its whole lifecycle. Once a feature has become
+"implemented", major changes should get new EPs.
+-->
+# EP-10574: Basic Transformations for Request and Response 
+
+<!--
+This is the title of your EP. Keep it short, simple, and descriptive. A good
+title can help communicate what the EP is and should be considered as part of
+any review.
+-->
+
+* Issue: [https://github.com/kgateway-dev/kgateway/issues/10574](URL to GitHub issue)
+
+<!--
+A table of contents is helpful for quickly jumping to sections of a EP and for
+highlighting any additional information provided beyond the standard EP
+template.
+
+Ensure the TOC is wrapped with
+  <code>&lt;!-- toc --&rt;&lt;!-- /toc --&rt;</code>
+tags, and then generate with `hack/update-toc.sh`.
+-->
+
+## Background 
+The previous project had a pretty widely used staged transformation concept which could be slotted into both request and response paths. This Transformation filter most critically was used to perform simple modification with inja templates and to extract data for usage in access logging. Because some applications of transformation can be rather heavy (rewrite whole responses perform regex lookups) this was built as a custom filter and packaged with a seperate repository called envoy-gloo to make sure that it is secure and performant. 
+
+
+
+## Motivation
+Being able to peform conditional header mutation and enrich access logs via dynamic metadata in a performant way is a great quality of life feature that the previous project had and we should support.
+
+
+### Goals
+* Allow for extraction of data from a header and place it in dynamic metadata or filter state such that it can be pulled into access logs
+* Allow for mutations that are based on the contents of the request / response 
+* Have transformations support be stabley defined and not require a network hop
+* Have an opinionated api to keep functionality be locked to transforming envoy constructs such as filter state, headers, body, trailers and metadata
+
+
+### Non-Goals 
+* Introduce a new repository to support this feature
+
+## Implementation Details
+As part of this we should look at leveraging the quickly stabilizing [Dynamic Module support](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/dynamic_modules) to reduce overhead in the repository and to let us have a potential move to using raw envoy releases as the default standard for kgateway builds. An example of how we might start to do this can be found in [poc](https://github.com/kgateway-dev/kgateway/pull/10677). This would allow for side by side of old and new functionality to make sure that as users migrate to kgateway 2 we can in the near term flip an os flag to find any gaps between the new transformation approach and the older approach that is based on the envoy-gloo repo.
+Ideally we can then move to deprecate and remove the usage of a seperate enovy repository and solidify the entirety of kgateway inside of the kgateway repository.
+At first we should start with mutating headers by adding or setting them as well as the ability to set information in one of dynamic metadata or filter state so that we can enrich access logs.
+
+Given the above we should start with a minimal API where we expose Transformations as something like the following
+
+```
+type TransformationPolicy struct {
+	// +optional
+	Request *Transform `json:"request,omitempty"`
+	// +optional
+	Response *Transform `json:"response,omitempty"`
+}
+
+type Transform struct {
+
+	// +optional
+	// +listType=map
+	// +listMapKey=name
+	// +kubebuilder:validation:MaxItems=16
+	Set []HeaderTransformation `json:"set,omitempty"`
+
+	// +optional
+	// +listType=map
+	// +listMapKey=name
+	// +kubebuilder:validation:MaxItems=16
+	Add []HeaderTransformation `json:"add,omitempty"`
+
+
+	// +optional
+	// +listType=map
+	// +listMapKey=name
+	// +kubebuilder:validation:MaxItems=16
+	SetMetdata []MetadataTransformation `json:"set,omitempty"`
+
+}
+
+type HeaderTransformation struct {
+
+	// +required
+	Name  gwv1.HeaderName `json:"name,omitempty"`
+	Value InjaTemplate    `json:"value,omitempty"`
+}
+
+type MetadataTransformation struct {
+
+  // +required
+  Namespace string      `json:"name,omitempty"`
+	// +required
+	Name  string          `json:"name,omitempty"`
+	Value InjaTemplate    `json:"value,omitempty"`
+}
+
+```
+
+We then can start building common tooling to enhance inja to support repeated use cases.
+For example the previous project supported the following inja enhancements:
+ "substring" , "trim" , "base64_encode" , "base64url_encode" , "base64_decode" , "base64url_decode" , "replace_with_random" , "raw_string" , "header" , "request_header" , "extraction" , "body" , "dynamic_metadata" , "data_source" , "host_metadata" , "cluster_metadata" , "context" , "env"
+
+In order to properly implement enriching access logs with information based on headers we would minimally need the "header" extension.
+
+Given this the first addition here should contain the following functions:
+"base64_encode", "header", "replace_with_random"
+
+### Plugin
+Given that transformations are tightly coupled with route based decisions this new transformation policy should be part of the existing route policy and be colocated in the code base with the existing routepolicy.
+
+
+### Test Plan 
+Testing should include unit tests per Inja extension, end-to-end tests for basic inja functionality and a integration test with enriched access logs to prove out the two user stories.
+
+
+## Alternatives
+Port the existing Transformation from the old project and figure out how to migrate the custom envoy to github actions.
+
+
+
+## Open Questions
+What is the timeline to removal of the current extended envoy image.
+How do we want to handle route level overrides. 
+
+