Create General Purpose Config Processor for Discovered Upstreams (sol…

…o-io#5452) * Create General Service Converter * Update service converter testing * create settings.DiscoveryOptions.GeneralAnnotationUpstreamConfig setting * Use widely supported method of iterating over struct fields * Don't overwrite preexisting config in new converter * Enable general purpose annotation config processor by default * Remove no-longer-used general_annotation_upstream_config setting * Update tests, add new test to confirm that general converter doesn't override Http2 converter * Merge branch 'master' into general-annotation-config-processor * Add changelog entry * Adding changelog file to new location * Deleting changelog file from old location * Ensure that general annotation converter is applied first * Restore check against values set on upstream spec * Merge branch 'general-annotation-config-processor' of github.com:solo-io/gloo into general-annotation-config-processor * trigger build * Revert to init() implementation of serviceConverter intialization * Refactor GeneralServiceConverter.ConvertService for readability, clarifying comments * Create translator.MergeUpstreams util * Testing, clarifying comments around new translator utility * Generalize struct merge logic * Remove unnecessary assertion in mergeStructs * Remove init() approach to creating default service converters, create DefaultServiceConverters method * Move mergeUpstreams util out of gateway, into general annotation converter * Allow general purpose converter to overwrite config from other converters * Define init function in service_converter_interface.go * Swap position of src and dst arguments in mergeUpstreams * Handle output of mergeUpstreams * Don't return upstream from mergeUpstreams * Dont return anything from mergeUpstream * trigger build * Remove isEmptyValue from general_annotation_converter.go * Add user guide for general annotation config * resolve outstanding issues with general annotation converter * ci(deploy-to-kind-cluster): added default env vars (solo-io#5473) * ci(deploy-to-kind-cluster): added default env vars \# Description * classify between `darwin` vs. `linux` systems when running `ci/deploy-to-kind-cluster.sh` * default `KUBE2E_TESTS` to `eds`, rather than nothing This commit can be used to run `ci/deploy-to-kind-cluster.sh` on a mac off of a fresh clone with no configuration \# Context One of the more common use-cases for `deploy-to-kind-cluster` is in local development. _Currently_, when running the script, some tweaks are needed to have it _succeed_ on a Mac. This is a bit painful, and is time that not every person on a Mac should need to figure out in order to test locally. * Update docs/content/guides/traffic_management/destination_types/discovered_upstream/discovered-upstream-configuration.md Co-authored-by: Art <artberger@users.noreply.github.com> Update docs/content/guides/traffic_management/destination_types/discovered_upstream/discovered-upstream-configuration.md Co-authored-by: Art <artberger@users.noreply.github.com> Update docs/content/guides/traffic_management/destination_types/discovered_upstream/discovered-upstream-configuration.md Co-authored-by: Art <artberger@users.noreply.github.com> Update docs/content/guides/traffic_management/destination_types/discovered_upstream/discovered-upstream-configuration.md Co-authored-by: Art <artberger@users.noreply.github.com> Update docs/content/guides/traffic_management/destination_types/discovered_upstream/discovered-upstream-configuration.md Co-authored-by: Art <artberger@users.noreply.github.com> Update docs/content/guides/traffic_management/destination_types/discovered_upstream/discovered-upstream-configuration.md Co-authored-by: Art <artberger@users.noreply.github.com> * Merge branch 'master' into general-annotation-config-processor * Resolve failing tests * Merge branch 'general-annotation-config-processor' of github.com:solo-io/gloo into general-annotation-config-processor * Return Spec to function-level scope, clarify fields that cannot be written to * Update general config serviceconverter to no longer require top-level spec field in incoming config * Update docs
gunnar-solo · Oct 15, 2021 · 5b1339b · 5b1339b
1 parent 1892429
commit 5b1339b
Show file tree

Hide file tree

Showing 8 changed files with 458 additions and 92 deletions.
diff --git a/changelog/v1.10.0-beta2/general-annotation-config-processor.yaml b/changelog/v1.10.0-beta2/general-annotation-config-processor.yaml
@@ -0,0 +1,5 @@
+changelog:
+  - type: NEW_FEATURE
+    issueLink: https://github.com/solo-io/gloo/issues/3717
+    resolvesIssue: true
+    description: Expand annotation-based configuration for discovered upstreams to respect all Upstream config options.
diff --git a/...ment/destination_types/discovered_upstream/discovered-upstream-configuration.md b/...ment/destination_types/discovered_upstream/discovered-upstream-configuration.md
@@ -0,0 +1,101 @@
+---
+title: Discovered Upstream Configuration via Annotations
+weight: 101
+---
+
+Gloo Edge looks for discovered upstream configuration in the annotations of any Kubernetes service that it identifies. For Gloo Edge to discover the upstream configuration, include an annotation in the service in a key-value format. The key is `gloo.solo.io/upstream_config` and the value is the upstream configuration, formatted as JSON.
+
+For example, we can set the initial stream window size on the discovered upstream using the a modified version of the pet store manifest provided in the parent document:
+
+{{< highlight yaml "hl_lines=7" >}}
+kubectl apply -f - <<EOF
+# petstore service
+apiVersion: v1
+kind: Service
+metadata:
+  annotations:
+    gloo.solo.io/upstream_config: '{"initial_stream_window_size": 2048}'
+  name: petstore
+  namespace: default
+  labels:
+    service: petstore
+spec:
+  ports:
+  - port: 8080
+    protocol: TCP
+  selector:
+    app: petstore
+---
+#petstore deployment 
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+    app: petstore
+  name: petstore
+  namespace: default
+spec:
+  selector:
+    matchLabels:
+      app: petstore
+  replicas: 1
+  template:
+    metadata:
+      labels:
+        app: petstore
+    spec:
+      containers:
+      - image: soloio/petstore-example:latest
+        name: petstore
+        ports:
+        - containerPort: 8080
+          name: http
+EOF
+{{< /highlight >}}
+
+Now that you created the pet store app, check for the discovered upstream. In the output of the following command, note the upstream with the namespace, name, and port of the service , `default-petstore-8080`. 
+
+    kubectl get upstreams -n gloo-system
+
+Review the upstream to make sure that the configuration from the Kubernetes service was picked up in the upstream.
+
+
+```shell
+kubectl get upstream -n gloo-system default-petstore-8080 -oyaml
+```
+
+{{< highlight yaml "hl_lines=5, 25" >}}
+apiVersion: gloo.solo.io/v1
+kind: Upstream
+metadata:
+  annotations:
+    gloo.solo.io/upstream_config: '{"initial_stream_window_size": 2048}'
+    kubectl.kubernetes.io/last-applied-configuration: |
+      {"apiVersion":"v1","kind":"Service","metadata":{"annotations":{"gloo.solo.io/upstream_config":" {\"initial_stream_window_size\": 2048}"},"labels":{"service":"petstore"},"name":"petstore","namespace":"default"},"spec":{"ports":[{"port":8080,"protocol":"TCP"}],"selector":{"app":"petstore"}}}
+  creationTimestamp: "2021-10-14T13:22:12Z"
+  generation: 2
+  labels:
+    discovered_by: kubernetesplugin
+  name: default-petstore-8080
+  namespace: default
+  resourceVersion: "5679"
+  uid: 0ab14ba5-6377-40c5-a781-ce33b7755cdc
+spec:
+  discoveryMetadata:
+    labels:
+      service: petstore
+  kube:
+    selector:
+      app: petstore
+    serviceName: petstore
+    serviceNamespace: default
+    servicePort: 8080
+  initialStreamWindowSize: 2048
+status:
+  statuses:
+    default:
+      reportedBy: gloo
+      state: 1
+{{< /highlight >}}
+
+As you can see, the configuration set `spec.initialStreamWindowSize` to `2048` on the discovered upstream! 
diff --git a/projects/gloo/pkg/plugins/kubernetes/serviceconverter/general_annotation_converter.go b/projects/gloo/pkg/plugins/kubernetes/serviceconverter/general_annotation_converter.go
@@ -0,0 +1,62 @@
+package serviceconverter
+
+import (
+	"encoding/json"
+	"reflect"
+
+	v1 "github.com/solo-io/gloo/projects/gloo/pkg/api/v1"
+	"github.com/solo-io/solo-kit/pkg/utils/protoutils"
+	"google.golang.org/protobuf/proto"
+	kubev1 "k8s.io/api/core/v1"
+)
+
+const GlooAnnotationPrefix = "gloo.solo.io/upstream_config"
+
+type GeneralServiceConverter struct{}
+
+func (s *GeneralServiceConverter) ConvertService(svc *kubev1.Service, port kubev1.ServicePort, us *v1.Upstream) error {
+	upstreamConfigJson, ok := svc.Annotations[GlooAnnotationPrefix]
+	if !ok {
+		return nil
+	}
+
+	var upstreamConfigMap map[string]interface{}
+	if err := json.Unmarshal([]byte(upstreamConfigJson), &upstreamConfigMap); err != nil {
+		return err
+	}
+
+	upstreamConfig := v1.Upstream{}
+	if err := protoutils.UnmarshalMap(upstreamConfigMap, &upstreamConfig); err != nil {
+		return err
+	}
+
+	mergeUpstreams(&upstreamConfig, us)
+	return nil
+}
+
+// Merges the fields of src into dst.
+func mergeUpstreams(src, dst *v1.Upstream) {
+	if src == nil {
+		return
+	}
+
+	if dst == nil {
+		dst = proto.Clone(src).(*v1.Upstream)
+		return
+	}
+
+	dstValue, srcValue := reflect.ValueOf(dst).Elem(), reflect.ValueOf(src).Elem()
+
+	for i := 0; i < dstValue.NumField(); i++ {
+		dstField, srcField := dstValue.Field(i), srcValue.Field(i)
+
+		if srcField.IsValid() && dstField.CanSet() && !srcField.IsZero() {
+			fieldName := reflect.Indirect(reflect.ValueOf(dst)).Type().Field(i).Name
+			// Information critical to proper UDS operation is contained in these fields,
+			// so do not allow this serviceconverter to overwrite them.
+			if fieldName != "Metadata" && fieldName != "DiscoveryMetadata" && fieldName != "NamespacedStatuses" {
+				dstField.Set(srcField)
+			}
+		}
+	}
+}
diff --git a/projects/gloo/pkg/plugins/kubernetes/serviceconverter/service_converter_interface.go b/projects/gloo/pkg/plugins/kubernetes/serviceconverter/service_converter_interface.go
@@ -5,6 +5,15 @@ import (
 	kubev1 "k8s.io/api/core/v1"
 )
 
+func init() {
+	DefaultServiceConverters = []ServiceConverter{
+		&UseHttp2Converter{},
+		&UseSslConverter{},
+		// The General Service Converter is applied last, and is capable of overriding settings applied by prior converters
+		&GeneralServiceConverter{},
+	}
+}
+
 // ServiceConverters apply extra changes to an upstream spec before the upstream is created
 // use this to support things like custom config from annotations
 type ServiceConverter interface {

diff --git a/projects/gloo/pkg/plugins/kubernetes/serviceconverter/use_http2_annotation_converter.go b/projects/gloo/pkg/plugins/kubernetes/serviceconverter/use_http2_annotation_converter.go
@@ -8,10 +8,6 @@ import (
 	kubev1 "k8s.io/api/core/v1"
 )
 
-func init() {
-	DefaultServiceConverters = append(DefaultServiceConverters, &UseHttp2Converter{})
-}
-
 const GlooH2Annotation = "gloo.solo.io/h2_service"
 
 var http2PortNames = []string{

diff --git a/projects/gloo/pkg/plugins/kubernetes/serviceconverter/use_ssl_annotation_converter.go b/projects/gloo/pkg/plugins/kubernetes/serviceconverter/use_ssl_annotation_converter.go
@@ -9,10 +9,6 @@ import (
 	kubev1 "k8s.io/api/core/v1"
 )
 
-func init() {
-	DefaultServiceConverters = append(DefaultServiceConverters, &UseSslConverter{})
-}
-
 /*
 The values for these annotations can be provided in one of two ways: