Starting to move components from prototype repo (julienschmidt#2)

* starting the scaler and interceptor Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Adding interceptor Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * link to issue Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * operator and CLI start Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * starting on the dockerfiles Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Improving the operator Dockerfile Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * fix Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * note Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * adding go based operator Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * a bit of progress Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * moving rust operator to old will delete this shortly Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * moving goperator to operator forgot to do this last commit Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * moving go.mod to root Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * marking deprecated Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * it builds gs Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * progress on the operator Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * more progress Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * more Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * All my changes and implementations (julienschmidt#2) * Fix error with existing variable * Rename ScaledObject to HTTPScaledObject * Add draft implementation for status spec * Rename files to match new naming * Remove scaffolding comments * Change requeue time to 50s * Variable renaming to better read * Add new spec field to make polling configurable * Remove previously added objects in case of error * Set Statuses Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Fix Typos (julienschmidt#4) Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Add RBAC rules for services, pods and deployments (julienschmidt#5) * Fix Typos * Add RBAC rules for services, pods and deployments Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Bunch of changes (julienschmidt#6) * Fix Typos * Implement object deletion * Add prefix for logging * Remove unused status interface * Add status check for "Ready" * Golint * Generate CRD Manifests * Reorder RBAC Roles * Change file namings * Remove CLI part for now * Add internal build context for operator Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Charts, docs, readability and other changes (julienschmidt#8) * Fix Typos * Implement object deletion * Add prefix for logging * Remove unused status interface * Add status check for "Ready" * Golint * Generate CRD Manifests * Reorder RBAC Roles * Change file namings * Remove CLI part for now * Add internal build context for operator * Remove Rust-related things * Add comment docs and reorder packages * Change variable name and add package docs * Add package docs * Add package docs * Rename var for readability * Adds package docs * Remove commented code * Remove unused admin echo server and rename current one * Add more logging and docs * Fix doc * Update readme on cli * Add scaffold to operator chart * Add docs * Golint * Update name in kustomization * Add descriptions to fields * Update default image name * Change resource names * Add version tag to kustomization * Add make command for charts * Create base values file * Add patch for image name * Add environment variable to manager * Update charts/operator/Chart.yaml Co-authored-by: Tom Kerkhove <kerkhove.tom@gmail.com> * Update charts/operator/Chart.yaml Co-authored-by: Tom Kerkhove <kerkhove.tom@gmail.com> * Correct dir name and reuse step * Rename dir to keda-http-operator * Add envs to deployment pkg * Add new statuses for extended objetct * Fixing interceptor listening port * Include new "Terminating" status * Add namespaces for logs * Change variable names * Add new statuses * Code reorder * Create interceptor * Add missing return * Remove comment * Fix link * Remove all references to external scaler address This is because we'll use one external scaler for each new application, so there's no point in having a single environment variable controlling the address of the external scaler. The address will be defined by the operator for each new app and a new scaler will be created. * Fixing external scaler port * Code reorder and creation of external scaler * Improve code readability * Golint * Add colons * Further clean the code * Move functions to other file * Rename files * Add more meaninful names * Rename method * Update roles and enums * Make yamls * Add next TODO * Add todo Co-authored-by: Tom Kerkhove <kerkhove.tom@gmail.com> Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Fixing the operator Dockerfile also adding env vars to allow users to customize docker image names Fixes kedacore/http-add-on#5 Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Updating to the KEDA 2.0 external scaler gRPC interface The notable change here is that there is an added StreamIsActive RPC call. The New and Close RPCs are removed. Fixes kedacore/http-add-on#3 Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Making the scaler talk to interceptors to get pending HTTP queue sizes Fixes kedacore/http-add-on#4 Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Initial install docs Includes various tweaks needed to make install work Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * more Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * more progress Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * proper port and path Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * splitting up helm chart Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * fix chart, example yaml, and fix panic in operator logging Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * fixing log errors Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * minor changes Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * gracefully moving on when deleting kube resources Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * creating dynamic client Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * using proper group for scaled objects Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * adding scaledobject to ClusterRole Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Helm installation in the Makefile Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * more loggign Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * properly specifying namespaces etc... Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * setting operator deployment pull policy to always Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * adding more logs to resource creation Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * fixing namespace problems and simplifying code Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * proper API version Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * aking image names and ports for external scaler and interceptor configurable Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * adding docs Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * proper env var names Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Specifying proper external scaler ports Also moving the application config in the operator into operator/config Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * ports Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * updated ScaledObject spec Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * proper port formatting for interceptor Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * more logging and not requeueing Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * cscaler => kedahttp Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * trying something out Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * adding finalizer, simplifying get code Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * finalizers Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * ignoring more stuff Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * removing the right things Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * updating deletion & error logic Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * logging status updates Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * check for already exists inside specific creator logic Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Fiddling with status Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * upgrading CRD version Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Adding instructions on microk8s installs Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * Restoring the status file Don't know why I deleted it in arschles/http-add-on@021b5f1 Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * myapp => xkcd Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * service type Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * logging Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * configuring services properly Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> * readme Signed-off-by: Aaron Schlesinger <aaron@ecomaz.net> Co-authored-by: Lucas Santos <lhs.santoss@gmail.com> Co-authored-by: Tom Kerkhove <kerkhove.tom@gmail.com>
similarweb · Jan 27, 2021 · 7b75262 · 7b75262
1 parent 71e93d3
commit 7b75262
Show file tree

Hide file tree

Showing 85 changed files with 5,116 additions and 2 deletions.
diff --git a/.dockerignore b/.dockerignore
@@ -0,0 +1,13 @@
+/target
+/bin
+.git
+.gitignore
+/charts
+/.vscode
+/bin
+/cli
+/examples
+/docs
+/.envrc
+CONTRIBUTING.md
+Makefile
diff --git a/.gitignore b/.gitignore
@@ -347,3 +347,11 @@ cover.out
 
 # GO debug binary
 cmd/manager/debug.test
+
+/bin
+
+admin/target
+admin/Cargo.lock
+
+/target
+.envrc
diff --git a/Makefile b/Makefile
@@ -0,0 +1,74 @@
+GIT_TAG?=$(shell git rev-parse --short HEAD)
+SCALER_DOCKER_IMG?=arschles/keda-http-scaler:${GIT_TAG}
+INTERCEPTOR_DOCKER_IMG?=arschles/keda-http-interceptor:${GIT_TAG}
+OPERATOR_DOCKER_IMG?=arschles/keda-http-operator:${GIT_TAG}
+NAMESPACE?=kedahttp
+
+.PHONY: gen-scaler
+gen-scaler:
+	protoc \
+		--go_out=scaler/gen/ \
+		--go_opt=paths=source_relative \
+		--go-grpc_out=scaler/gen/ \
+		--go-grpc_opt=paths=source_relative \
+		scaler/scaler.proto
+
+.PHONY: build-scaler
+build-scaler:
+	go build -o bin/scaler ./scaler
+
+.PHONY: docker-build-scaler
+docker-build-scaler:
+	docker build -t ${SCALER_DOCKER_IMG} -f scaler/Dockerfile .
+
+.PHONY: docker-push-scaler
+docker-push-scaler: docker-build-scaler
+	docker push ${SCALER_DOCKER_IMG}
+
+.PHONY: build-interceptor
+build-interceptor:
+	go build -o bin/interceptor ./interceptor
+
+.PHONY: docker-build-interceptor
+docker-build-interceptor:
+	docker build -t ${INTERCEPTOR_DOCKER_IMG} -f interceptor/Dockerfile .
+
+.PHONY: docker-push-interceptor
+docker-push-interceptor: docker-build-interceptor
+	docker push ${INTERCEPTOR_DOCKER_IMG}
+
+# .PHONY: build-operator
+# build-operator-cli:
+# 	cargo build --bin operator
+
+.PHONY: build-operator
+build-operator:
+	go build -o bin/operator ./operator
+
+.PHONY: docker-build-operator
+docker-build-operator:
+	docker build -t ${OPERATOR_DOCKER_IMG} -f operator/Dockerfile .
+
+.PHONY: docker-push-operator
+docker-push-operator: docker-build-operator
+	docker push ${OPERATOR_DOCKER_IMG}
+
+.PHONY: helm-upgrade-operator
+helm-upgrade-operator:
+	helm upgrade kedahttp ./charts/keda-http-operator \
+    --install \
+    --namespace ${NAMESPACE} \
+    --create-namespace \
+    --set image=${OPERATOR_DOCKER_IMG} \
+	--set externalScalerImage=${SCALER_DOCKER_IMG} \
+	--set interceptorImage=${INTERCEPTOR_DOCKER_IMG}
+
+
+.PHONY: build-all
+build-all: build-scaler build-interceptor build-operator
+
+.PHONY: docker-build-all
+docker-build-all: docker-build-scaler docker-build-interceptor docker-build-operator
+
+.PHONY: docker-push-all
+docker-push-all: docker-push-scaler docker-push-interceptor docker-push-operator
diff --git a/README.md b/README.md
@@ -7,7 +7,14 @@ KEDA allows for fine-grained autoscaling (including to/from zero) for event driv
 
 | 🚧 **Experimental - Not for production** 🚧|
 |---------------------------------------------|
-| ⚠ The HTTP add-on is in experimental stage and not ready for production. <br /><br />It is provided as-is without support and not a guarantee that it will become an official scaler. |
+| ⚠ The HTTP add-on is in [experimental stage](https://github.com/kedacore/keda/issues/538) and not ready for production. <br /><br />It is provided as-is without support.
 
+>In addition to the above, this codebase moves very quickly. We can't currently guarantee that any part of it will work. Neither the complete featureset nor known issues may not be fully documented. Similarly, issues filed against this project may not be responded to. **We will release and announce a beta release of this project**, and after we do that, we will document and respond to issues properly.
+
+## Installation
+
+Please see the [complete installation instructions](./docs/install.md).
+
+---
 We are a Cloud Native Computing Foundation (CNCF) sandbox project.
-<p align="center"><img src="https://raw.githubusercontent.com/kedacore/keda/main/images/logo-cncf.svg" height="75px"></p>
+<p align="center"><img src="https://raw.githubusercontent.com/kedacore/keda/main/images/logo-cncf.svg" height="75px"></p>
diff --git a/charts/keda-http-operator/.helmignore b/charts/keda-http-operator/.helmignore
@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/
diff --git a/charts/keda-http-operator/Chart.yaml b/charts/keda-http-operator/Chart.yaml
@@ -0,0 +1,23 @@
+apiVersion: v2
+name: keda-http-operator
+description: KEDA HTTP add-on operator chart
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.1.0
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+appVersion: 0.1.0
diff --git a/charts/keda-http-operator/crds/httpscaledobjects.http.keda.sh.yaml b/charts/keda-http-operator/crds/httpscaledobjects.http.keda.sh.yaml
@@ -0,0 +1,128 @@
+
+---
+apiVersion: apiextensions.k8s.io/v1
+kind: CustomResourceDefinition
+metadata:
+  annotations:
+    controller-gen.kubebuilder.io/version: v0.4.1
+  creationTimestamp: null
+  name: httpscaledobjects.http.keda.sh
+spec:
+  group: http.keda.sh
+  names:
+    kind: HTTPScaledObject
+    listKind: HTTPScaledObjectList
+    plural: httpscaledobjects
+    singular: httpscaledobject
+  scope: Namespaced
+  versions:
+  - name: v1alpha1
+    schema:
+      openAPIV3Schema:
+        description: HTTPScaledObject is the Schema for the scaledobjects API
+        properties:
+          apiVersion:
+            description: 'APIVersion defines the versioned schema of this representation
+              of an object. Servers should convert recognized schemas to the latest
+              internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
+            type: string
+          kind:
+            description: 'Kind is a string value representing the REST resource this
+              object represents. Servers may infer this from the endpoint the client
+              submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
+            type: string
+          metadata:
+            type: object
+          spec:
+            description: HTTPScaledObjectSpec defines the desired state of HTTPScaledObject
+            properties:
+              app_image:
+                description: The image this application will use.
+                type: string
+              app_name:
+                description: (optional) The name of the application to be created.
+                type: string
+              polling_interval:
+                description: (optional) The interval to check for changes.
+                format: int32
+                type: integer
+              port:
+                description: The port this application will serve on.
+                format: int32
+                type: integer
+            required:
+            - app_image
+            - port
+            type: object
+          status:
+            description: HTTPScaledObjectStatus defines the observed state of HTTPScaledObject
+            properties:
+              deployment_status:
+                description: HTTPScaledObjectCreationStatus describes the creation
+                  status of the scaler's additional resources such as Services, Ingresses
+                  and Deployments
+                enum:
+                - Created
+                - Error
+                - Pending
+                - Unknown
+                - Terminating
+                type: string
+              externalscaler_status:
+                description: HTTPScaledObjectCreationStatus describes the creation
+                  status of the scaler's additional resources such as Services, Ingresses
+                  and Deployments
+                enum:
+                - Created
+                - Error
+                - Pending
+                - Unknown
+                - Terminating
+                type: string
+              interceptor_status:
+                description: HTTPScaledObjectCreationStatus describes the creation
+                  status of the scaler's additional resources such as Services, Ingresses
+                  and Deployments
+                enum:
+                - Created
+                - Error
+                - Pending
+                - Unknown
+                - Terminating
+                type: string
+              ready:
+                type: boolean
+              scaledobject_status:
+                description: HTTPScaledObjectCreationStatus describes the creation
+                  status of the scaler's additional resources such as Services, Ingresses
+                  and Deployments
+                enum:
+                - Created
+                - Error
+                - Pending
+                - Unknown
+                - Terminating
+                type: string
+              service_status:
+                description: HTTPScaledObjectCreationStatus describes the creation
+                  status of the scaler's additional resources such as Services, Ingresses
+                  and Deployments
+                enum:
+                - Created
+                - Error
+                - Pending
+                - Unknown
+                - Terminating
+                type: string
+            type: object
+        type: object
+    served: true
+    storage: true
+    subresources:
+      status: {}
+status:
+  acceptedNames:
+    kind: ""
+    plural: ""
+  conditions: []
+  storedVersions: []
diff --git a/charts/keda-http-operator/templates/deployment.yaml b/charts/keda-http-operator/templates/deployment.yaml
@@ -0,0 +1,50 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  labels:
+    control-plane: controller-manager
+    httpscaledobjects.http.keda.sh/version: {{ .Values.version }}
+    keda.sh/addon: keda-http-addon
+  name: keda-http-addon-controller-manager
+  namespace: {{ .Release.Namespace }}
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      control-plane: controller-manager
+      httpscaledobjects.http.keda.sh/version: {{ .Values.version }}
+      keda.sh/addon: keda-http-addon
+  template:
+    metadata:
+      labels:
+        control-plane: controller-manager
+        httpscaledobjects.http.keda.sh/version: {{ .Values.version }}
+        keda.sh/addon: keda-http-addon
+    spec:
+      containers:
+      - args:
+        - --secure-listen-address=0.0.0.0:8443
+        - --upstream=http://127.0.0.1:8080/
+        - --logtostderr=true
+        - --v=10
+        image: gcr.io/kubebuilder/kube-rbac-proxy:v0.5.0
+        name: kube-rbac-proxy
+      - args:
+        - --metrics-addr=127.0.0.1:8080
+        - --enable-leader-election
+        image: {{ .Values.image }}
+        imagePullPolicy: Always
+        name: manager
+        env:
+        - name: KEDAHTTP_OPERATOR_EXTERNAL_SCALER_IMAGE
+          value: {{ .Values.externalScalerImage }}
+        - name: KEDAHTTP_OPERATOR_INTERCEPTOR_IMAGE
+          value: {{ .Values.interceptorImage }}
+        # resources:
+        #   limits:
+        #     cpu: 0.5
+        #     memory: 30Mi
+        #   requests:
+        #     cpu: 0.5
+        #     memory: 20Mi
+      terminationGracePeriodSeconds: 10