Added Integration Tests, Resolved Bugs With Annotations (#255)

* First commit - made manifests for test deployments, made manifests for i tests for other deployment strategies * broke down blue/green * added latest tags to test manifests for new tags * remade tester * ready to test bgi * using all but first index of argv * careless error with dicts * added test to namespace * realized i was silencing error * indexing containers * keyerror * logging bc python errors are weird * expected still string * parsed args behaving weirdly * test seems to be working now, applying changes to other YAMLs now * blue/green ready to test * oops * oops * Added additional labels to check * hyphen * Added our annotations * lol * added our labels to services too * nonetype issue' * nonetype issue' * narrowing down parameter * fixed annotations issue with promote * adding debhug statement to figure out why services aren't getting annotations * this should fix annotations issue for service * not sure why this wasn't caught by intellisense * should be fixed with removing comma but adding logs in case * added linkerd install * verification * upgraded kubernetes version * removing crds * proxy option * Added smi extension * logging service * smi svcs also getting labeled now * matching ts type * not sure where stable service is going * remaining svc and deployment should match * keeping stable service and ts object * updated tests to reflect keeping ts object * no green svc after promote * duh * lol * canary work * canary test ready * logging for ing, filename for canary * changed ingress svc key and returning svc files from smi canary deployment * ts name * forgot about baseline in first deploy * * * * * smi canary should annotate, fixed cleanup * typescript issue plus percentage * forgot to type extra method * removed cleaned up objects from annotate list * logging because services aren't getting removed * moving to try/catch strategy of annotation since deletion can fail silently/with warnings * moved label to individual * removing canary service check after promote * pod ready for testing * set weights to 1000 * selectors * * * percentage * * * typing * mixed up pod and smi * fixed tests * prettier * forgot to remove canary * cleanup * Added oliver's feedback + more cleanup * ncc as dev dependency * npx * going back to global ncc install bc npm is being weird * prettier * removed unnecessary post step
Azure · Nov 1, 2022 · e9693a7 · e9693a7
1 parent a6cfc31
commit e9693a7
Show file tree

Hide file tree

Showing 24 changed files with 1,672 additions and 317 deletions.
diff --git a/.github/workflows/run-integration-tests-basic.yml b/.github/workflows/run-integration-tests-basic.yml
@@ -0,0 +1,70 @@
+name: Minikube Integration Tests - basic
+on:
+   pull_request:
+      branches:
+         - main
+         - 'releases/*'
+   push:
+      branches:
+         - main
+         - 'releases/*'
+   workflow_dispatch:
+
+jobs:
+   run-integration-test:
+      name: Run Minikube Integration Tests
+      runs-on: ubuntu-latest
+      env:
+         KUBECONFIG: /home/runner/.kube/config
+         NAMESPACE: test-${{ github.run_id }}
+      steps:
+         - uses: actions/checkout@v3
+
+         - name: Install dependencies
+           run: |
+              rm -rf node_modules/
+              npm install
+         - name: Install ncc
+           run: npm i -g @vercel/ncc
+         - name: Build
+           run: ncc build src/run.ts -o lib
+
+         - uses: Azure/setup-kubectl@v3
+           name: Install Kubectl
+
+         - id: setup-minikube
+           name: Setup Minikube
+           uses: manusa/actions-setup-minikube@v2.4.2
+           with:
+              minikube version: 'v1.24.0'
+              kubernetes version: 'v1.21.0'
+              driver: 'none'
+           timeout-minutes: 3
+
+         - name: Create namespace to run tests
+           run: kubectl create ns ${{ env.NAMESPACE }}
+
+         - uses: actions/setup-python@v2
+           name: Install Python
+           with:
+              python-version: '3.x'
+
+         - name: Cleaning any previously created items
+           run: |
+              python test/integration/k8s-deploy-delete.py 'Service' 'all' ${{ env.NAMESPACE }}
+              python test/integration/k8s-deploy-delete.py 'Deployment' 'all' ${{ env.NAMESPACE }}
+              python test/integration/k8s-deploy-delete.py 'Ingress' 'all' ${{ env.NAMESPACE }}
+
+         - name: Executing deploy action for pod
+           uses: ./
+           with:
+              namespace: ${{ env.NAMESPACE }}
+              images: nginx:1.14.2
+              manifests: |
+                 test/integration/manifests/test.yml
+              action: deploy
+
+         - name: Checking if deployments and services were created with canary labels and original tag
+           run: |
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Deployment name=nginx-deployment containerName=nginx:1.14.2 labels=app:nginx,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_basic selectorLabels=app:nginx
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Service name=nginx-service labels=workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_basic selectorLabels=app:nginx
diff --git a/.github/workflows/run-integration-tests-bluegreen-ingress.yml b/.github/workflows/run-integration-tests-bluegreen-ingress.yml
@@ -0,0 +1,178 @@
+name: Minikube Integration Tests - blue-green ingress
+on:
+   pull_request:
+      branches:
+         - main
+         - 'releases/*'
+   push:
+      branches:
+         - main
+         - 'releases/*'
+   workflow_dispatch:
+
+jobs:
+   run-integration-test:
+      name: Run Minikube Integration Tests
+      runs-on: ubuntu-latest
+      env:
+         KUBECONFIG: /home/runner/.kube/config
+         NAMESPACE: test-${{ github.run_id }}
+      steps:
+         - uses: actions/checkout@v3
+
+         - name: Install dependencies
+           run: |
+              rm -rf node_modules/
+              npm install
+         - name: Install ncc
+           run: npm i -g @vercel/ncc
+         - name: Build
+           run: ncc build src/run.ts -o lib
+
+         - uses: Azure/setup-kubectl@v3
+           name: Install Kubectl
+
+         - id: setup-minikube
+           name: Setup Minikube
+           uses: manusa/actions-setup-minikube@v2.4.2
+           with:
+              minikube version: 'v1.24.0'
+              kubernetes version: 'v1.21.0'
+              driver: 'none'
+           timeout-minutes: 3
+
+         - name: Create namespace to run tests
+           run: kubectl create ns ${{ env.NAMESPACE }}
+
+         - uses: actions/setup-python@v2
+           name: Install Python
+           with:
+              python-version: '3.x'
+
+         - name: Cleaning any previously created items
+           run: |
+              python test/integration/k8s-deploy-delete.py 'Service' 'nginx-service' ${{ env.NAMESPACE }}
+              python test/integration/k8s-deploy-delete.py 'Service' 'nginx-service-green' ${{ env.NAMESPACE }}
+              python test/integration/k8s-deploy-delete.py 'Deployment' 'nginx-deployment-green' ${{ env.NAMESPACE }}
+              python test/integration/k8s-deploy-delete.py 'Deployment' 'nginx-deployment' ${{ env.NAMESPACE }}
+              python test/integration/k8s-deploy-delete.py 'Ingress' 'nginx-ingress' ${{ env.NAMESPACE }}
+
+         - name: Executing deploy action for ingress
+           uses: ./
+           with:
+              namespace: ${{ env.NAMESPACE }}
+              images: nginx:1.14.2
+              manifests: |
+                 test/integration/manifests/blue-green/test-ingress.yml
+              strategy: blue-green
+              route-method: ingress
+              action: deploy
+
+         - name: Checking if deployments, services and ingresses were created with green labels and original tag
+           run: |
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Deployment name=nginx-deployment-green containerName=nginx:1.14.2 labels=k8s.deploy.color:green,app:nginx,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:green
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Service name=nginx-service-green labels=k8s.deploy.color:green,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:green
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Ingress name=nginx-ingress ingressServices=nginx-service-green,unrouted-service
+
+         - name: Executing promote action for ingress
+           uses: ./
+           with:
+              namespace: ${{ env.NAMESPACE }}
+              images: nginx:1.14.2
+              manifests: |
+                 test/integration/manifests/blue-green/test-ingress.yml
+              strategy: blue-green
+              route-method: ingress
+              action: promote
+
+         - name: Checking if deployments, services and ingresses were created with none labels after first promote
+           run: |
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Deployment name=nginx-deployment containerName=nginx:1.14.2 labels=k8s.deploy.color:None,app:nginx,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:None
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Service name=nginx-service labels=k8s.deploy.color:None,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:None
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Ingress name=nginx-ingress ingressServices=nginx-service,unrouted-service
+
+         - name: Executing second deploy action for ingress with new tag
+           uses: ./
+           with:
+              namespace: ${{ env.NAMESPACE }}
+              images: nginx:latest
+              manifests: |
+                 test/integration/manifests/blue-green/test-ingress.yml
+              strategy: blue-green
+              route-method: ingress
+              action: deploy
+
+         - name: Checking if deployments (with new tag), services and ingresses were created with green labels after deploy, and old deployment persists
+           run: |
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Deployment name=nginx-deployment-green containerName=nginx:latest labels=k8s.deploy.color:green,app:nginx,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:green
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Deployment name=nginx-deployment containerName=nginx:1.14.2 labels=k8s.deploy.color:None,app:nginx,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:None
+
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Service name=nginx-service-green labels=k8s.deploy.color:green,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:green
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Service name=nginx-service labels=k8s.deploy.color:None,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:None
+
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Ingress name=nginx-ingress ingressServices=nginx-service-green,unrouted-service
+
+         - name: Executing second promote action for ingress now using new image tag
+           uses: ./
+           with:
+              namespace: ${{ env.NAMESPACE }}
+              images: nginx:latest
+              manifests: |
+                 test/integration/manifests/blue-green/test-ingress.yml
+              strategy: blue-green
+              route-method: ingress
+              action: promote
+
+         - name: Checking if deployments, services and ingresses were created with none labels after promote for new tag
+           run: |
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Deployment name=nginx-deployment containerName=nginx:latest labels=k8s.deploy.color:None,app:nginx,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:None
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Service name=nginx-service labels=k8s.deploy.color:None,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:None
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Ingress name=nginx-ingress ingressServices=nginx-service,unrouted-service
+
+         - name: Executing deploy action for ingress to be rejected using old tag
+           uses: ./
+           with:
+              namespace: ${{ env.NAMESPACE }}
+              images: nginx:1.14.2
+              manifests: |
+                 test/integration/manifests/blue-green/test-ingress.yml
+              strategy: blue-green
+              route-method: ingress
+              action: deploy
+
+         - name: Checking if new deployments (with old tag), services and ingresses were created with green labels after deploy, and old deployment (with latest tag) persists
+           run: |
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Deployment name=nginx-deployment-green containerName=nginx:1.14.2 labels=k8s.deploy.color:green,app:nginx,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:green
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Deployment name=nginx-deployment containerName=nginx:latest labels=k8s.deploy.color:None,app:nginx,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:None
+
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Service name=nginx-service-green labels=k8s.deploy.color:green,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:green
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Service name=nginx-service labels=k8s.deploy.color:None,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:None
+
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Ingress name=nginx-ingress ingressServices=nginx-service-green,unrouted-service
+
+         - name: Executing reject action for ingress to reject new deployment with 1.14.2 tag
+           uses: ./
+           with:
+              namespace: ${{ env.NAMESPACE }}
+              images: nginx:1.14.2
+              manifests: |
+                 test/integration/manifests/blue-green/test-ingress.yml
+              strategy: blue-green
+              route-method: ingress
+              action: reject
+
+         # MAY BE USEFUL TO ADD AN ANTI-CHECK - CHECK TO MAKE SURE CERTAIN OBJECTS DON'T EXIST
+         - name: Checking if deployments, services and ingresses were created with none labels and latest tag after reject
+           run: |
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Deployment name=nginx-deployment containerName=nginx:latest labels=k8s.deploy.color:None,app:nginx,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:None
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Service name=nginx-service labels=k8s.deploy.color:None,workflow:actions.github.com-k8s-deploy,workflowFriendlyName:Minikube_Integration_Tests_-_blue-green_ingress selectorLabels=app:nginx,k8s.deploy.color:None
+              python test/integration/k8s-deploy-test.py namespace=${{ env.NAMESPACE }} kind=Ingress name=nginx-ingress ingressServices=nginx-service,unrouted-service
+
+         - name: Cleaning up current set up
+           run: |
+              python test/integration/k8s-deploy-delete.py 'Service' 'nginx-service' ${{ env.NAMESPACE }}
+              python test/integration/k8s-deploy-delete.py 'Deployment' 'nginx-deployment' ${{ env.NAMESPACE }}
+
+         - if: ${{ always() }}
+           name: Delete created namespace
+           run: kubectl delete ns ${{ env.NAMESPACE }}