Take readiness probes into account #1
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Liveness probes might not be enough for GSLB scenario, we need to be sure that the application is ready to accept the traffic after some time after the start(java warm up is a classic example of that). Thus, for reliable load balancing we should check for readiness probe status and also strongly advise application team to configure one for their workload(s)
|
@donovanmuller can't assign reviewers so pinging you here |
kuritka
added a commit
that referenced
this pull request
Nov 5, 2020
kuritka
added a commit
that referenced
this pull request
Nov 11, 2020
deploy-full-local-setup fix localtargets name in documentation local playground passed wait deploy-gslb-operator-14 k8gb terratest, moving to makefile terratest-pipeline fix #1 terratest-pipeline fix #2 terratest-pipeline fix #3 terratest-pipeline fix #4 terratest-pipeline fix #5 move stable repo location from https://kubernetes-charts.storage.googleapis.com to https://charts.helm.sh/stable see: https://helm.sh/blog/new-location-stable-incubator-charts/ highlight large script sections fix version fix version #2 fix version #3 fix version #4 simplify release pipeline, move logic to Makefile fix terratest by adding image-repo between deploy-local-cluster args registry.sh #1 registry.sh; installing docker registry always in deploy-full-terratest-setup; remove registry.sh remove full.sh, remove registry.sh test; functionality, hide generate, controller gent, manifest remove GOPATH guardian as GO terraform-test runs clean ubuntu image, no GOLANG image refactor generate and manifest move kustomize run manager install infoblox-secret retreive version from shell , deploy fix deploy, remove kustomize udf bundle build-bundle, debug-local docker-test-build-push change git-last-commit-hash into shell COMMIT_HASH lowercase comments alphabetically sort targets remove deploy, bundle, bundle-build remove debug-local and create debug-local-idea instead refactor debug-local-idea to debug-idea refactor debug-local-idea to debug-idea
kuritka
added a commit
that referenced
this pull request
Nov 11, 2020
deploy-full-local-setup fix localtargets name in documentation local playground passed wait deploy-gslb-operator-14 k8gb terratest, moving to makefile terratest-pipeline fix #1 terratest-pipeline fix #2 terratest-pipeline fix #3 terratest-pipeline fix #4 terratest-pipeline fix #5 move stable repo location from https://kubernetes-charts.storage.googleapis.com to https://charts.helm.sh/stable see: https://helm.sh/blog/new-location-stable-incubator-charts/ highlight large script sections fix version fix version #2 fix version #3 fix version #4 simplify release pipeline, move logic to Makefile fix terratest by adding image-repo between deploy-local-cluster args registry.sh #1 registry.sh; installing docker registry always in deploy-full-terratest-setup; remove registry.sh remove full.sh, remove registry.sh test; functionality, hide generate, controller gent, manifest remove GOPATH guardian as GO terraform-test runs clean ubuntu image, no GOLANG image refactor generate and manifest move kustomize run manager install infoblox-secret retreive version from shell , deploy fix deploy, remove kustomize udf bundle build-bundle, debug-local docker-test-build-push change git-last-commit-hash into shell COMMIT_HASH lowercase comments alphabetically sort targets remove deploy, bundle, bundle-build remove debug-local and create debug-local-idea instead refactor debug-local-idea to debug-idea refactor debug-local-idea to debug-idea
kuritka
added a commit
that referenced
this pull request
Nov 12, 2020
deploy-full-local-setup fix localtargets name in documentation local playground passed wait deploy-gslb-operator-14 k8gb terratest, moving to makefile terratest-pipeline fix #1 terratest-pipeline fix #2 terratest-pipeline fix #3 terratest-pipeline fix #4 terratest-pipeline fix #5 move stable repo location from https://kubernetes-charts.storage.googleapis.com to https://charts.helm.sh/stable see: https://helm.sh/blog/new-location-stable-incubator-charts/ highlight large script sections fix version fix version #2 fix version #3 fix version #4 simplify release pipeline, move logic to Makefile fix terratest by adding image-repo between deploy-local-cluster args registry.sh #1 registry.sh; installing docker registry always in deploy-full-terratest-setup; remove registry.sh remove full.sh, remove registry.sh test; functionality, hide generate, controller gent, manifest remove GOPATH guardian as GO terraform-test runs clean ubuntu image, no GOLANG image refactor generate and manifest move kustomize run manager install infoblox-secret retreive version from shell , deploy fix deploy, remove kustomize udf bundle build-bundle, debug-local docker-test-build-push change git-last-commit-hash into shell COMMIT_HASH lowercase comments alphabetically sort targets remove deploy, bundle, bundle-build remove debug-local and create debug-local-idea instead refactor debug-local-idea to debug-idea refactor debug-local-idea to debug-idea
kuritka
added a commit
that referenced
this pull request
Nov 12, 2020
deploy-full-local-setup fix localtargets name in documentation local playground passed wait deploy-gslb-operator-14 k8gb terratest, moving to makefile terratest-pipeline fix #1 terratest-pipeline fix #2 terratest-pipeline fix #3 terratest-pipeline fix #4 terratest-pipeline fix #5 move stable repo location from https://kubernetes-charts.storage.googleapis.com to https://charts.helm.sh/stable see: https://helm.sh/blog/new-location-stable-incubator-charts/ highlight large script sections fix version fix version #2 fix version #3 fix version #4 simplify release pipeline, move logic to Makefile fix terratest by adding image-repo between deploy-local-cluster args registry.sh #1 registry.sh; installing docker registry always in deploy-full-terratest-setup; remove registry.sh remove full.sh, remove registry.sh test; functionality, hide generate, controller gent, manifest remove GOPATH guardian as GO terraform-test runs clean ubuntu image, no GOLANG image refactor generate and manifest move kustomize run manager install infoblox-secret retreive version from shell , deploy fix deploy, remove kustomize udf bundle build-bundle, debug-local docker-test-build-push change git-last-commit-hash into shell COMMIT_HASH lowercase comments alphabetically sort targets remove deploy, bundle, bundle-build remove debug-local and create debug-local-idea instead refactor debug-local-idea to debug-idea refactor debug-local-idea to debug-idea
kuritka
added a commit
that referenced
this pull request
Nov 13, 2020
deploy-full-local-setup fix localtargets name in documentation local playground passed wait deploy-gslb-operator-14 k8gb terratest, moving to makefile terratest-pipeline fix #1 terratest-pipeline fix #2 terratest-pipeline fix #3 terratest-pipeline fix #4 terratest-pipeline fix #5 move stable repo location from https://kubernetes-charts.storage.googleapis.com to https://charts.helm.sh/stable see: https://helm.sh/blog/new-location-stable-incubator-charts/ highlight large script sections fix version fix version #2 fix version #3 fix version #4 simplify release pipeline, move logic to Makefile fix terratest by adding image-repo between deploy-local-cluster args registry.sh #1 registry.sh; installing docker registry always in deploy-full-terratest-setup; remove registry.sh remove full.sh, remove registry.sh test; functionality, hide generate, controller gent, manifest remove GOPATH guardian as GO terraform-test runs clean ubuntu image, no GOLANG image refactor generate and manifest move kustomize run manager install infoblox-secret retreive version from shell , deploy fix deploy, remove kustomize udf bundle build-bundle, debug-local docker-test-build-push change git-last-commit-hash into shell COMMIT_HASH lowercase comments alphabetically sort targets remove deploy, bundle, bundle-build remove debug-local and create debug-local-idea instead refactor debug-local-idea to debug-idea refactor debug-local-idea to debug-idea
kuritka
added a commit
that referenced
this pull request
Oct 26, 2021
related to #697 `k8s.io/*` bundle to `v0.22.2` (controller-gen tool makes changes in CRD description after bump) - `github.com/golang/mock` to `v1.6.0` (is not compiled into final binary but checked by dependabot) - `github.com/miekg/dns` to `v1.1.43` - `github.com/infobloxopen/infoblox-go-client` to `v1.1.1` (changes argument CreateTXTRecord from integer to uint) The latest `sigs.k8s.io/external-dns` is problematic in terms it uses latest `github.com/go-logr/logr` which is incompatible with previous versions. This causes issues in `controller-runtime` and will require code changes on our side (another PR). [This PR](kubernetes-sigs/controller-runtime#1593 (comment)) would be unblock - github.com/go-logr/logr - sigs.k8s.io/external-dns Signed-off-by: kuritka <kuritka@gmail.com>
kuritka
added a commit
that referenced
this pull request
Oct 26, 2021
related to #697 `k8s.io/*` bundle to `v0.22.2` (controller-gen tool makes changes in CRD description after bump) - `github.com/golang/mock` to `v1.6.0` (is not compiled into final binary but checked by dependabot) - `github.com/miekg/dns` to `v1.1.43` - `github.com/infobloxopen/infoblox-go-client` to `v1.1.1` (changes argument CreateTXTRecord from integer to uint) `sigs.k8s.io/controller-runtime` to `v0.10.2` The latest `sigs.k8s.io/external-dns` is problematic in terms it uses latest `github.com/go-logr/logr` which is incompatible with previous versions. This causes issues in `controller-runtime` and will require code changes on our side (another PR). [This PR](kubernetes-sigs/controller-runtime#1593 (comment)) would be unblock - github.com/go-logr/logr - sigs.k8s.io/external-dns Signed-off-by: kuritka <kuritka@gmail.com>
kuritka
added a commit
that referenced
this pull request
Oct 26, 2021
related to #697 `k8s.io/*` bundle to `v0.22.2` (controller-gen tool makes changes in CRD description after bump) - `github.com/golang/mock` to `v1.6.0` (is not compiled into final binary but checked by dependabot) - `github.com/miekg/dns` to `v1.1.43` - `github.com/infobloxopen/infoblox-go-client` to `v1.1.1` (changes argument CreateTXTRecord from integer to uint) `sigs.k8s.io/controller-runtime` to `v0.10.2` requires more effort, will be part of follow-up PR `sigs.k8s.io/external-dns` is problematic in terms it uses latest `github.com/go-logr/logr` which is incompatible with previous versions. Will be part of followup PR Signed-off-by: kuritka <kuritka@gmail.com>
kuritka
added a commit
that referenced
this pull request
Oct 27, 2021
related to #697 `k8s.io/*` bundle to `v0.22.2` (controller-gen tool makes changes in CRD description after bump) - `github.com/golang/mock` to `v1.6.0` (is not compiled into final binary but checked by dependabot) - `github.com/miekg/dns` to `v1.1.43` - `github.com/infobloxopen/infoblox-go-client` to `v1.1.1` (changes argument CreateTXTRecord from integer to uint) - `sigs.k8s.io/controller-runtime` to `v0.10.2`, had to fix the test, see comment below. - `sigs.k8s.io/external-dns` is problematic in terms it uses latest `github.com/go-logr/logr` which is incompatible with previous versions. Will be part of followup PR Controller-runtime works on copies of annotations rather than their pointers, so I had to modify the test. ```go func TestGslbProperlyPropagatesAnnotationDownToIngress(t *testing.T) { // arrange settings := provideSettings(t, predefinedConfig) expectedAnnotations := map[string]string{"annotation": "test"} settings.gslb.Annotations = expectedAnnotations err := settings.client.Update(context.TODO(), settings.gslb) require.NoError(t, err, "Can't update gslb") // act reconcileAndUpdateGslb(t, settings) err2 := settings.client.Get(context.TODO(), settings.request.NamespacedName, settings.ingress) // assert assert.NoError(t, err2, "Failed to get expected ingress") assert.Equal(t, expectedAnnotations, settings.ingress.Annotations) } ``` If I extend fial assertion the passing test is doing this: ```go assert.Equal(t, map[string]string{"annotation": "test", "k8gb.io/strategy": "roundRobin"}, settings.ingress.Annotations) assert.Equal(t, map[string]string{"annotation": "test", "k8gb.io/strategy": "roundRobin"}, settings.gslb.Annotations) assert.Equal(t, map[string]string{"annotation": "test", "k8gb.io/strategy": "roundRobin"}, expectedAnnotations) ``` The `"k8gb.io/strategy": "roundRobin"` is added to `expectedAnnotations` during `reconcileAndUpdateGslb` and controlle-runtime is the guy which extends `expectedAnnotations`. I bumped controller runtime to `v0.10.2`, and only this concrete test has to be updated, because expectedAnnotations are not altered within controller-runtime: ```go assert.Equal(t, map[string]string{"annotation": "test", "k8gb.io/strategy": "roundRobin"}, settings.ingress.Annotations) assert.Equal(t, map[string]string{"annotation": "test", "k8gb.io/strategy": "roundRobin"}, settings.gslb.Annotations) assert.Equal(t, map[string]string{"annotation": "test"}, expectedAnnotations) ``` Signed-off-by: kuritka <kuritka@gmail.com>
kuritka
added a commit
that referenced
this pull request
Oct 30, 2021
related to #697 `k8s.io/*` bundle to `v0.22.2` (controller-gen tool makes changes in CRD description after bump) - `github.com/golang/mock` to `v1.6.0` (is not compiled into final binary but checked by dependabot) - `github.com/miekg/dns` to `v1.1.43` - `github.com/infobloxopen/infoblox-go-client` to `v1.1.1` (changes argument CreateTXTRecord from integer to uint) - `sigs.k8s.io/controller-runtime` to `v0.10.2`, had to fix the test, see comment below. - `sigs.k8s.io/external-dns` is problematic in terms it uses latest `github.com/go-logr/logr` which is incompatible with previous versions. Will be part of followup PR Controller-runtime works on copies of annotations rather than their pointers, so I had to modify the test. ```go func TestGslbProperlyPropagatesAnnotationDownToIngress(t *testing.T) { // arrange settings := provideSettings(t, predefinedConfig) expectedAnnotations := map[string]string{"annotation": "test"} settings.gslb.Annotations = expectedAnnotations err := settings.client.Update(context.TODO(), settings.gslb) require.NoError(t, err, "Can't update gslb") // act reconcileAndUpdateGslb(t, settings) err2 := settings.client.Get(context.TODO(), settings.request.NamespacedName, settings.ingress) // assert assert.NoError(t, err2, "Failed to get expected ingress") assert.Equal(t, expectedAnnotations, settings.ingress.Annotations) } ``` If I extend fial assertion the passing test is doing this: ```go assert.Equal(t, map[string]string{"annotation": "test", "k8gb.io/strategy": "roundRobin"}, settings.ingress.Annotations) assert.Equal(t, map[string]string{"annotation": "test", "k8gb.io/strategy": "roundRobin"}, settings.gslb.Annotations) assert.Equal(t, map[string]string{"annotation": "test", "k8gb.io/strategy": "roundRobin"}, expectedAnnotations) ``` The `"k8gb.io/strategy": "roundRobin"` is added to `expectedAnnotations` during `reconcileAndUpdateGslb` and controlle-runtime is the guy which extends `expectedAnnotations`. I bumped controller runtime to `v0.10.2`, and only this concrete test has to be updated, because expectedAnnotations are not altered within controller-runtime: ```go assert.Equal(t, map[string]string{"annotation": "test", "k8gb.io/strategy": "roundRobin"}, settings.ingress.Annotations) assert.Equal(t, map[string]string{"annotation": "test", "k8gb.io/strategy": "roundRobin"}, settings.gslb.Annotations) assert.Equal(t, map[string]string{"annotation": "test"}, expectedAnnotations) ``` Signed-off-by: kuritka <kuritka@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Liveness probes might not be enough for GSLB scenario, we need
to be sure that the application is ready to accept the traffic after
some time after the start(java warm up is a classic example of that).
Thus, for reliable load balancing we should check for readiness
probe status and also strongly advise application team to configure one
for their workload(s)