Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running local-setup makefile locally throw errors #879

Closed
AugustasV opened this issue Apr 28, 2022 · 8 comments
Closed

Running local-setup makefile locally throw errors #879

AugustasV opened this issue Apr 28, 2022 · 8 comments

Comments

@AugustasV
Copy link
Contributor

AugustasV commented Apr 28, 2022 •
edited
Loading

While I tried to run make deploy-full-local-setup, received errors

Deploy GSLB cr 
kubectl apply -f deploy/crds/test-namespace.yaml
namespace/test-gslb created
sed -i 's/cloud\.example\.com/cloud.example.com/g' "deploy/crds/k8gb.absa.oss_v1beta1_gslb_cr.yaml"
kubectl apply -f "deploy/crds/k8gb.absa.oss_v1beta1_gslb_cr.yaml"
error: error validating "deploy/crds/k8gb.absa.oss_v1beta1_gslb_cr.yaml": error validating data: [ValidationError(Gslb.spec.ingress.rules[0].http.paths[0].backend): unknown field "serviceName" in oss.absa.k8gb.v1beta1.Gslb.spec.ingress.rules.http.paths.backend, ValidationError(Gslb.spec.ingress.rules[0].http.paths[0].backend): unknown field "servicePort" in oss.absa.k8gb.v1beta1.Gslb.spec.ingress.rules.http.paths.backend, ValidationError(Gslb.spec.ingress.rules[0].http.paths[0]): missing required field "pathType" in oss.absa.k8gb.v1beta1.Gslb.spec.ingress.rules.http.paths, ValidationError(Gslb.spec.ingress.rules[1].http.paths[0].backend): unknown field "serviceName" in oss.absa.k8gb.v1beta1.Gslb.spec.ingress.rules.http.paths.backend, ValidationError(Gslb.spec.ingress.rules[1].http.paths[0].backend): unknown field "servicePort" in oss.absa.k8gb.v1beta1.Gslb.spec.ingress.rules.http.paths.backend, ValidationError(Gslb.spec.ingress.rules[1].http.paths[0]): missing required field "pathType" in oss.absa.k8gb.v1beta1.Gslb.spec.ingress.rules.http.paths, ValidationError(Gslb.spec.ingress.rules[2].http.paths[0].backend): unknown field "serviceName" in oss.absa.k8gb.v1beta1.Gslb.spec.ingress.rules.http.paths.backend, ValidationError(Gslb.spec.ingress.rules[2].http.paths[0].backend): unknown field "servicePort" in oss.absa.k8gb.v1beta1.Gslb.spec.ingress.rules.http.paths.backend, ValidationError(Gslb.spec.ingress.rules[2].http.paths[0]): missing required field "pathType" in oss.absa.k8gb.v1beta1.Gslb.spec.ingress.rules.http.paths]; if you choose to ignore these errors, turn validation off with --validate=false

When I manually added --validate=false argument in make file, ran make destroy-full-local-setup and make deploy-full-local-setup output was a bit different, but still errors

Deploy GSLB cr 
kubectl apply -f deploy/crds/test-namespace.yaml
namespace/test-gslb created
sed -i 's/cloud\.example\.com/cloud.example.com/g' "deploy/crds/k8gb.absa.oss_v1beta1_gslb_cr.yaml"
kubectl apply -f "deploy/crds/k8gb.absa.oss_v1beta1_gslb_cr.yaml" --validate=false
The Gslb "test-gslb" is invalid: spec.ingress.rules.http.paths.pathType: Required value
make[3]: *** [Makefile:190: deploy-test-apps] Error 1
make[3]: Leaving directory '/home/namai/Documents/k8gb'
make[2]: *** [Makefile:174: deploy-local-cluster] Error 2
@AugustasV AugustasV changed the title Running local-setup makefile locally through errors Running local-setup makefile locally throw errors Apr 28, 2022
@k0da
Copy link
Collaborator

k0da commented Apr 28, 2022

It looks like created cluster is not Ingress v1 compatible

@AugustasV
Copy link
Contributor Author

why is that? k3d version is updated

@k0da
Copy link
Collaborator

k0da commented Apr 28, 2022

@AugustasV ah. unknown field "serviceName" can you please make sure your working copy is up2date? especially if it have 0d0cb59 commit

@AugustasV
Copy link
Contributor Author

@k0da I cloned repo again, ran same commands and script executed successfully, but after running make terratest I see that there are still some problems with ingress, it was not install correctly.

=== CONT  TestK8gbBasicAppExample
    retry.go:59: 'Wait for ingress test-gslb to be provisioned.' unsuccessful after 60 retries
TestK8gbBasicAppExample 2022-04-28T23:47:06+03:00 logger.go:66: Running command kubectl with args [--namespace k8gb-test-basic-app-vivxsc delete -f /home/namai/Documents/k8gb/terratest/examples/roundrobin.yaml]
TestK8gbBasicAppExample 2022-04-28T23:47:06+03:00 logger.go:66: gslb.k8gb.absa.oss "test-gslb" deleted
TestK8gbBasicAppExample 2022-04-28T23:47:06+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context 
--- FAIL: TestK8gbBasicAppExample (62.12s)
=== CONT  TestK8gbSpecKeepsStableAfterIngressUpdates
    retry.go:59: 'Wait for ingress test-gslb-lifecycle to be provisioned.' unsuccessful after 60 retries
TestK8gbSpecKeepsStableAfterIngressUpdates 2022-04-28T23:47:07+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context 
--- FAIL: TestK8gbSpecKeepsStableAfterIngressUpdates (62.16s)
=== CONT  TestK8gbSplitFailoverExample
    retry.go:59: 'Wait for ingress test-gslb to be provisioned.' unsuccessful after 60 retries
TestK8gbSplitFailoverExample 2022-04-28T23:47:07+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb2
TestK8gbSplitFailoverExample 2022-04-28T23:47:07+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
--- FAIL: TestK8gbSplitFailoverExample (62.23s)
=== CONT  TestK8gbRepeatedlyRecreatedFromIngress
    retry.go:59: 'Wait for ingress test-gslb-failover-simple to be provisioned.' unsuccessful after 60 retries
TestK8gbRepeatedlyRecreatedFromIngress 2022-04-28T23:47:07+03:00 logger.go:66: Running command kubectl with args [--namespace k8gb-test-repeatedly-recreated-from-ingress-wgwflu delete -f /home/namai/Documents/k8gb/terratest/examples/ingress-annotation-failover-simple.yaml]
TestK8gbRepeatedlyRecreatedFromIngress 2022-04-28T23:47:07+03:00 logger.go:66: ingress.networking.k8s.io "test-gslb-failover-simple" deleted
TestK8gbRepeatedlyRecreatedFromIngress 2022-04-28T23:47:07+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context 
--- FAIL: TestK8gbRepeatedlyRecreatedFromIngress (62.31s)
TestFullFailover 2022-04-28T23:47:07+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFullFailover 2022-04-28T23:47:07+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFullFailover 2022-04-28T23:47:07+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestK8gbIngressAnnotationRR 2022-04-28T23:47:07+03:00 logger.go:66: Running command kubectl with args [--namespace k8gb-test-ingress-annotation-rr-hclzol get gslb test-gslb-annotation -o custom-columns=SERVICESTATUS:.status.serviceHealth --no-headers]
TestK8gbIngressAnnotationRR 2022-04-28T23:47:07+03:00 logger.go:66: map[ingress-rr-notfound.cloud.example.com:NotFound ingress-rr-unhealthy.cloud.example.com:NotFound ingress-rr.cloud.example.com:NotFound]
=== CONT  TestK8gbIngressAnnotationRR
    utils.go:94: Wait for expected ServiceHealth status... does not match expected result. Expected:([map[ingress-roundrobin.cloud.example.com:NotFound ingress-rr-notfound.cloud.example.com:NotFound ingress-rr-unhealthy.cloud.example.com:NotFound]]). Actual:([map[ingress-rr-notfound.cloud.example.com:NotFound ingress-rr-unhealthy.cloud.example.com:NotFound ingress-rr.cloud.example.com:NotFound]]). Sleeping for 1s and will try again.
TestFailoverPlayground 2022-04-28T23:47:07+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFailoverPlayground 2022-04-28T23:47:07+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFailoverPlayground 2022-04-28T23:47:07+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestFullFailover 2022-04-28T23:47:08+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFullFailover 2022-04-28T23:47:08+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFullFailover 2022-04-28T23:47:08+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestK8gbIngressAnnotationRR 2022-04-28T23:47:08+03:00 logger.go:66: Running command kubectl with args [--namespace k8gb-test-ingress-annotation-rr-hclzol get gslb test-gslb-annotation -o custom-columns=SERVICESTATUS:.status.serviceHealth --no-headers]
TestFailoverPlayground 2022-04-28T23:47:08+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFailoverPlayground 2022-04-28T23:47:08+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFailoverPlayground 2022-04-28T23:47:08+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestK8gbIngressAnnotationRR 2022-04-28T23:47:08+03:00 logger.go:66: map[ingress-rr-notfound.cloud.example.com:NotFound ingress-rr-unhealthy.cloud.example.com:NotFound ingress-rr.cloud.example.com:NotFound]
    utils.go:94: Wait for expected ServiceHealth status... does not match expected result. Expected:([map[ingress-roundrobin.cloud.example.com:NotFound ingress-rr-notfound.cloud.example.com:NotFound ingress-rr-unhealthy.cloud.example.com:NotFound]]). Actual:([map[ingress-rr-notfound.cloud.example.com:NotFound ingress-rr-unhealthy.cloud.example.com:NotFound ingress-rr.cloud.example.com:NotFound]]). Sleeping for 1s and will try again.
TestFullFailover 2022-04-28T23:47:09+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFullFailover 2022-04-28T23:47:09+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFullFailover 2022-04-28T23:47:09+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestFailoverPlayground 2022-04-28T23:47:09+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFailoverPlayground 2022-04-28T23:47:09+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFailoverPlayground 2022-04-28T23:47:09+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestK8gbIngressAnnotationRR 2022-04-28T23:47:09+03:00 logger.go:66: Running command kubectl with args [--namespace k8gb-test-ingress-annotation-rr-hclzol get gslb test-gslb-annotation -o custom-columns=SERVICESTATUS:.status.serviceHealth --no-headers]
TestK8gbIngressAnnotationRR 2022-04-28T23:47:09+03:00 logger.go:66: map[ingress-rr-notfound.cloud.example.com:NotFound ingress-rr-unhealthy.cloud.example.com:NotFound ingress-rr.cloud.example.com:NotFound]
    utils.go:94: Wait for expected ServiceHealth status... does not match expected result. Expected:([map[ingress-roundrobin.cloud.example.com:NotFound ingress-rr-notfound.cloud.example.com:NotFound ingress-rr-unhealthy.cloud.example.com:NotFound]]). Actual:([map[ingress-rr-notfound.cloud.example.com:NotFound ingress-rr-unhealthy.cloud.example.com:NotFound ingress-rr.cloud.example.com:NotFound]]). Sleeping for 1s and will try again.
TestFullFailover 2022-04-28T23:47:10+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFullFailover 2022-04-28T23:47:10+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFullFailover 2022-04-28T23:47:10+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestFailoverPlayground 2022-04-28T23:47:10+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFailoverPlayground 2022-04-28T23:47:10+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFailoverPlayground 2022-04-28T23:47:10+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
    k8gb_ingress_annotation_rr_test.go:64: 
                Error Trace:    utils.go:190
                                                        k8gb_ingress_annotation_rr_test.go:64
                Error:          Received unexpected error:
                                'Wait for expected ServiceHealth status...' unsuccessful after 60 retries
                Test:           TestK8gbIngressAnnotationRR
TestK8gbIngressAnnotationRR 2022-04-28T23:47:10+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context 
--- FAIL: TestK8gbIngressAnnotationRR (65.93s)
TestFullFailover 2022-04-28T23:47:11+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFullFailover 2022-04-28T23:47:11+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFullFailover 2022-04-28T23:47:11+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestFailoverPlayground 2022-04-28T23:47:11+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFailoverPlayground 2022-04-28T23:47:11+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFailoverPlayground 2022-04-28T23:47:11+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestFullFailover 2022-04-28T23:47:12+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFullFailover 2022-04-28T23:47:12+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFullFailover 2022-04-28T23:47:12+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestFailoverPlayground 2022-04-28T23:47:12+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFailoverPlayground 2022-04-28T23:47:12+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFailoverPlayground 2022-04-28T23:47:12+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
TestFullFailover 2022-04-28T23:47:13+03:00 retry.go:91: Wait for ingress test-gslb to be provisioned.
TestFullFailover 2022-04-28T23:47:13+03:00 client.go:42: Configuring Kubernetes client using config file /home/namai/.kube/config with context k3d-test-gslb1
TestFullFailover 2022-04-28T23:47:13+03:00 retry.go:103: Wait for ingress test-gslb to be provisioned. returned an error: Ingress test-gslb is not available. Sleeping for 1s and will try again.
=== CONT  TestFailoverPlayground
    retry.go:59: 'Wait for ingress test-gslb to be provisioned.' unsuccessful after 60 retries
--- FAIL: TestFailoverPlayground (68.83s)
=== CONT  TestFullFailover
    retry.go:59: 'Wait for ingress test-gslb to be provisioned.' unsuccessful after 60 retries
--- FAIL: TestFullFailover (69.54s)
FAIL
exit status 1
FAIL    k8gbterratest/test      69.553s
make: *** [Makefile:432: terratest] Error 1

#834

@somaritane
Copy link
Contributor

yep, experiencing the same issue with the same test on fresh repo clone

@kuritka kuritka mentioned this issue Apr 29, 2022
Closed
@kuritka
Copy link
Collaborator

kuritka commented Apr 29, 2022 •
edited
Loading

@AugustasV , thx for opening this. I can simply reproduce failing terratests locally. To avoid mixing the Makefile issues you mentioned earlier with terartest issues, I created a new bug.

@kuritka kuritka closed this as completed Apr 29, 2022
@jkremser
Copy link
Member

The first issue you saw stems (i think) from the fact that make deploy-full-local-setup deploys the latest _locally present git tag. So that it deployed some older version of k8gb that was not ingress@v1 ready. After you cloned the repo again, it pulled the latest tag. Perhaps we should put the git fetch --tags somewhere in the makefile to make sure it deploys the latest version.

The other issue in terratests seems to be more difficult to handle. I can look into it. My first impression is that there is some naming mismatch

ingress-roundrobin.cloud.example.com:NotFound != ingress-rr.cloud.example.com:NotFound

What worries me is that our terratest check in github are green. It looks like the errors are being ignored.

@jkremser
Copy link
Member

I've also opened #883 b/c of the errors in terratests ignored by gh pipeline

@jkremser jkremser mentioned this issue May 2, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@somaritane @jkremser @k0da @kuritka @AugustasV