Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenFaaS does not work on a Istio enabled cluster #369

Closed
stefanprodan opened this issue Feb 3, 2019 · 5 comments
Closed

OpenFaaS does not work on a Istio enabled cluster #369

stefanprodan opened this issue Feb 3, 2019 · 5 comments

Comments

@stefanprodan
Copy link
Contributor

stefanprodan commented Feb 3, 2019

OpenFaaS installed with asyc enabled does not work with Istio.

Expected Behaviour

OpenFaaS should work with Istio v1.0.3 or newer.

Current Behaviour

OpenFaaS gateway, faas-idler and queue-worker are crashing at startup:

NAME                            READY   STATUS             RESTARTS   AGE
alertmanager-9c757596d-4l6tr    2/2     Running            0          8m
faas-idler-c6d6966bd-7z72f      1/2     CrashLoopBackOff   6          8m
gateway-6b6585884f-m5l8r        2/3     CrashLoopBackOff   6          8m
nats-85fddd8596-lc5x9           2/2     Running            0          8m
prometheus-dc74c8b54-8r4z4      2/2     Running            0          8m
queue-worker-5f9b67b6f9-f2qf9   1/2     CrashLoopBackOff   6          8m

faas-idler logs:

panic: invalid character 'o' in literal null (expecting 'u')

goroutine 1 [running]:
main.main()
	/go/src/github.com/openfaas-incubator/faas-idler/main.go:63 +0x66d

gateway logs:

2019/02/03 09:00:13 Opening connection to nats://nats.openfaas.svc.cluster.local:4222
2019/02/03 09:00:16 read tcp 10.40.0.29:45576->10.43.240.237:4222: i/o timeout

queue-worker logs:

Can't connect: read tcp 10.40.0.32:42966->10.43.240.237:4222: i/o timeout

When installing OpenFaaS with async off, the gateway starts but Prometheus can't scrape it using the SD discovery. Disabling the Istio sidecar for Prometheus fixes this.

nats:// is not supported by Envoy, so it must be disable nats-io/nats-operator#111

Possible Solution

Disable Istio sidecar injection on NATS, prometheus, alertmanager and faas-idler see #363

Steps to Reproduce (for bugs)

  1. Install Istio
  2. Install OpenFaaS with helm or with yamls
  3. kubectl -n openfaas get po

Context

OpenFaaS was working with Istio see #279

Your Environment

GKE 1.11 with Istio add-on (1.0.3)

@stefanprodan
Copy link
Contributor Author

Fixed by #363

@lomeshpatel
Copy link

I'm having the exact problem described in the original post. I've verified that the Istio sidecar injection has been disabled according to the PR.

My Environment

  • GKE: 1.13.7-gke.8
  • Istio: 1.1.7

Please let me know if you need more information. I've currently got it working with async=false.

@alexellis
Copy link
Member

Hi @lomeshpatel,

As far as I am aware @stefanprodan runs Istio + OpenFaaS on his cluster, so I wouldn't expect any issues for you.

You will need to create a new issue so that we gather everything we need from you.

Alex

@alexellis
Copy link
Member

Derek lock

@derek derek bot locked and limited conversation to collaborators Jul 5, 2019
@alexellis
Copy link
Member

--
Join Slack to connect with the community
https://docs.openfaas.com

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants