examples/k8s: use ingress to route traffic to dex

[ericchiang]

For real clusters, dex won't be exposed directly to the outside, it'll go through some sort of ingress controller. Make the example manifests do this. This also creates a cleaner point of customization, allowing users to re-purpose the examples by switching out the ingress for whatever their setup is.

Related to #682

Need to chat with @dghubble about possible setups.

[set321go]

An example of ingress using nginx auth_request would probably help a lot of people. Here is an example ingress I use with traefik on gce.

The important part here for proxying is kubernetes.io/ingress.class: "traefik" which prevents gce from wiring it up to an external LB, this is an adapted version found in nginx tutorials.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: traefik-ui
 namespace: kube-system
 annotations:
   kubernetes.io/ingress.class: "traefik"
   traefik.frontend.rule.type: pathprefixstrip
spec:
 rules:
 - host: demo.domain.com
   http:
     paths:
     - path: /admin/proxy
       backend:
        serviceName: traefik-web-ui
        servicePort: 80

One of the most difficult things i've found trying to learn dex is the documentation is not very 'real world' I know there are some tickets for this.

[ericchiang]

An example of ingress using nginx auth_request would probably help a lot of people.

That module seems to be dealing with authorizing client request, while this issue is about how we should recommend using ingress to allow traffic to dex Maybe you're asking for a different feature?

One of the most difficult things i've found trying to learn dex is the documentation is not very 'real world' I know there are some tickets for this.

Fair enough. Though yeah, that's why we open these kinds of issues :)

[set321go]

Maybe i'm confusing two different problems. I'm thinking of auth for my application(s) running in k8s where the k8s example provided in the project is about authenticating clients for cluster mgmt right?

[ericchiang]

Yeah those are two different issues. Let me open an issue for the one you're describing though. We need it badly.

[roffe]

Got lead here by @rithujohn191 reply in #682 But i cannot see how this issue has anything with the actual issue in #682 to do.

[ericchiang]

@roffe if you run dex behind ingress, ingress does the TLS termination. Would let you run dex in the cluster on HTTP and not have the HTTPS check fail.

Also we added OpenSSL to the container a long time ago #685, so wget should be able to query HTTPS.

[kiich]

Just to add, we have Dex running in k8s cluster exposed via service (with appropriate RBAC policy) instead of ingress but would very much like to see this feature as it would mean not requiring separate tls for dex.

[SachinMaharana]

I am having a hard time to understand how to expose dex as https with custom CA, when the ssl termination is being done by the dex. Should i use ingress or a simple nodeport will do? I can access httpp://NODEIP:NODEPORT but apiserver ca flag complains.

[mabushey]

@SachinMaharana Ever figure this out? It's like no one grasps that there's going to be a load balancer doing SSL termination.

[laurelnaiad]

I have just configured dex to run behind an ingress. The ingress is responsible for TLS, and gets its cert from letsencrypt via cert-manager. dex and its service run plain http.

It took me a little while to get the magic recipe. I was under the impression, by apparently misreading various statements on the subject, that I needed to create a ca file for the dex instance and give that to the kube-apiserver, then rely on the server cert created from that ca, whereas in fact, if your issuer endpoint is running with a trusted certificate (e.g. one from letsencrypt), then you don't need to mess with ca certs. Key point: dex doesn't need a ca certs if the ingress in front of it has a valid one. So, the kube-apiserver config – that is here, with no oidc-ca-file: https://github.com/laurelnaiad/k8s-on-eoan/blob/master/platform-install/kubeadm-kubelet-kubectl-install.sh#L91

https://github.com/laurelnaiad/k8s-on-eoan/blob/master/04-k8s-config.sh for where the actual dex config is done, in context with other services, some of which are related, noting that some of the environment variables in these scripts are coming from the two .sh files in the root of the repo. You could just prepend a few assignments to the top of the file. shows the order in which I am building it into a larger deployment.

I'm not quite finished with that repository -- I just pushed it up there before it was really finished so I could share the dex config. The files layout might change a little in the near future. If the cited paths don't work, just go to the repo, the dex stuff will be there somewhere. :) I'm going to try to work on setting up pusher/oauth2_proxy with the dashboard this afternoon.

EDIT: just one FYI – if you're running nginx-ingress and you want your dex instance exposed on something other than port 443, you'll need to configure a separate ingress-controller to listen on that port. That's the nature of the nginx-ingress. Me, I wanted it on 443, anyway, so I was not bothered by this.

[sagikazarmark]

See the helm chart for an official example.