Add authentication to metrics endpoint

[jkroepke]

Description

Add authentication to the metrics endpoint. It should be only accessible with an specific user role oder admin account.

Expected Behavior

http://127.0.0.1:8080/auth/realms/master/metrics should be return 503 unless a valid session with specific role logged in.

Actual Behavior

http://127.0.0.1:8080/auth/realms/master/metrics is would accessible.

[pb82]

@jkroepke having a way to restrict access to the metrics endpoint sounds reasonable. Usually this endpoint is scraped by Prometheus which won't have a valid keycloak session. Wondering what's the best way here, i'll try to give it some thought.

[jkroepke]

Some inspiration from here https://github.com/AndreyVMarkelov/jira-prometheus-exporter/

https://github.com/AndreyVMarkelov/jira-prometheus-exporter/blob/master/src/main/java/ru/andreymarkelov/atlas/plugins/promjiraexporter/servlet/PrometheusExporter.java#L35

This exporter generates a static key. If you want to scrape the endpoint you must attach the key as get parameter. Otherwise you get an unauthorized messages.

2 possible ways to store the static key inside keycloak.

• The User must be define the key via -D property on startup
• The realm object (https://www.keycloak.org/docs-api/5.0/rest-api/index.html#_realmrepresentation) supports a attribute attributes which is designed for user properties. Maybe it's possible to safe the key there.

[danifr]

I am interested in this. Having those keycloak metrics open to everyone does not seem like good idea.

@jkroepke did you managed to configure it? Thanks

[danifr]

As a workaround and since I have an HAproxy in front of my Keycloak servers, I added a http-request deny to my HAproxy frontend configuration:

frontend https-keycloak
  ...
  balance  roundrobin
  http-request  deny if { path -i -m reg /auth/realms/.+\/metrics } !{ src <IP_PROMETHEUS_SERVER> }
  mode  http

This config seems to do the trick.

[giner]

One way to restrict metrics endpoint is mentioned under Keycloak admin docs https://www.keycloak.org/docs/11.0/server_admin/#admin-endpoints-and-console where you replace path-prefix('/auth/admin') by
either path-template('/auth/realms/{realm}/metrics')
or (path-prefix('/auth/admin') or path-template('/auth/realms/{realm}/metrics'))

[jelly]

I've fixed this problem with an nginx location rule and basic auth (ofcourse servering on https)

    location ~ /auth/realms/[a-z]+/metrics  {
        auth_basic "Prometheus exporter";
        auth_basic_user_file /etc/nginx/auth/prometheus;
        proxy_set_header    Host               $host;
        proxy_set_header    X-Real-IP          $remote_addr;
        proxy_set_header    X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto  $scheme;
        proxy_ssl_verify    off;
        proxy_pass https://localhost:8443;
    }

    location / {
        proxy_set_header    Host               $host;
        proxy_set_header    X-Real-IP          $remote_addr;
        proxy_set_header    X-Forwarded-For    $proxy_add_x_forwarded_for;
        proxy_set_header    X-Forwarded-Proto  $scheme;
        proxy_ssl_verify    off;
        proxy_pass https://localhost:8443;
    }

Prometheus itself supports base64 authentication.

[jkroepke]

I still would like to see an in-tree solution without the need to configure an reverse proxy.

[Tim-Schwalbe]

Here is my NGINX Ingress Config to block requests to /auth/realms/master/metrics

   annotations: 
    kubernetes.io/ingress.class: "nginx"
    kubernetes.io/tls-acme: "true"
    nginx.ingress.kubernetes.io/server-snippet: |
      location /auth/realms/master/metrics {
          return 403;
      }

[ghost]

The location should probably be location ~ /auth/realms/[^/]*/metrics to get all realms.

[Cyben]

I think it is really important to restrict this endpoint from regular users as well.

Seems like the best and easiest way is configuring it in the keycloak configuration.

As @giner mentioned:

One way to restrict metrics endpoint is mentioned under Keycloak admin docs https://www.keycloak.org/docs/11.0/server_admin/#admin-endpoints-and-console where you replace path-prefix('/auth/admin') by
either path-template('/auth/realms/{realm}/metrics')
or (path-prefix('/auth/admin') or path-template('/auth/realms/{realm}/metrics'))

Maybe just add the jboss cli command in the documentation, at least for now.

There is also a way to implement everything as and "admin rest api" but it requires some work, so I'm not sure if this is the approach you want to go with.

[TDog42]

It seems that with KeyCloak 17 there is no way to rextrict endpoints anymore within the server configuration (keycloak/keycloak#12394)

Maybe just add an option for some basic authentication. This way not the whole internet will be able to sniff out the metrics