Proxy is instrumented with OTEL and will emit metrics and traces. To collect them OTEL Collector needs to deployed and configured.
- Proxy request: requests are forwarded using this path. Successful request will have 3 spans:
- ProxyRequest: This is the entry point.
- modifyRequest: This is where proxy will modify host and add BearerToken.
- modifyResponse: This is where proxy validates response.
- Access token refresh: This path is used by background thread to refresh token periodically.
- Readiness check: This path is used by liveness and readiness checks.
| Metric name | Type | Labels | Description |
|---|---|---|---|
| aad_auth_proxy_requests_total | counter | target_host, method, path, user_agent, status_code | A counter of total number of requests the proxy has received. This is incremented when response for a given request is received or failed to forward request. |
| aad_auth_proxy_request_duration_milliseconds | histogram | target_host, method, path, user_agent, status_code | A histogram of request to response duration. This calculated as the time duration between the time request was received and response was returned back to client. |
| aad_auth_proxy_request_bytes_total | counter | target_host, method, path, user_agent | A counter of total number of bytes received in incoming request. This is updated when request is received. |
| aad_auth_proxy_response_bytes_total | counter | target_host, method, path, user_agent, status_code | A counter of total number of bytes received in response. This is updated when response is received. |
| aad_auth_proxy_token_refresh_total | counter | audience, client_id, tenant_id, identity_type, is_success | A counter of total number of token refreshes. This is updated when token is refreshed. |
Note: Since aad_auth_proxy_request_duration_milliseconds is of type histogram it will emit three metrics
- aad_auth_proxy_request_duration_milliseconds_bucket
- aad_auth_proxy_request_duration_milliseconds_count
- aad_auth_proxy_request_duration_milliseconds_sum
Grafana dashboard can be used to visualize metrics. Job dropdown contains value of the config parameter OTEL_SERVICE_NAME specified in proxy config. In case multiple proxies send metrics to the same store, customer may need to choose particular instance of the proxy in this dropdown.
OTEL Collector to receive and export telemetry. sample-otel-collector-config.yaml shows that we are receiving traces and metrics via grpc. Traces are being exported to Jaeger tracing and to Application Insights via Azure Monitor exporter. Metrics are being exported to Azure Monitor Workspace via another proxy deployment in ingestion mode using prometheus remote write.