Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce Management Interface #30506

Merged
merged 1 commit into from
Mar 16, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/native-tests.json
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@
{
"category": "HTTP",
"timeout": 95,
"test-modules": "elytron-resteasy, resteasy-jackson, elytron-resteasy-reactive, resteasy-mutiny, resteasy-reactive-kotlin/standard, vertx, vertx-http, vertx-web, vertx-web-jackson, vertx-graphql, virtual-http, rest-client, rest-client-reactive, rest-client-reactive-stork, rest-client-reactive-multipart, websockets",
"test-modules": "elytron-resteasy, resteasy-jackson, elytron-resteasy-reactive, resteasy-mutiny, resteasy-reactive-kotlin/standard, vertx, vertx-http, vertx-web, vertx-web-jackson, vertx-graphql, virtual-http, rest-client, rest-client-reactive, rest-client-reactive-stork, rest-client-reactive-multipart, websockets, management-interface",
"os-name": "ubuntu-latest"
},
{
Expand Down
12 changes: 11 additions & 1 deletion docs/src/main/asciidoc/http-reference.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,16 @@ As an example, if an extension configures a `service` path, that endpoint will b

The link:https://quarkus.io/blog/path-resolution-in-quarkus/[Path Resolution in Quarkus] blog post further explains how path resolution works for both user and extension defined paths.

[IMPORTANT]
.Management Interface
====
`quarkus.http.root-path` is only used for the main HTTP server.
If you enabled the management interface (using the `quarkus.management.enabled=true` property), you can configure the root path of the management interface using:
`quarkus.management.root-path`.
Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.
====

[[ssl]]
== Supporting secure connections with SSL

Expand Down Expand Up @@ -261,7 +271,7 @@ include::{generated-dir}/config/quarkus-vertx-http-config-group-filter-config.ad
== Support 100-Continue in vert.x

In order to support `100-continue`, the `quarkus.http.handle-100-continue-automatically` option needs to be enabled explicitly
For additional information check https://datatracker.ietf.org/doc/html/rfc7231#section-5.1.1[100-continue= and the related
For additional information check https://datatracker.ietf.org/doc/html/rfc7231#section-5.1.1[100-continue] and the related
https://vertx.io/docs/apidocs/io/vertx/core/http/HttpServerOptions.html#DEFAULT_HANDLE_100_CONTINE_AUTOMATICALLY[Vert.x documentation].

[source,properties]
Expand Down
180 changes: 180 additions & 0 deletions docs/src/main/asciidoc/management-interface-reference.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
////
This guide is maintained in the main Quarkus repository
and pull requests should be submitted there:
https://github.com/quarkusio/quarkus/tree/main/docs/src/main/asciidoc
////
= Management interface reference
include::_attributes.adoc[]
:categories: observability
:summary: Management interface configuration
:numbered:
:sectnums:
:sectnumlevels: 4

By default, Quarkus exposes the _management_ endpoints under `/q` on the main HTTP server.
The same HTTP server provides the application endpoints and the management endpoints.

This document presents how you can use a separate HTTP server (bound to a different network interface and port) for the management endpoints.
It avoids exposing these endpoints on the main server and, therefore, prevents undesired accesses.

== Enabling the management interface

To enable the management interface, use the following **build-time** property:

[source, properties]
----
quarkus.management.enabled=true
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@cescoffier for clarification — without this option, everything works as it now, and served on localhost:8080/q (including, say, arc and Dev UI); and without this option it is served on ${quarkus.management.host}:${q.m.port}${q.m.root-path}, by default localhost:9000/q ? In that case, quarkus.management.enabled is probably not the best name, since(from user's perspective), we do not enable something but merely moving it to a separate address.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What property name would you propose?

For the framework it enables the second server.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe something like management.server.enabled? Or management.separate-server?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, after local testing it looks like q/arc and q/dev are always on the main server, is this correct?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we don't expose all /q, just the management endpoints.

----

By default, management endpoints will be exposed on: `http://0.0.0.0:9000/q`.
For example, `http://0.0.0.0:9000/q/health/ready` for the readiness probe.

SmallRye Health Checks, SmallRye Metrics, and Micrometer endpoints will be declared as management endpoints when the management interface is enabled.

== Configure the host, port and scheme

By default, the management interface is exposed on the interface: `0.0.0.0` (all interfaces) and on the port `9000` (`9001` in test mode).
It does not use TLS (`https`) by default.

You can configure the host, ports, and TLS certificates using the following properties:

* `quarkus.management.host` - the interface / host
* `quarkus.management.port` - the port
* `quarkus.management.test-port` - the port to use in test mode
* `quarkus.management.ssl` - the TLS configuration, xref:http-reference#ssl[same as for the main HTTP server].

Here is a configuration example exposing the management interface on _https://localhost:9002_:

[source, properties]
----
quarkus.management.enabled=true
quarkus.management.host=localhost
quarkus.management.port=9002
quarkus.management.ssl.certificate.key-store-file=server-keystore.jks
quarkus.management.ssl.certificate.key-store-password=secret
----

IMPORTANT: Unlike the main HTTP server, the management interface does not handle _http_ and _https_ at the same time.
If _https_ is configured, plain HTTP requests will be rejected.

== Configure the root path

cescoffier marked this conversation as resolved.
Show resolved Hide resolved
Management endpoints are configured differently than standard HTTP endpoints.
They use a unique root path, which is `/q` by default.
This management root path can be configured using the `quarkus.management.root-path property`.
For example, if you want to expose the management endpoints under `/management` use:

[source, properties]
----
quarkus.management.enabled=true
quarkus.management.root-path=/management
----

The mounting rules of the management endpoints slightly differ from the ones used when using the main HTTP server:

* Management endpoints configured using a _relative_ path (not starting with `/`) will be served from the configured root path.
For example, if the endpoint path is `health` and the root path is `management`, the resulting path is `/management/health`
* Management endpoints configured using an _absolute_ path (starting with `/`) will be served from the root.
For example, if the endpoint path is `/health`, the resulting path is `/health`, regardless of the root path
* The management interface does not use the HTTP root path from the main HTTP server.

[IMPORTANT]
====
The `quarkus.http.root-path` property is only applied to the main HTTP server and not to the management interface.
In addition, the `quarkus.http.non-application-root-path` property is not used for endpoint exposed on the management interface.
====

== Create a management endpoint

SmallRye Health Checks, SmallRye Metrics, and Micrometer endpoints will be declared as management endpoints when the management interface is enabled.

NOTE: if you do not enable the management interface, these endpoints will be served using the main HTTP server (under `/q` by default).

Extensions can create a management endpoint by defining a _non application_ route and calling `management()` method:

[source, java]
----
@BuildStep
void createManagementRoute(BuildProducer<RouteBuildItem> routes,
NonApplicationRootPathBuildItem nonApplicationRootPathBuildItem,
MyRecorder recorder) {
routes.produce(nonApplicationRootPathBuildItem.routeBuilder()
.management() // Must be called BEFORE the routeFunction method
.routeFunction("my-path", recorder.route())
.handler(recorder.getHandler())
.blockingRoute()
.build());
//...
}
----

If the management interface is enabled, the endpoint will be exposed on: `http://0.0.0.0:9000/q/my-path`.
Otherwise, it will be exposed on: `http://localhost:8080/q/my-path`.

IMPORTANT: Management endpoints can only be declared by extensions and not from the application code.

== Management Interface Configuration

include::{generated-dir}/config/quarkus-management-management-management-interface-build-time-config.adoc[leveloffset=+1, opts=optional]

include::{generated-dir}/config/quarkus-management-management-management-interface-configuration.adoc[leveloffset=+1, opts=optional]


[[reverse-proxy]]
== Running behind a reverse proxy


Quarkus can be accessed through proxies that generate headers (e.g. `X-Forwarded-Host`) to preserve information about the original request.
Quarkus can be configured to automatically update information like protocol, host, port and URI to use the values from those headers.

IMPORTANT: Activating this feature can expose the server to security issues like information spoofing.
Activate it only when running behind a reverse proxy.

To set up this feature for the management interface, include the following lines in `src/main/resources/application.properties`:
[source,properties]
----
quarkus.management.proxy.proxy-address-forwarding=true
----

To constrain this behavior to the standard `Forwarded` header (and ignore `X-Forwarded` variants) by setting `quarkus.management.proxy.allow-forwarded` in `src/main/resources/application.properties`:
[source,properties]
----
quarkus.management.proxy.allow-forwarded=true
----

Alternatively, you can prefer `X-Forwarded-*` headers using the following configuration in `src/main/resources/application.properties` (note `allow-x-forwarded` instead of `allow-forwarded`):
[source,properties]
----
quarkus.management.proxy.proxy-address-forwarding=true
quarkus.management.proxy.allow-x-forwarded=true
quarkus.management.proxy.enable-forwarded-host=true
quarkus.management.proxy.enable-forwarded-prefix=true
----

Supported forwarding address headers are:

* `Forwarded`
* `X-Forwarded-Proto`
* `X-Forwarded-Host`
* `X-Forwarded-Port`
* `X-Forwarded-Ssl`
* `X-Forwarded-Prefix`

cescoffier marked this conversation as resolved.
Show resolved Hide resolved
If both header variants (`Forwarded` and `X-Forwarded-*`) are enabled, the `Forwarded` header will have precedence.

IMPORTANT: Using both `Forwarded` and `X-Forwarded` headers can have security implications as it may allow clients to forge requests with a header that is not overwritten by the proxy.

Ensure that your proxy is configured to strip unexpected `Forwarded` or `X-Forwarded-*` headers from the client request.

== Kubernetes

When Quarkus generates the Kubernetes metadata, it checks if the management interface is enabled and configures the probes accordingly.
The resulting descriptor defines the main HTTP port (named `http`) and the management port (named `management`).
Health probes (using HTTP actions) and Prometheus scrape URLs are configured using the `management` port.

[IMPORTANT]
.KNative
====
Until https://github.com/knative/serving/issues/8471[KNative#8471] is resolved, you cannot use the management interface, as KNative does not support containers will multiple exposed ports.
====
30 changes: 30 additions & 0 deletions docs/src/main/asciidoc/micrometer.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,15 @@ When should you use a Gauge? Only if you can't use something else. Never gauge s
less straight-forward to use than counters. If what you are measuring can be counted (because the value always
increments), use a counter instead.

[NOTE]
.Management interface
====
By default, the metrics are exposed on the main HTTP server.
You can expose them on a separate network interface and port by enabling the management interface with the
`quarkus.management.enabled=true` property.
Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.
====

=== Counters

Counters are used to measure values that only increase. In the example below, you will count the number of times you
Expand Down Expand Up @@ -598,6 +607,27 @@ implementation("org.eclipse.microprofile.metrics:microprofile-metrics-api")

NOTE: The MP Metrics API compatibility layer will be moved to a different extension in the future.

== Management interface

By default, the metrics are exposed on the main HTTP server.
You can expose them on a separate network interface and port by setting `quarkus.management.enabled=true` in your application configuration.
Note that this property is a build-time property.
The value cannot be overridden at runtime.

If you enable the management interface without customizing the management network interface and port, the metrics are exposed under: `http://0.0.0.0:9000/q/metrics`.
You can configure the path of each exposed format using:
[source, properties]
----
quarkus.micrometer.export.json.enabled=true # Enable json metrics
quarkus.micrometer.export.json.path=metrics/json
quarkus.micrometer.export.prometheus.path=metrics/prometheus
----

With such a configuration, the json metrics will be available from `http://0.0.0.0:9000/q/metrics/json`.
The prometheus metrics will be available from `http://0.0.0.0:9000/q/metrics/prometheus`.

Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.

== Configuration Reference

include::{generated-dir}/config/quarkus-micrometer.adoc[opts=optional, leveloffset=+1]
21 changes: 21 additions & 0 deletions docs/src/main/asciidoc/smallrye-health.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,15 @@ The general `status` of the health check is computed as a logical AND of all the
declared health check procedures. The `checks` array is empty as we have not specified
any health check procedure yet so let's define some.

[NOTE]
.Management interface
====
By default, the health checks are exposed on the main HTTP server.
You can expose them on a separate network interface and port by enabling the management interface with the
`quarkus.management.enabled=true` property.
Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.
====

== Creating your first health check

In this section, we create our first simple health check procedure.
Expand Down Expand Up @@ -410,6 +419,18 @@ The Quarkus `smallrye-health` extension ships with `health-ui` and enables it by

image:health-ui-screenshot01.png[alt=Health UI]

== Management interface

By default, the health checks are exposed on the main HTTP server.
You can expose them on a separate network interface and port by setting `quarkus.management.enabled=true` in your application configuration.
Note that this property is a build-time property.
The value cannot be overridden at runtime.

If you enable the management interface without customizing the management network interface and port, the health checks are exposed under: `http://0.0.0.0:9000/q/health`.
You can configure the _path_ (the `health` segment in the previous URL) using the `quarkus.smallrye-health.root-path` property.

Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.

== Conclusion

SmallRye Health provides a way for your application to distribute information
Expand Down
9 changes: 9 additions & 0 deletions docs/src/main/asciidoc/smallrye-metrics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -228,6 +228,15 @@ You will receive a response such as:

NOTE: If you prefer an OpenMetrics export rather than the JSON format, remove the `-H"Accept: application/json"` argument from your command line.

[NOTE]
.Management interface
====
By default, the metrics are exposed on the main HTTP server.
You can expose them on a separate network interface and port by enabling the management interface with the
`quarkus.management.enabled=true` property.
Refer to the xref:./management-interface-reference.adoc[management interface reference] for more information.
====

.Configuration Reference

include::{generated-dir}/config/quarkus-smallrye-metrics.adoc[opts=optional, leveloffset=+1]
Original file line number Diff line number Diff line change
Expand Up @@ -131,4 +131,4 @@ public void postBuild(ContainerImageInfoBuildItem image, List<ContainerImageBuil
//So, we now always perform this step
ExecUtil.exec("kind", "load", "docker-image", image.getImage());
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -118,4 +118,4 @@ public List<DecoratorBuildItem> createDecorators(ApplicationInfoBuildItem applic
startupPath,
roles, roleBindings, customProjectRoot);
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
/**
* A build item for selecting which port to use for probes using an {@literal HTTP get} action.
*/
public class KubernetesProbePortNameBuildItem extends SimpleBuildItem {
public final class KubernetesProbePortNameBuildItem extends SimpleBuildItem {

private final String name;

Expand Down
4 changes: 4 additions & 0 deletions extensions/kubernetes/vanilla/deployment/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kubernetes-spi</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-vertx-http-deployment-spi</artifactId>
</dependency>
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-kubernetes-client-internal-deployment</artifactId>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -159,4 +159,4 @@ private static int getStablePortNumberInRange(String input, int min, int max) {
throw new RuntimeException("Unable to generate stable port number from input string: '" + input + "'", e);
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -367,4 +367,4 @@ private static List<DecoratorBuildItem> createVolumeDecorators(Optional<Project>
});
return result;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -809,4 +809,4 @@ private static Map<String, Integer> verifyPorts(List<KubernetesPortBuildItem> ku
}
return result;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -372,4 +372,4 @@ void externalizeInitTasks(
decorators);
}
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -286,4 +286,4 @@ void externalizeInitTasks(
decorators);
}
}
}
}
Loading