Skip to content

Commit

Permalink
GEOMESA-3302 Prometheus metric support (#3002)
Browse files Browse the repository at this point in the history
  • Loading branch information
elahrvivaz authored Oct 13, 2023
1 parent 9645271 commit 2a96493
Show file tree
Hide file tree
Showing 9 changed files with 428 additions and 2 deletions.
5 changes: 5 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,11 @@ To build HTML versions of the manuals:
If you do not have Sphinx installed the manual will not be built.
The outputted files are written to the ``target/html`` directory.

To view the docs, use the provided ``docs-server.sh`` script (requires Docker), then browse to
``http://localhost:8080``:

$ ./docs/docs-server.sh

To build a PDF version:

$ mvn clean install -Pdocs,latex
Expand Down
8 changes: 8 additions & 0 deletions docs/docs-server.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#! /usr/bin/env bash

dir="$(cd "$(dirname "$0")" || exit 1; pwd)"

docker run --rm \
-p 8080:8080 \
-v "$dir/target/html:/usr/local/tomcat/webapps/ROOT" \
tomcat:9.0-jdk11
59 changes: 57 additions & 2 deletions docs/user/appendix/metrics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@ GeoMesa Metrics
GeoMesa provides integration with the `DropWizard Metrics <https://metrics.dropwizard.io/>`__ library for real-time
reporting with the ``geomesa-metrics`` module.

Reporters are available for `SLF4J <https://www.slf4j.org/>`__, `CloudWatch <https://aws.amazon.com/cloudwatch/>`__,
`Graphite <https://graphiteapp.org/>`__, and `Ganglia <https://ganglia.sourceforge.net/>`__.
Reporters are available for `CloudWatch <https://aws.amazon.com/cloudwatch/>`__,
`Prometheus <https://prometheus.io/>`__, `Graphite <https://graphiteapp.org/>`__,
`Ganglia <https://ganglia.sourceforge.net/>`__, and `SLF4J <https://www.slf4j.org/>`__.

Configuration
-------------
Expand Down Expand Up @@ -86,6 +87,60 @@ Example configuration:
zero-values = false
}

Prometheus Reporter
-------------------

The Prometheus reporter can be included by adding a dependency on
``org.locationtech.geomesa:geomesa-metrics-prometheus``. The Prometheus reporter supports normal Prometheus scraping
as well as the Prometheus Pushgateway. Note that the unit and interval configurations described above do not apply
to Prometheus reporters.

Prometheus Scraping
^^^^^^^^^^^^^^^^^^^

The standard Prometheus reporter will expose an HTTP endpoint to be scraped by Prometheus.

====================== ===============================================================================================
Configuration Property Description
====================== ===============================================================================================
``type`` Must be ``prometheus``
``port`` The port used to expose metrics
``suffix`` A suffix to append to all metric names
====================== ===============================================================================================

Example configuration:

::

{
type = "prometheus"
port = "9090"
}

Prometheus Pushgateway
^^^^^^^^^^^^^^^^^^^^^^

For short-lived jobs, metrics can be sent to a Prometheus Pushgateway instead of being exposed for scraping.

====================== ===============================================================================================
Configuration Property Description
====================== ===============================================================================================
``type`` Must be ``prometheus-pushgateway``
``gateway`` The Pushgateway host
``job-name`` The name of the batch job being run
``suffix`` A suffix to append to all metric names
====================== ===============================================================================================

Example configuration:

::

{
type = "prometheus-pushgateway"
gateway = "http://pushgateway:8080/"
job-name = "my-job"
}

Ganglia Reporter
----------------

Expand Down
52 changes: 52 additions & 0 deletions geomesa-metrics/geomesa-metrics-prometheus/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<parent>
<groupId>org.locationtech.geomesa</groupId>
<artifactId>geomesa-metrics_2.12</artifactId>
<version>4.1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>

<artifactId>geomesa-metrics-prometheus_2.12</artifactId>
<name>GeoMesa Metrics Prometheus Reporter</name>

<dependencies>
<dependency>
<groupId>org.locationtech.geomesa</groupId>
<artifactId>geomesa-metrics-core_${scala.binary.version}</artifactId>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_dropwizard</artifactId>
<version>0.16.0</version>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_httpserver</artifactId>
<version>0.16.0</version>
</dependency>
<dependency>
<groupId>io.prometheus</groupId>
<artifactId>simpleclient_pushgateway</artifactId>
<version>0.16.0</version>
</dependency>

<!-- test dependencies -->
<dependency>
<groupId>org.specs2</groupId>
<artifactId>specs2-core_${scala.binary.version}</artifactId>
</dependency>
<dependency>
<groupId>org.specs2</groupId>
<artifactId>specs2-junit_${scala.binary.version}</artifactId>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty</artifactId>
<version>6.1.26</version>
<scope>test</scope>
</dependency>
</dependencies>

</project>
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
org.locationtech.geomesa.metrics.prometheus.PrometheusReporterFactory
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
port = 9090
Original file line number Diff line number Diff line change
@@ -0,0 +1,154 @@
/***********************************************************************
* Copyright (c) 2013-2023 Commonwealth Computer Research, Inc.
* All rights reserved. This program and the accompanying materials
* are made available under the terms of the Apache License, Version 2.0
* which accompanies this distribution and is available at
* http://www.opensource.org/licenses/apache2.0.php.
***********************************************************************/

package org.locationtech.geomesa.metrics.prometheus

import com.codahale.metrics._
import com.typesafe.config.{Config, ConfigFactory}
import io.prometheus.client.Collector.MetricFamilySamples
import io.prometheus.client.CollectorRegistry
import io.prometheus.client.dropwizard.DropwizardExports
import io.prometheus.client.dropwizard.samplebuilder.{DefaultSampleBuilder, SampleBuilder}
import io.prometheus.client.exporter.{HTTPServer, PushGateway}
import org.locationtech.geomesa.metrics.core.ReporterFactory
import org.locationtech.geomesa.utils.io.CloseWithLogging
import pureconfig.{ConfigReader, ConfigSource}

import java.net.URL
import java.util.Locale
import java.util.concurrent.TimeUnit

class PrometheusReporterFactory extends ReporterFactory {

import PrometheusReporterFactory._

override def apply(
conf: Config,
registry: MetricRegistry,
rates: TimeUnit,
durations: TimeUnit): Option[ScheduledReporter] = {
val typ = if (conf.hasPath("type")) { Option(conf.getString("type")).map(_.toLowerCase(Locale.US)) } else { None }
typ match {
case Some("prometheus") =>
val config = ConfigSource.fromConfig(conf.withFallback(Defaults)).loadOrThrow[PrometheusConfig]
Some(new PrometheusReporter(registry, config.suffix, config.port))

case Some("prometheus-pushgateway") =>
val config = ConfigSource.fromConfig(conf.withFallback(Defaults)).loadOrThrow[PrometheusPushgatewayConfig]
Some(new PushgatewayReporter(registry, config.suffix, config.gateway, config.jobName))

case _ =>
None
}
}
}

object PrometheusReporterFactory {

import pureconfig.generic.semiauto._

private val Defaults = ConfigFactory.parseResourcesAnySyntax("prometheus-defaults").resolve()

case class PrometheusConfig(port: Int, suffix: Option[String])
case class PrometheusPushgatewayConfig(gateway: URL, jobName: String, suffix: Option[String])

implicit val PrometheusConfigReader: ConfigReader[PrometheusConfig] = deriveReader[PrometheusConfig]
implicit val PrometheusPushgatewayConfigReader: ConfigReader[PrometheusPushgatewayConfig] =
deriveReader[PrometheusPushgatewayConfig]

/**
* Prometheus reporter
*
* @param registry registry
* @param suffix metrics suffix
* @param port http server port, or zero to use an ephemeral open port
*/
class PrometheusReporter(registry: MetricRegistry, suffix: Option[String], port: Int)
extends BasePrometheusReporter(registry, CollectorRegistry.defaultRegistry, suffix) {

private val server = new HTTPServer.Builder().withPort(port).build()

def getPort: Int = server.getPort

override def close(): Unit = {
CloseWithLogging(server)
super.close()
}
}

/**
* Pushgateway reporter
*
* Note: we use a new collector registry, as generally with pushgateway you don't want to expose things
* from the default registry like memory, etc
*
* @param registry registry
* @param suffix metrics suffix
* @param gateway pushgateway url
* @param jobName pushgateway job name
*/
class PushgatewayReporter(registry: MetricRegistry, suffix: Option[String], gateway: URL, jobName: String)
extends BasePrometheusReporter(registry, new CollectorRegistry(), suffix) {
override def close(): Unit = {
try { new PushGateway(gateway).pushAdd(collectorRegistry, jobName) } finally {
super.close()
}
}
}

/**
* Placeholder reporter that doesn't report any metrics, but handles the lifecycle of the prometheus
* pusher or server
*
* @param registry registry
* @param collectorRegistry prometheus registry
* @param suffix metrics suffix
*/
class BasePrometheusReporter(
registry: MetricRegistry,
protected val collectorRegistry: CollectorRegistry,
suffix: Option[String]
) extends ScheduledReporter(registry, "prometheus", MetricFilter.ALL, TimeUnit.MILLISECONDS, TimeUnit.MILLISECONDS) {

private val sampler = suffix match {
case None => new DefaultSampleBuilder()
case Some(s) => new SuffixSampleBuilder(s)
}
new DropwizardExports(registry, sampler).register(collectorRegistry)

// since prometheus scrapes metrics, we don't have to report them here

override def start(initialDelay: Long, period: Long, unit: TimeUnit): Unit = {}

override def report(
gauges: java.util.SortedMap[String, Gauge[_]],
counters: java.util.SortedMap[String, Counter],
histograms: java.util.SortedMap[String, Histogram],
meters: java.util.SortedMap[String, Meter],
timers: java.util.SortedMap[String, Timer]): Unit = {}
}

/**
* Adds a suffix to each metric
*
* @param suffix suffix
*/
class SuffixSampleBuilder(suffix: String) extends SampleBuilder {

private val builder = new DefaultSampleBuilder()

override def createSample(
dropwizardName: String,
nameSuffix: String,
additionalLabelNames: java.util.List[String],
additionalLabelValues: java.util.List[String],
value: Double): MetricFamilySamples.Sample = {
builder.createSample(dropwizardName, suffix, additionalLabelNames, additionalLabelValues, value)
}
}
}
Loading

0 comments on commit 2a96493

Please sign in to comment.