Filter and Customized the metrics exported

[alvgarvilla]

Is your feature request related to a problem? Please describe.
Monitoring platforms charge for the amount of metrics you generate (i.e. Grafana Cloud). Being able to selectively decide which metrics to export is becoming an important feature if you want to reduce your bill with the monitoring SaaS providers.

We are migrating our Spring Boot services from Micrometer with Prometheus exporter to OpenTelemetry Agent. These are some features that we are missing which would be quite useful:

• Metrics filtering (MeterFilter in Micrometer). Be able to filter metrics (exclude and include) based on names, regex expressions, tag values, etc. We are able to achieve this applying the filter at OpenTelemetry Collector level, but this feature should be included in the agent.
• Histogram / Summary. By default, "http." metrics are considered Histograms, which include all buckets for calculating the percentiles. With Micrometer you are able to specify whether a timed series should be considered an histogram or a summary, reducing the cardinality of that metric considerably. OpenTemeletry agent ignores this option when integrating with Micrometer, exposing all metrics as histograms.

Describe the solution you'd like

• Being able to filter out metrics through configuration.
• Make the agent aware of the settings define in Micrometer. i.e., use summary instead of histogram when specified.

Describe alternatives you've considered
If you use OpenTelemetry Collector, it is possible to apply those filters in the processor.
I cannot find any alternative at the moment for the histogram.
Our solution at the moment is to use the OpenTelemetry Collector to scrape from the prometheus endpoint exposed in the services, avoiding using the agent. In this case, we would have fully control of the metrics we expose. However, our end goal is using the agent.

[jack-berg]

Being able to filter out metrics through configuration.

This is possible today with views. Register a view which applies the Aggregation.drop() aggregation to the metrics you want to filter out.

    SdkMeterProvider.builder()
        .registerView(
            InstrumentSelector.builder().setName("my-instrument-name").build(),
            View.builder().setAggregation(Aggregation.drop()).build());

Make the agent aware of the settings define in Micrometer. i.e., use summary instead of histogram when specified.

Also possible with views (depending on your definition of summary). Register a view which configures the histogram aggregation to have no boundaries. This produces a "single bucket" histogram, which effectively works as a summary with sum, count, min, max fields.

    SdkMeterProvider.builder()
        .registerView(
            InstrumentSelector.builder().setName("my-instrument-name").build(),
            View.builder()
                .setAggregation(Aggregation.explicitBucketHistogram(Collections.emptyList()))
                .build());

[devurandom]

Monitoring platforms charge for the amount of metrics you generate

To us the "data points per minute" (DPM, cf. https://grafana.com/docs/grafana-cloud/billing-and-usage/active-series-and-dpm/#data-points-per-minute-dpm) are contributing more significantly to our Grafana Cloud costs than the "active series". We would like to keep the generated metric data points below the DPM included in our Grafana Cloud plan.

We gather metrics using the OpenTelemetry Java Agent, batch them using the OpenTelemetry Collector and export them to Grafana Cloud. If it was possible to aggregate the metrics generated by the OpenTelemetry Java Agent for each time series over an interval of 60s, or to reduce the sampling interval to 60s, that would be a great help.

Related: open-telemetry/opentelemetry-collector-contrib#4968

[jack-berg]

The default aggregation window for metrics is 60s and it's configurable. Not sure I understand the ask @devurandom.

[devurandom]

The default aggregation window for metrics is 60s and it's configurable. Not sure I understand the ask @devurandom.

You mean OTEL_METRIC_EXPORT_INTERVAL (https://github.com/open-telemetry/opentelemetry-java/tree/main/sdk-extensions/autoconfigure#periodic-metric-reader)? That controls how often the Java agent reports metrics for the service it instruments to its upstream collector? And it will never report more than one data point per series within one interval?

I am looking at the Grafana billing dashboard (https://grafana.com/docs/grafana-cloud/billing-and-usage/optimize-scrape-interval/#identify-high-dpm-per-series) and seemingly independent of the interval I set the maximum DPM is still well beyond 1.

[github-actions]

This has been automatically marked as stale because it has been marked as needing author feedback and has not had any activity for 7 days. It will be closed if no further activity occurs within 7 days of this comment.