Metric Name with multiple consecutive underscores has its underscores reduced to one thus resulting in change of name of the metric.

[pratik2294]

Component(s)

exporter/prometheus

What happened?

Description

Using Open Telemetry Java sdk, when i create a metric with name of format myprefix___l1_l2_actualName, then otel collector displays the metric on its Prometheus exporter as myprefix_l1_l2_actualName. (Underscores are reduced)
myprefix___l1_l2_actualName is a valid metric name according to Prometheus Data Model. myprefix___l1_l2_actualName does match the provided regex [a-zA-Z_:][a-zA-Z0-9_:]* according to prometheus data model. This can be verified with Regex 101 with PCRE PHP <7.3. Also this metric doesn't start with "__" (double underscores) which are reserved for internal use as per Prometheus' Metric names and labels guidelines. Hence it is a valid metric name.

If I use File exporter instead of Prometheus exporter then the expected name of the metric is observed in the file.
See the Otel configuration provided in the issue.

Steps to Reproduce Using Otel Java SDK.

import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.api.metrics.Meter;
import io.opentelemetry.api.metrics.ObservableDoubleGauge;
import io.opentelemetry.sdk.autoconfigure.AutoConfiguredOpenTelemetrySdk;
import io.opentelemetry.sdk.metrics.SdkMeterProvider;
import io.opentelemetry.sdk.metrics.export.PeriodicMetricReader;
import io.opentelemetry.exporter.otlp.metrics.OtlpGrpcMetricExporter;

public class GaugeExample {
    public static void main(String[] args) {
        // Initialize OpenTelemetry SDK with OTLP exporter
        OpenTelemetry openTelemetry = AutoConfiguredOpenTelemetrySdk.initialize().getOpenTelemetrySdk();

        // Configure OTLP exporter
        OtlpGrpcMetricExporter metricExporter = OtlpGrpcMetricExporter.builder()
            .setEndpoint("http://localhost:4317") // Replace with your OTLP endpoint
            .build();

        // Set up the metric reader with the exporter
        SdkMeterProvider meterProvider = SdkMeterProvider.builder()
            .registerMetricReader(PeriodicMetricReader.builder(metricExporter).build())
            .build();

        // Get a meter
        Meter meter = meterProvider.get("io.opentelemetry.example.metrics");

        // Create a gauge with the specified metric name
        ObservableDoubleGauge gauge = meter.gaugeBuilder("myprefix___l1_l2_actualName")
            .setDescription("This is a gauge for myprefix___l1_l2_actualName")
            .setUnit("1")
            .buildWithCallback(result -> {
                // Record the gauge value
                double value = getCurrentValue();
                result.record(value);
            });
    }

    // Method to get the current value for the gauge
    private static double getCurrentValue() {
        // Replace with your logic to get the actual value
        return Math.random() * 100;
    }
}

Expected Result

Metric with name myprefix___l1_l2_actualName should be available on collector's 8089/Metrics endpoint. (Maintain all the given underscores).

Actual Result

Metric with name myprefix_l1_l2_actualName is be available on collector's 8089/Metrics endpoint.

Collector version

v0.104.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")
I was using docker image "otel/opentelemetry-collector-contrib:0.104.0-amd64"

OpenTelemetry Collector configuration

receivers:  
  otlp:
    protocols:
      grpc:
        endpoint: :4317
exporters:
  prometheus:
    endpoint: "0.0.0.0:8089"
  file/no_rotation:
    path: /output/metrics.json
    flush_interval: 60s
    append: false

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [prometheus,file/no_rotation]

Log output

There is no log output.

Additional context

This affects backward compatibility in as the name of the metric that appears on collector's prometheus exporter (/metrics) is different from the one that was originally in use. All the existing Grafana dashboards and Prometheus alerts will be affected as the metric name is has less underscores.

[github-actions]

Pinging code owners:

• exporter/prometheus: @Aneurysm9 @dashpole

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dashpole]

Similar to open-telemetry/opentelemetry-go#5755, we should switch the prometheus exporter to use model.EscapeName, which I think should fix this.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/prometheus: @Aneurysm9 @dashpole

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[github-actions]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

• exporter/prometheus: @Aneurysm9 @dashpole @ArthurSens

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[dashpole]

cc @ywwg

I think you were going to work on switching to model.EscapeName

[ywwg]

yes, though as I looked in to it, the otel normalization algorithm does this underscore removal as part of its larger normalization process and it is not trivial to fix. I am thinking about a more general rewrite / refactor of the normalization code that could fix this, so I can take this as an assignment (which I don't have the permissions for)