Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Collectd receiver is not processing meta #30036

Closed
shubhamkr619 opened this issue Dec 18, 2023 · 6 comments
Closed

Collectd receiver is not processing meta #30036

shubhamkr619 opened this issue Dec 18, 2023 · 6 comments

Comments

@shubhamkr619
Copy link

Component(s)

receiver/collectd

What happened?

Description

Collectd receiver is not forwarding the meta payload for processing. This is a issue where duplicate timeseries is not allowed, such as googlemanagedprometheus exporter.

Steps to Reproduce

Otle config

extensions:
  memory_ballast:
    size_mib: 512
  zpages:
    endpoint: localhost:55679

receivers:
  collectd:
  collectd/one:
    endpoint: "localhost:12345"
    timeout: "50s"

  # Receiver will look for query params that are prefixed with this value
  # and add them as attributes to all the metrics supplied by the request.
  # For example, if the receiver receives a request at `/post-collectd?dap_k=v`
  # and attributes_prefix is set to "dap"_, then all metrics supplied by the
  # request will have an attribute with key `k` and value `v`.
    attributes_prefix: "_"

  # Which encoding format should the receiver try to decode the request with.
  # Receiver only supports JSON. This options only exists to make keep things
  # explicit and as a placeholder for any formats added in future.
    encoding: "json"
  #  statsd:
  #  statsd/receiver_settings:
  #    endpoint: "127.0.0.1:9999"
  #    aggregation_interval: 60s
  #    enable_metric_type: false
  #    timer_histogram_mapping:
  #      - statsd_type: "histogram"
  #        observer_type: "gauge"
  #      - statsd_type: "timing"
  #        observer_type: "histogram"
  #        histogram:
  #          max_size: 170
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:9190
        cors:
          allowed_origins:
            - '*'
            # Origins can have wildcards with *, use * by itself to match any origin.
          max_age: 7200

processors:
  resource:
    attributes:
    - key: "location"
      value: "us-central1-a"
      action: upsert
  resourcedetection:
    detectors: ["gcp"]
  batch:
    send_batch_size: 10000
    send_batch_max_size: 11000
    timeout: 10s
  memory_limiter:
    # 75% of maximum memory up to 2G
    limit_mib: 1536
    # 25% of limit up to 2G
    spike_limit_mib: 512
    check_interval: 5s
  metricstransform:
    transforms:
      - include: ^spark.(driver|executor)(.*)$
        match_type: regexp
        action: update
        operations:
          - action: add_label
            new_label: app_name
            new_value: app_name
          - action: add_label
            new_label: app_id
            new_value: app_id
  filter/spark_driver_or_executor_metrics:
    metrics:
      include:
        match_type: regexp
        metric_names:
        - ^spark.(driver|executor)(.*)$
  transform/p1:
    error_mode: ignore
    metric_statements:
    - context: datapoint
      statements:
        - set(attributes["app_name"], Split(metric.name, "spark.driver.app_name##"))
        - set(attributes["app_id"], Split(metric.name, ".app_id##"))
    - context: datapoint
      statements:
        - set(attributes["app_name"], attributes["app_name"][1])
        - set(attributes["app_id"], attributes["app_id"][1])
    - context: datapoint
      statements:
        - set(attributes["app_name"], Split(attributes["app_name"], ".app_id##"))
        - set(attributes["app_id"], Split(attributes["app_id"], ".driver."))
    - context: datapoint
      statements:
        - set(attributes["app_name"], attributes["app_name"][0])
        - set(attributes["app_id"], attributes["app_id"][0])
  transform/p2:
    error_mode: ignore
    metric_statements:
    - context: metric
      statements:
      - replace_pattern(name, "^spark.driver.app_name.(.*).driver.", "replace")
      - replace_pattern(name, "\\.", "_")
      - replace_pattern(name, "\\/", "_")
      - set(name, Concat(["driver", name], "/"))


exporters:
  googlemanagedprometheus:

  logging:
  file:
    path: /tmp/otelfile.log


service:
  extensions: [memory_ballast, zpages]
  pipelines:
    metrics:
      receivers: [otlp, collectd, collectd/one]
      processors: [batch, resourcedetection, transform/p2]
      exporters: [file,googlemanagedprometheus, logging]
  telemetry:
    logs:
      level: "debug"
      output_paths: ["/tmp/otle.log"]
curl -X POST -d @json.json http://localhost:12345/v1/metrics -v 
[
  {
    "values": [
      0
    ],
    "dstypes": [
      "gauge"
    ],
    "dsnames": [
      "value"
    ],
    "time": 1702904313.119,
    "interval": 60,
    "host": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal",
    "plugin": "statsd",
    "plugin_instance": "spark",
    "type": "gauge",
    "type_instance": "BlockManager/disk/diskSpaceUsed_MB",
    "meta": {
      "_id": "application_1701071617570_0156",
      "_app_name": "hello-world.py",
      "label:app_id": "application_1701071617570_0156",
      "label:app_name": "hello-world.py",
      "stackdriver_metric_type": "dataproc.googleapis.com/spark/driver/BlockManager/disk/diskSpaceUsed_MB"
    }
  },
  {
    "values": [
      0
    ],
    "dstypes": [
      "gauge"
    ],
    "dsnames": [
      "value"
    ],
    "time": 1702904313.119,
    "interval": 60,
    "host": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal",
    "plugin": "statsd",
    "plugin_instance": "spark",
    "type": "gauge",
    "type_instance": "BlockManager/memory/maxMem_MB",
    "meta": {
      "_id": "application_1701071617570_0156",
      "_app_name": "hello-world.py",
      "label:app_id": "application_1701071617570_0156",
      "label:app_name": "hello-world.py",
      "stackdriver_metric_type": "dataproc.googleapis.com/spark/driver/BlockManager/memory/maxMem_MB"
    }
  },
  {
    "values": [
      0
    ],
    "dstypes": [
      "gauge"
    ],
    "dsnames": [
      "value"
    ],
    "time": 1702904313.119,
    "interval": 60,
    "host": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal",
    "plugin": "statsd",
    "plugin_instance": "spark",
    "type": "gauge",
    "type_instance": "BlockManager/memory/memUsed_MB",
    "meta": {
      "_id": "application_1701071617570_0156",
      "_app_name": "hello-world.py",
      "label:app_id": "application_1701071617570_0156",
      "label:app_name": "hello-world.py",
      "stackdriver_metric_type": "dataproc.googleapis.com/spark/driver/BlockManager/memory/memUsed_MB"
    }
  },
  {
    "values": [
      1
    ],
    "dstypes": [
      "gauge"
    ],
    "dsnames": [
      "value"
    ],
    "time": 1702904313.119,
    "interval": 60,
    "host": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal",
    "plugin": "statsd",
    "plugin_instance": "spark",
    "type": "gauge",
    "type_instance": "DAGScheduler/job/allJobs",
    "meta": {
      "_id": "application_1701071617570_0156",
      "_app_name": "hello-world.py",
      "label:app_id": "application_1701071617570_0156",
      "label:app_name": "hello-world.py",
      "stackdriver_metric_type": "dataproc.googleapis.com/spark/driver/DAGScheduler/job/allJobs"
    }
  },
  {
    "values": [
      0
    ],
    "dstypes": [
      "gauge"
    ],
    "dsnames": [
      "value"
    ],
    "time": 1702904313.119,
    "interval": 60,
    "host": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal",
    "plugin": "statsd",
    "plugin_instance": "spark",
    "type": "gauge",
    "type_instance": "DAGScheduler/stage/waitingStages",
    "meta": {
      "_id": "application_1701071617570_0156",
      "_app_name": "hello-world.py",
      "label:app_id": "application_1701071617570_0156",
      "label:app_name": "hello-world.py",
      "stackdriver_metric_type": "dataproc.googleapis.com/spark/driver/DAGScheduler/stage/waitingStages"
    }
  },
  {
    "values": [
      0
    ],
    "dstypes": [
      "gauge"
    ],
    "dsnames": [
      "value"
    ],
    "time": 1702904313.119,
    "interval": 60,
    "host": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal",
    "plugin": "statsd",
    "plugin_instance": "spark",
    "type": "gauge",
    "type_instance": "DAGScheduler/stage/failedStages",
    "meta": {
      "_id": "application_1701071617570_0156",
      "_app_name": "hello-world.py",
      "label:app_id": "application_1701071617570_0156",
      "label:app_name": "hello-world.py",
      "stackdriver_metric_type": "dataproc.googleapis.com/spark/driver/DAGScheduler/stage/failedStages"
    }
  }
]

Expected Result

It should get process and otel should add meta fields as the labels.

Actual Result

Meta fields are dropped post processing causing the issue of duplicate timeseries in googlemanagedprometheus exporter.

{
  "resourceMetrics": [
    {
      "resource": {
        "attributes": [
          {
            "key": "cloud.provider",
            "value": {
              "stringValue": "gcp"
            }
          },
          {
            "key": "cloud.account.id",
            "value": {
              "stringValue": "skshubham-01"
            }
          },
          {
            "key": "cloud.platform",
            "value": {
              "stringValue": "gcp_compute_engine"
            }
          },
          {
            "key": "cloud.availability_zone",
            "value": {
              "stringValue": "us-central1-f"
            }
          },
          {
            "key": "cloud.region",
            "value": {
              "stringValue": "us-central1"
            }
          },
          {
            "key": "host.type",
            "value": {
              "stringValue": "projects/702789716705/machineTypes/n2-standard-4"
            }
          },
          {
            "key": "host.id",
            "value": {
              "stringValue": "6049932627543716908"
            }
          },
          {
            "key": "host.name",
            "value": {
              "stringValue": "cluster-3a83-m"
            }
          }
        ]
      },
      "scopeMetrics": [
        {
          "scope": {},
          "metrics": [
            {
              "name": "driver/gauge_BlockManager_memory_maxMem_MB",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      },
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "0"
                  }
                ]
              }
            },
            {
              "name": "driver/gauge_BlockManager_memory_memUsed_MB",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      },
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "0"
                  }
                ]
              }
            },
            {
              "name": "driver/gauge_DAGScheduler_stage_failedStages",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      },
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "0"
                  }
                ]
              }
            },
            {
              "name": "driver/gauge_DAGScheduler_stage_waitingStages",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      },
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "0"
                  }
                ]
              }
            },
            {
              "name": "driver/gauge_BlockManager_disk_diskSpaceUsed_MB",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      },
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "0"
                  }
                ]
              }
            },
            {
              "name": "driver/gauge_DAGScheduler_job_allJobs",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      },
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "1"
                  }
                ]
              }
            },
            {
              "name": "driver/gauge_BlockManager_disk_diskSpaceUsed_MB",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      },
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "0"
                  }
                ]
              }
            }
          ]
        }
      ],
      "schemaUrl": "https://opentelemetry.io/schemas/1.6.1"
    },
    {
      "resource": {
        "attributes": [
          {
            "key": "cloud.provider",
            "value": {
              "stringValue": "gcp"
            }
          },
          {
            "key": "cloud.account.id",
            "value": {
              "stringValue": "skshubham-01"
            }
          },
          {
            "key": "cloud.platform",
            "value": {
              "stringValue": "gcp_compute_engine"
            }
          },
          {
            "key": "cloud.availability_zone",
            "value": {
              "stringValue": "us-central1-f"
            }
          },
          {
            "key": "cloud.region",
            "value": {
              "stringValue": "us-central1"
            }
          },
          {
            "key": "host.type",
            "value": {
              "stringValue": "projects/702789716705/machineTypes/n2-standard-4"
            }
          },
          {
            "key": "host.id",
            "value": {
              "stringValue": "6049932627543716908"
            }
          },
          {
            "key": "host.name",
            "value": {
              "stringValue": "cluster-3a83-m"
            }
          }
        ]
      },
      "scopeMetrics": [
        {
          "scope": {},
          "metrics": [
            {
              "name": "driver/gauge_BlockManager_memory_maxMem_MB",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      },
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "0"
                  }
                ]
              }
            },
            {
              "name": "driver/gauge_BlockManager_memory_memUsed_MB",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      },
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "0"
                  }
                ]
              }
            },
            {
              "name": "driver/gauge_DAGScheduler_job_allJobs",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      },
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "1"
                  }
                ]
              }
            },
            {
              "name": "driver/gauge_DAGScheduler_stage_failedStages",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      },
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "0"
                  }
                ]
              }
            },
            {
              "name": "driver/gauge_DAGScheduler_stage_waitingStages",
              "gauge": {
                "dataPoints": [
                  {
                    "attributes": [
                      {
                        "key": "plugin",
                        "value": {
                          "stringValue": "statsd"
                        }
                      },
                      {
                        "key": "plugin_instance",
                        "value": {
                          "stringValue": "spark"
                        }
                      },
                      {
                        "key": "host",
                        "value": {
                          "stringValue": "cluster-3a83-m.us-central1-f.c.skshubham-01.internal"
                        }
                      },
                      {
                        "key": "dsname",
                        "value": {
                          "stringValue": "value"
                        }
                      }
                    ],
                    "timeUnixNano": "1702905993119000064",
                    "asInt": "0"
                  }
                ]
              }
            }
          ]
        }
      ],
      "schemaUrl": "https://opentelemetry.io/schemas/1.6.1"
    }
  ]
}

Collector version

0.80.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04")
Compiler(if manually compiled): (e.g., "go 14.2")

root@cluster-3a83-m:/etc/otelcol-contrib# cat /etc/*release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
NAME="Debian GNU/Linux"
VERSION_ID="11"
VERSION="11 (bullseye)"
VERSION_CODENAME=bullseye
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"
root@cluster-3a83-m:/etc/otelcol-contrib#

OpenTelemetry Collector configuration

extensions:
  memory_ballast:
    size_mib: 512
  zpages:
    endpoint: localhost:55679

receivers:
  collectd:
  collectd/one:
    endpoint: "localhost:12345"
    timeout: "50s"

  # Receiver will look for query params that are prefixed with this value
  # and add them as attributes to all the metrics supplied by the request.
  # For example, if the receiver receives a request at `/post-collectd?dap_k=v`
  # and attributes_prefix is set to "dap"_, then all metrics supplied by the
  # request will have an attribute with key `k` and value `v`.
    attributes_prefix: "_"

  # Which encoding format should the receiver try to decode the request with.
  # Receiver only supports JSON. This options only exists to make keep things
  # explicit and as a placeholder for any formats added in future.
    encoding: "json"
  #  statsd:
  #  statsd/receiver_settings:
  #    endpoint: "127.0.0.1:9999"
  #    aggregation_interval: 60s
  #    enable_metric_type: false
  #    timer_histogram_mapping:
  #      - statsd_type: "histogram"
  #        observer_type: "gauge"
  #      - statsd_type: "timing"
  #        observer_type: "histogram"
  #        histogram:
  #          max_size: 170
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:9190
        cors:
          allowed_origins:
            - '*'
            # Origins can have wildcards with *, use * by itself to match any origin.
          max_age: 7200

processors:
  resource:
    attributes:
    - key: "location"
      value: "us-central1-a"
      action: upsert
  resourcedetection:
    detectors: ["gcp"]
  batch:
    send_batch_size: 10000
    send_batch_max_size: 11000
    timeout: 10s
  memory_limiter:
    # 75% of maximum memory up to 2G
    limit_mib: 1536
    # 25% of limit up to 2G
    spike_limit_mib: 512
    check_interval: 5s
  metricstransform:
    transforms:
      - include: ^spark.(driver|executor)(.*)$
        match_type: regexp
        action: update
        operations:
          - action: add_label
            new_label: app_name
            new_value: app_name
          - action: add_label
            new_label: app_id
            new_value: app_id
  filter/spark_driver_or_executor_metrics:
    metrics:
      include:
        match_type: regexp
        metric_names:
        - ^spark.(driver|executor)(.*)$
  transform/p1:
    error_mode: ignore
    metric_statements:
    - context: datapoint
      statements:
        - set(attributes["app_name"], Split(metric.name, "spark.driver.app_name##"))
        - set(attributes["app_id"], Split(metric.name, ".app_id##"))
    - context: datapoint
      statements:
        - set(attributes["app_name"], attributes["app_name"][1])
        - set(attributes["app_id"], attributes["app_id"][1])
    - context: datapoint
      statements:
        - set(attributes["app_name"], Split(attributes["app_name"], ".app_id##"))
        - set(attributes["app_id"], Split(attributes["app_id"], ".driver."))
    - context: datapoint
      statements:
        - set(attributes["app_name"], attributes["app_name"][0])
        - set(attributes["app_id"], attributes["app_id"][0])
  transform/p2:
    error_mode: ignore
    metric_statements:
    - context: metric
      statements:
      - replace_pattern(name, "^spark.driver.app_name.(.*).driver.", "replace")
      - replace_pattern(name, "\\.", "_")
      - replace_pattern(name, "\\/", "_")
      - set(name, Concat(["driver", name], "/"))


exporters:
  googlemanagedprometheus:

  logging:
  file:
    path: /tmp/otelfile.log


service:
  extensions: [memory_ballast, zpages]
  pipelines:
    metrics:
      receivers: [otlp, collectd, collectd/one]
      processors: [batch, resourcedetection, transform/p2]
      exporters: [file,googlemanagedprometheus, logging]
  telemetry:
    logs:
      level: "debug"
      output_paths: ["/tmp/otle.log"]

Log output

No response

Additional context

No response

@shubhamkr619 shubhamkr619 added bug Something isn't working needs triage New item requiring triage labels Dec 18, 2023
Copy link
Contributor

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@shubhamkr619
Copy link
Author

any update on this?

@atoulme
Copy link
Contributor

atoulme commented Jan 2, 2024

Not sure how to exactly help you. Do you want to open a failing test PR with your exact requirements?

@atoulme atoulme removed the needs triage New item requiring triage label Jan 2, 2024
@shubhamkr619
Copy link
Author

@atoulme , as you can see from the above payload meta attribute is pushed from the collectd http plugin, however collectd receiver plugin the dropping that information. Without that information data pushed via googlemanagedprometheus exporter shall fail stating the error duplicate timeseries as those fileds are essential to be kept into scraped metrics. Let me know this helps, i can add more detail if required.

Arguably, another fix would simply be that meta attributes are pushed into the scraped metrics so make each entry uniq, further avoiding the issue of duplicate metrics.

Copy link
Contributor

github-actions bot commented Mar 4, 2024

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

See Adding Labels via Comments if you do not have permissions to add labels yourself.

@github-actions github-actions bot added the Stale label Mar 4, 2024
Copy link
Contributor

github-actions bot commented May 3, 2024

This issue has been closed as inactive because it has been stale for 120 days with no activity.

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale May 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants