Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

prometheusremotewrite doesn't honor all resource attributes #10023

Closed
newly12 opened this issue May 14, 2022 · 1 comment
Closed

prometheusremotewrite doesn't honor all resource attributes #10023

newly12 opened this issue May 14, 2022 · 1 comment

Comments

@newly12
Copy link
Contributor

newly12 commented May 14, 2022

Is your feature request related to a problem? Please describe.
The scenario is scraping cadvisor metrics, enrich them via k8sattrs processor based on certain metric labels, so eventually metrics could have additional labels e.g. deployment name, any interested labels and/or annotations.

As currently k8sattrs processor only supports pod association from resource attributes, groupbyattrs processor was used first in order to move attributes of a metric(e.g. namespace and pod) to resource attributes, this is working but currently only a few specific resource attributes are used/extracted by prometheusremotewrite exporter.

Describe the solution you'd like
Support additional resource attributes in prometheusremotewrite exporter. As label convention in prometheus is different than open telemetry, resource attribute names may need to be sanitized, or updated by a processor as proposed in #9994.

Describe alternatives you've considered
Support pod association from attribute of a metric, log, span in k8sattrs processor.

Additional context
Sample config

    processors:
      batch:
      groupbyattrs:
        keys:
          - namespace
          - pod
      resource/copy:
        attributes:
          - key: k8s.namespace.name
            from_attribute: namespace
            action: insert
          - key: k8s.pod.name
            from_attribute: pod
            action: insert
      k8sattributes:
        auth_type: kubeConfig
        passthrough: false
        extract:
          metadata:
          labels:
            - key_regex: (.*)
              tag_name: $$0
        pod_association:
          - sources:
              - from: resource_attribute
                name: k8s.namespace.name
              - from: resource_attribute
                name: k8s.pod.name
    service:
      pipelines:
        metrics:
          receivers: [prometheus]
          processors: [batch, groupbyattrs, resource/copy, k8sattributes]
          exporters: [ prometheusremotewrite ]
@newly12
Copy link
Contributor Author

newly12 commented May 20, 2022

noticed resource_to_telemetry_conversion config in remote write exporter which does the trick.

@newly12 newly12 closed this as completed May 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant