Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bring Kafkamatrics receiver into the discovery mode #4903

Merged
merged 4 commits into from
May 30, 2024
Merged

Conversation

samiura
Copy link

@samiura samiura commented May 29, 2024

Description: Simple attempt to enable discovery mode for kafkamatrics receiver

Link to Splunk idea: #4903

Testing: Tested for host observer, docker observer and k8s observer without any crashes.

Sample --dry-run output:

samiura@r91395f3 [03:09:07 PM] [~/test]                                                                                                                                                                                                                                                                                                           
-> % docker run --name otelcol2 -d  \                                                                                                                                                                                                                                                                                                             
    --pid host \                                                                                                                                                                                                                                                                                                                                  
  --network host \                                                                                                                                                                                                                                                                                                                                
  --volume /var/run/docker.sock:/var/run/docker.sock \                                                                                                                                                                                                                                                                                            
   -v "${PWD}":/home/ \                                                                                                                                                                                                                                                                                                                           
       -p 4318:4318 \                                                               
quay.io/splunko11ytest/otelcol_test/otelcol:latest \                                
--config /home/splunk-otel-config1.yaml \                                           
--discovery --dry-run;                                                              
samiura@r91395f3 [03:09:11 PM] [~/test]                                             
-> % docker logs -f otelcol2

-> % docker logs -f otelcol2
2024/05/29 22:09:07 settings.go:471: Set config to [/home/splunk-otel-config1.yaml]
2024/05/29 22:09:07 settings.go:532: Set memory limit to 460 MiB
2024/05/29 22:09:07 settings.go:517: Set soft memory limit set to 460 MiB
2024/05/29 22:09:07 settings.go:407: set "SPLUNK_LISTEN_INTERFACE" to "0.0.0.0"
Discovering for next 10s...
Successfully discovered "kafkametrics" using "docker_observer" endpoint "3ced9172994ad850ff1457cd5fafc940c9e17ddba49250b899abb43b144752d3:9092".
Discovery complete.
exporters:
  signalfx:
    access_token: ACCESS_TOKEN_STRING
    drop_histogram_buckets: true
    realm: us0
  signalfx/internal:
    access_token: ACCESS_TOKEN_STRING
    realm: us0
    sync_host_metadata: true
extensions:
  docker_observer: {}
receivers:
  hostmetrics:
    collection_interval: 10s
    scrapers:
      cpu: null
      disk: null
      filesystem: null
      memory: null
      network: null
  receiver_creator/discovery:
    receivers:
      kafkametrics:
        config:
          protocol_version: 2.0.0
          scrapers:
          - brokers
          - topics
          - consumers
        resource_attributes: {}
        rule: type == "container" and any([name, image, command], {# matches "(?i)kafka.*"})
          and not (command matches "splunk.discovery")
    watch_observers:
    - docker_observer
service:
  extensions:
  - docker_observer
  pipelines:
    metrics:
      exporters:
      - signalfx
      receivers:
      - hostmetrics
      - receiver_creator/discovery
    metrics/internal:
      exporters:
      - signalfx/internal
      processors: null
      receivers:
      - hostmetrics
  telemetry:
    resource:
      splunk_autodiscovery: "true"

Documentation:

@samiura samiura requested review from a team as code owners May 29, 2024 22:10
@samiura samiura force-pushed the kafkametrics-branch branch from fea19a2 to cd8d4bb Compare May 29, 2024 22:53
@samiura samiura force-pushed the kafkametrics-branch branch from cd8d4bb to d8aa7f8 Compare May 30, 2024 22:43
@samiura samiura merged commit 6ed7e4f into main May 30, 2024
16 checks passed
@samiura samiura deleted the kafkametrics-branch branch May 30, 2024 22:43
@github-actions github-actions bot locked and limited conversation to collaborators May 30, 2024
@samiura samiura changed the title Initial attempt to bring Kafkamatrics receiver into the discovery mode Bring Kafkamatrics receiver into the discovery mode Jun 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants