Kafka Exporter Not Creating Topics

[kadhamecha-conga]

Component(s)

exporter/kafka

What happened?

hi team,

The Kafka exporter is failing to automatically create topics as expected. This issue affects the following default topics:

otlp_spans (for traces)
otlp_metrics (for metrics)

Expected Behavior:
The Kafka exporter should automatically create these topics if they do not already exist when data is sent for export.

Current Behavior:
Topics are not being created automatically, causing potential data loss or export failures.

Collector version

0.83.0

Environment information

Environment

eks 1.28

OpenTelemetry Collector configuration

receivers:
      otlp:
        protocols:
          grpc:
            keepalive:
              server_parameters:
                max_connection_age: 1h 
                max_connection_age_grace: 10m

          http:
            cors:
              allowed_origins:
              - "*"

    processors:
    extensions:
      health_check:

    exporters:
      kafka:
        brokers:
          - kafka:9092
        protocol_version: 2.0.0
      logging:
        loglevel: info
      logging/debug:
        loglevel: DEBUG
      logging/detail:
        verbosity: detailed

    service:
      extensions: [health_check ]

      pipelines:
        traces:
          receivers: [otlp]
          processors:
            - memory_limiter
            - batch
          exporters:
            - kafka
        
        metrics:
          receivers: [otlp]
          processors:
            - memory_limiter
            - batch
          exporters:
            - kafka

        logs:
          receivers: [otlp]
          processors:
            - memory_limiter
            - batch
          exporters:
            - kafka

Log output

No response

Additional context

No response

[github-actions]

Pinging code owners:

• exporter/kafka: @pavolloffay @MovieStoreGuy

See Adding Labels via Comments if you do not have permissions to add labels yourself.