Kafdrop connection to an kafka servier with just SSL (not SASL) does not work

[krishna-prasad-s]

In the readme its described as its working, but when I provide the required details. It throws an exception (mainly because the server refuses)
the docker command

docker run  --rm -p 9000:9000 \
    --name kafdrop \
    -e KAFKA_BROKERCONNECT="172.17.0.3:9092" \
    -e KAFKA_ISSECURED="false" \
    -e KAFKA_PROPERTIES="$(cat kafka.properties | base64)" \
    -e KAFKA_TRUSTSTORE="$(cat ../truststore/kafka.truststore.jks | base64)" \
    -e KAFKA_KEYSTORE="$(cat ../kafdrop-keystore/kafdrop.keystore.jks | base64)" \
    -e JVM_OPTS="-Xms32M -Xmx64M" \
    -e SERVER_SERVLET_CONTEXTPATH="/" \
    obsidiandynamics/kafdrop

my kafka.properties
ssl.security.protocol=SSL
ssl.endpoint.identification.algorithm=
ssl.protocol=TLS

ssl.truststore.password="*****"
ssl.truststore.type=JKS

ssl.keystore.password="*****"
ssl.keystore.type=JKS

the exception

2020-10-25 14:26:57.533  INFO ${sys:PID} [           main] k.Kafdrop$EnvironmentSetupListener       : Initializing JAAS config
2020-10-25 14:26:57.542  INFO ${sys:PID} [           main] k.Kafdrop$EnvironmentSetupListener       : env: null .isSecured kafka: false
2020-10-25 14:26:57.543  INFO ${sys:PID} [           main] k.Kafdrop$EnvironmentSetupListener       : Env: null
2020-10-25 14:26:57.681  INFO 1 [           main] o.s.b.StartupInfoLogger                  : Starting application on 7d0eb263b4cc with PID 1 (started by root in /)
2020-10-25 14:26:57.682  INFO 1 [           main] o.s.b.SpringApplication                  : No active profile set, falling back to default profiles: default
2020-10-25 14:26:59.035  INFO 1 [           main] i.u.s.s.ServletContextImpl               : Initializing Spring embedded WebApplicationContext
2020-10-25 14:26:59.036  INFO 1 [           main] w.s.c.ServletWebServerApplicationContext : Root WebApplicationContext: initialization completed in 1314 ms
2020-10-25 14:26:59.383  INFO 1 [           main] k.c.KafkaConfiguration                   : Checking truststore file kafka.truststore.jks
2020-10-25 14:26:59.383  INFO 1 [           main] k.c.KafkaConfiguration                   : Assigning truststore location to kafka.truststore.jks
2020-10-25 14:26:59.384  INFO 1 [           main] k.c.KafkaConfiguration                   : Checking keystore file kafka.keystore.jks
2020-10-25 14:26:59.384  INFO 1 [           main] k.c.KafkaConfiguration                   : Assigning keystore location to kafka.keystore.jks
2020-10-25 14:26:59.385  INFO 1 [           main] k.c.KafkaConfiguration                   : Checking properties file kafka.properties
2020-10-25 14:26:59.385  INFO 1 [           main] k.c.KafkaConfiguration                   : Loading properties from kafka.properties
2020-10-25 14:26:59.457  WARN 1 [           main] o.a.k.c.c.AbstractConfig                 : The configuration 'ssl.security.protocol' was supplied but isn't a known config.
2020-10-25 14:26:59.463  INFO 1 [           main] k.c.KafkaConfiguration                   : Checking truststore file kafka.truststore.jks
2020-10-25 14:26:59.463  INFO 1 [           main] k.c.KafkaConfiguration                   : Assigning truststore location to kafka.truststore.jks
2020-10-25 14:26:59.464  INFO 1 [           main] k.c.KafkaConfiguration                   : Checking keystore file kafka.keystore.jks
2020-10-25 14:26:59.464  INFO 1 [           main] k.c.KafkaConfiguration                   : Assigning keystore location to kafka.keystore.jks
2020-10-25 14:26:59.464  INFO 1 [           main] k.c.KafkaConfiguration                   : Checking properties file kafka.properties
2020-10-25 14:26:59.464  INFO 1 [           main] k.c.KafkaConfiguration                   : Loading properties from kafka.properties
2020-10-25 14:26:59.485  WARN 1 [           main] o.a.k.c.c.AbstractConfig                 : The configuration 'ssl.truststore.location' was supplied but isn't a known config.
2020-10-25 14:26:59.486  WARN 1 [           main] o.a.k.c.c.AbstractConfig                 : The configuration 'ssl.security.protocol' was supplied but isn't a known config.
2020-10-25 14:26:59.486  WARN 1 [           main] o.a.k.c.c.AbstractConfig                 : The configuration 'ssl.keystore.password' was supplied but isn't a known config.
2020-10-25 14:26:59.486  WARN 1 [           main] o.a.k.c.c.AbstractConfig                 : The configuration 'ssl.keystore.location' was supplied but isn't a known config.
2020-10-25 14:26:59.486  WARN 1 [           main] o.a.k.c.c.AbstractConfig                 : The configuration 'ssl.truststore.password' was supplied but isn't a known config.
2020-10-25 14:26:59.495  INFO 1 [           main] k.s.BuildInfo                            : Kafdrop version: 3.27.0, build time: 2020-06-21T23:16:06.428Z
2020-10-25 14:26:59.687 ERROR 1 [| kafdrop-admin] o.a.k.c.u.KafkaThread$1                  : Uncaught exception in thread 'kafka-admin-client-thread | kafdrop-admin':

java.lang.OutOfMemoryError: Java heap space
	at java.base/java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:61)
	at java.base/java.nio.ByteBuffer.allocate(ByteBuffer.java:348)
	at org.apache.kafka.common.memory.MemoryPool$1.tryAllocate(MemoryPool.java:30)
	at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:112)
	at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:424)
	at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:385)
	at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:651)
	at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:572)
	at org.apache.kafka.common.network.Selector.poll(Selector.java:483)
	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:539)
	at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1152)
	at java.base/java.lang.Thread.run(Thread.java:835)

2020-10-25 14:26:59.909  INFO 1 [           main] o.s.b.a.e.w.EndpointLinksResolver        : Exposing 15 endpoint(s) beneath base path '/actuator'
2020-10-25 14:26:59.974  INFO 1 [           main] pertySourcedRequestMappingHandlerMapping : Mapped URL path [/v2/api-docs] onto method [public org.springframework.http.ResponseEntity<springfox.documentation.spring.web.json.Json> springfox.documentation.swagger2.web.Swagger2Controller.getDocumentation(java.lang.String,javax.servlet.http.HttpServletRequest)]
2020-10-25 14:27:00.068  INFO 1 [           main] o.s.s.c.ExecutorConfigurationSupport     : Initializing ExecutorService 'applicationTaskExecutor'
2020-10-25 14:27:00.338  INFO 1 [           main] d.s.w.p.DocumentationPluginsBootstrapper : Context refreshed
2020-10-25 14:27:00.349  INFO 1 [           main] d.s.w.p.DocumentationPluginsBootstrapper : Found 1 custom documentation plugin(s)
2020-10-25 14:27:00.388  INFO 1 [           main] s.d.s.w.s.ApiListingReferenceScanner     : Scanning for api listing references
2020-10-25 14:27:00.485  INFO 1 [           main] .d.s.w.r.o.CachingOperationNameGenerator : Generating unique operation named: brokerDetailsJsonUsingGET_1
2020-10-25 14:27:00.531  INFO 1 [           main] .d.s.w.r.o.CachingOperationNameGenerator : Generating unique operation named: getAllTopicsUsingGET_1
2020-10-25 14:27:00.563  INFO 1 [           main] o.x.Xnio                                 : XNIO version 3.3.8.Final
2020-10-25 14:27:00.569  INFO 1 [           main] o.x.n.NioXnio                            : XNIO NIO Implementation Version 3.3.8.Final
2020-10-25 14:27:00.604  INFO 1 [           main] o.s.b.w.e.u.UndertowServletWebServer     : Undertow started on port(s) 9000 (http) with context path ''
2020-10-25 14:27:00.607  INFO 1 [           main] o.s.b.StartupInfoLogger                  : Started application in 3.289 seconds (JVM running for 4.051)

[krishna-prasad-s]

I found that, to fix it all that I had to do was to
move out properties.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, securityProtocol);
from the below block

    if (isSecured) {
      LOG.warn("The 'isSecured' property is deprecated; consult README.md on the preferred way to configure security");
//      properties.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, securityProtocol);
      properties.put(SaslConfigs.SASL_MECHANISM, saslMechanism);
    }

    properties.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG, securityProtocol);

and I could simplify the kafka.properties to

ssl.security.protocol=SSL
ssl.endpoint.identification.algorithm=
ssl.truststore.password=******
ssl.truststore.type=JKS
ssl.keystore.password=*****
ssl.keystore.type=JKS
ssl.key.password=******

[rldeep2889]

Hello @krishna-prasad-s ,

Wanted one quick info. How are you able to pass your truststore file via argument. The truststore file is pretty huge and whenever i run based on the docker run command,.. i get bin/docker : too many arguments error

[krishna-prasad-s]

@rldeep2889

-e KAFKA_TRUSTSTORE="$(cat ../truststore/kafka.truststore.jks | base64)" \

if your file is too big why don't you create your own docker image using this as the base image and packaging your trust store.

[FedeBev]

Hi there,
I'm facing the same issue but I'm using neither SSL nor SASL, just the helm sample in the readme.

Here the error:

2021-01-03 10:27:13.093 ERROR 1 [| kafdrop-admin] o.a.k.c.u.KafkaThread$1                  : Uncaught exception in thread 'kafka-admin-client-thread | kafdrop-admin':

java.lang.OutOfMemoryError: Java heap space
	at java.base/java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:61)
	at java.base/java.nio.ByteBuffer.allocate(ByteBuffer.java:348)
	at org.apache.kafka.common.memory.MemoryPool$1.tryAllocate(MemoryPool.java:30)
	at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:112)
	at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:424)
	at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:385)
	at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:651)
	at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:572)
	at org.apache.kafka.common.network.Selector.poll(Selector.java:483)
	at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:539)
	at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1152)
	at java.base/java.lang.Thread.run(Thread.java:835)

Here my helm custom helm values:

service:
  annotations: {}
  type: ClusterIP
  port: 9000
  nodePort: ""

kafka:
  brokerConnect: kafka-kafka-bootstrap:9093

jvm:
  opts: "-Xms32M -Xmx64M"

image:
    tag: 3.27.0

[rldeep2889]

@rldeep2889

-e KAFKA_TRUSTSTORE="$(cat ../truststore/kafka.truststore.jks | base64)" \

if your file is too big why don't you create your own docker image using this as the base image and packaging your trust store.

Yeah tried that now. thanks !

[lfmunoz]

Works for me when I added without having to rebuild the source code.

security.protocol=SSL

what you have seems to give a warning

ssl.security.protocol=SSL

2021-12-07 22:38:36.939 WARN 1 [ main] o.a.k.c.c.AbstractConfig : The configuration 'ssl.security.protocol' was supplied but isn't a known config.

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.

[Bert-R]

This issue is still relevant to us and a fix is available in #202. Can you please look into that PR and merge it in? That way, we can use the standard Kafdrop image instead of having to maintain our own one.

[github-actions]

This issue is stale because it has been open for 30 days with no activity.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.