Skip to content

Commit

Permalink
kafka documentation updates
Browse files Browse the repository at this point in the history
I never got kafka-ui to actually work, but I'm 99.9% sure that the
BOOTSTRAPSERVERS env var was incorrect.
  • Loading branch information
ewollesen committed Jan 2, 2024
1 parent e0c6a10 commit 52a6010
Show file tree
Hide file tree
Showing 2 changed files with 29 additions and 2 deletions.
27 changes: 27 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ Of course, if you haven't already done so, you should check out [Tidepool](https
- [Working With Yarn For NodeJS Services](#working-with-yarn-for-nodejs-services)
- [Misc](#misc)
- [Tracing Internal Services](#tracing-internal-services)
- [Kafka CLI Usage](#kafka-cli-usage)
- [Troubleshooting](#troubleshooting)
- [Known Issues](#known-issues)

Expand Down Expand Up @@ -945,6 +946,32 @@ The good news is that we anticipate upgrading the Gloo version in the near futur

Stay Tuned :)


## Kafka CLI Usage

A few notes for how to access/use the Kafka CLI. Accurate as of 2023-12-15 at least. :)

The Kafka CLI (which is just a bunch a bash scripts) is accessible via the kafka pod:

$ kubectl exec -it kafka-kafka-0 -- bash
$ ls bin/kafka-* # to see all the various CLI commands

However, a lot of the commands require authentication credentials, and those more or less have to come from a file. The kafka pod is lacking in editors, so here's one way to get around that:

$ cat - > /tmp/kafka.config <<EOF
KafkaClient {
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username="kafka" password="<password from kafka secret>";
};
EOF
$ bin/kafka-topics.sh --command-config=/tmp/kafka.config --list --bootstrap-server localhost:9092

And remember that the password found in the kafka secret will be base64 encoded, so reverse that before you insert it in the file above.

$ kubectl get -o json secret kafka | jq -r '.data.password | @base64d'


## Troubleshooting

| Issue | Things to try |
Expand Down
4 changes: 2 additions & 2 deletions tools/kafka-ui/kafka-ui-configMap.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# Add config map:
# create configmap kafka-ui-helm-values --from-file tools/kafka-ui/kafka-ui-configMap.yaml
# kubectl create configmap kafka-ui-helm-values --from-file tools/kafka-ui/kafka-ui-configMap.yaml

# Provision kafka-ui with:
# helm repo add kafka-ui https://provectus.github.io/kafka-ui-charts
Expand All @@ -11,7 +11,7 @@ metadata:
name: kafka-ui-helm-values
data:
KAFKA_CLUSTERS_0_NAME: "kafka-kafka"
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: "kafka-kafka-bootstrap:9092"
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: "kafka-kafka-0:9092"
KAFKA_CLUSTERS_0_PROPERTIES_SASL_MECHANISM: "SCRAM-SHA-512"
KAFKA_CLUSTERS_0_PROPERTIES_SECURITY_PROTOCOL: "SASL_PLAINTEXT"
KAFKA_CLUSTERS_0_PROPERTIES_SASL_JAAS_CONFIG: "org.apache.kafka.common.security.scram.ScramLoginModule required username='kafka' password='**GET_FROM_KAFKA_SECRET**';"

0 comments on commit 52a6010

Please sign in to comment.