[Question] ...Unable to access kafka-cluster from outside cluster #3967
Comments
|
Right now, your
If you want to use also TLS client authentication you will need to enable it in the Kafka YAML: external:
type: loadbalancer
tls: true
authentication:
type: tls
overrides:
bootstrap:
loadBalancerIP: xxxxx
brokers:
- broker: 0
loadBalancerIP: xxxxxAnd then you can get the keystore from the user secret (in your case I guess the one called
|
|
Thanks for the help. Now I have upgraded strimzi operator to 0.20 and kafka version to 2.6.0, in this override option is depricated and my new listener configurations for type loadBalancer as below. I was facing the issue while deploying kafka cluster. Is there any configurations for mentioning specific Ip for bootstrap and kafka brokers as we did in earlier version. I found configuration in documentation but confused with ranges. Providing Url for reference kafka.yaml : apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
name: kafka-develop
annotations:
metallb.universe.tf/allow-shared-ip: smart
spec:
kafka:
version: 2.6.0
replicas: 1
listeners:
- name: plain
port: 9092
type: internal
tls: false
- name: tls
port: 9093
type: internal
tls: true
- name: external
port: 9094
type: loadbalancer
tls: true
config:
offsets.topic.replication.factor: 1
transaction.state.log.replication.factor: 1
transaction.state.log.min.isr: 1
log.message.format.version: "2.6"
storage:
type: persistent-claim
size: 10Gi
class: sc-paas
deleteClaim: false
zookeeper:
replicas: 1
storage:
type: persistent-claim
size: 10Gi
class: sc-paas
deleteClaim: falseIssue while deploying : |
|
I do not think you updated to Strimzi 0.20.0. I think the error suggests that at least your CRD resources are still from previous version. As for the |
|
Thank you for the quick support. I have now upgraded to 0.20 now and deployed kafka with following configurations. kubectl get all -n kafka-op-test NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE NAME READY UP-TO-DATE AVAILABLE AGE NAME DESIRED CURRENT READY AGE NAME READY AGE Kafka.yaml: Operator logs: |
|
Could you please use code formatting for the YAMLs and command outputs? It would make it much easier to decode it and help. The error says |
|
Thanks for reply. external listener is now working fine and but I want enable authentication using oauth and authorization using keycloak with tls enabled . I tried with following configurations and was facing the issue. Regarding to tlsTrustedCertificates: property which secret I need to use here. I found one of issue which is similar to my requirement, providing link below. Please suggest any configuration changes I need to be done. Thanks kafka.yaml: Issue:
|
|
Sorry, after posting the comment its original format changing and tried to edit the comment as well. I'll try to provide original format of kafka.yaml file. |
|
Well, the YAML you posted is wrong ... you have the |
|
I'm providing as screen shot now. |
|
Please read my answer above. |
|
this is my server.properties but while i am producing messages in the strimzi kafka it showing below error |
|
@Cshanmukh80 And how does it relate to this issue? Unless it is related you can ask your questions here: https://github.com/strimzi/strimzi-kafka-operator/discussions ... you should also provide all the information including full logs, configuration of your Kafka cluster (the Kafka custom resource etc.), explain how are you trying to access the cluster etc. |
I had deployed strimzi-kafka cluster with external listeners with load balancers. It is working fine within pod with 9092 port able to create topics. But actual issue facing while deploying kafka with tls. Here I don't know whether which method need to follow for accessing kafka from outside cluster. For apache kafka we used to conect through kafka client using keystore and truststore in properties file which can be provided in the command. Ex ./kafka-console-producer.sh --broker-list Ip:port --topic my-topic --producer.config ssl-properties.txt. This ssl-properties file contain path and passwords of truststore and keystore. For accessing strimzi kafka also do we need to create keystore and truststore or any other alternative method is there for accessing kafka clusters.
Providing my details of my deployment.
kafka.yaml:
apiVersion: kafka.strimzi.io/v1beta1
kind: Kafka
metadata:
name: my-cluster
#annotations:
metallb.universe.tf/allow-shared-ip: smart
spec:
kafka:
version: 2.5.0
replicas: 1
listeners:
plain: {}
external:
type: loadbalancer
tls: true
overrides:
bootstrap:
loadBalancerIP: xxxxx
brokers:
- broker: 0
loadBalancerIP: xxxxx
zookeeper:
replicas: 1
resources:
requests:
memory: 8Gi
cpu: "1"
limits:
memory: 24Gi
cpu: "3"
storage:
type: persistent-claim
size: 10Gi
class: sc-paas
deleteClaim: false
pods and services :
kubectl get all -n kafka-op-test
NAME READY STATUS RESTARTS AGE
pod/my-cluster-entity-operator-558ffb588d-24tlz 3/3 Running 0 25h
pod/my-cluster-kafka-0 2/2 Running 0 25h
pod/my-cluster-zookeeper-0 1/1 Running 0 25h
pod/strimzi-cluster-operator-55dd5ccd6f-pd4mk 1/1 Running 0 26h
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/my-cluster-kafka-0 LoadBalancer xxxxxxxxxx xxxxxxxxxxx 9094:30321/TCP 25h
service/my-cluster-kafka-bootstrap ClusterIP xxxxxxxxxx 9091/TCP,9092/TCP 25h
service/my-cluster-kafka-brokers ClusterIP None 9091/TCP,9092/TCP 25h
service/my-cluster-kafka-external-bootstrap LoadBalancer xxxxxxxxxx xxxxxxxxx 9094:31982/TCP 25h
service/my-cluster-zookeeper-client ClusterIP xxxxxxxxxx 2181/TCP 25h
service/my-cluster-zookeeper-nodes ClusterIP None 2181/TCP,2888/TCP,3888/TCP 25h
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/my-cluster-entity-operator 1/1 1 1 25h
deployment.apps/strimzi-cluster-operator 1/1 1 1 26h
NAME DESIRED CURRENT READY AGE
replicaset.apps/my-cluster-entity-operator-558ffb588d 1 1 1 25h
replicaset.apps/strimzi-cluster-operator-55dd5ccd6f 1 1 1 26h
NAME READY AGE
statefulset.apps/my-cluster-kafka 1/1 25h
statefulset.apps/my-cluster-zookeeper 1/1 25h
Secrets created by strimzi operator:
my-cluster-clients-ca Opaque 1 25h
my-cluster-clients-ca-cert Opaque 3 25h
my-cluster-cluster-ca Opaque 1 25h
my-cluster-cluster-ca-cert Opaque 3 25h
my-cluster-cluster-operator-certs Opaque 4 25h
my-cluster-entity-operator-certs Opaque 4 25h
my-cluster-entity-operator-token-zp5k2 kubernetes.io/service-account-token 3 25h
my-cluster-kafka-brokers Opaque 4 25h
my-cluster-kafka-token-bxff5 kubernetes.io/service-account-token 3 25h
my-cluster-zookeeper-nodes Opaque 4 25h
my-cluster-zookeeper-token-bfbx4 kubernetes.io/service-account-token 3 25h
my-secret Opaque 2 62d
my-user Opaque 5 60d
strimzi-cluster-operator-token-qcsjg kubernetes.io/service-account-token 3 78d
can you please suggest the next process to access kafka cluster with tls enabled.
The text was updated successfully, but these errors were encountered: