Fail to Approve Topic Request

[luke-zhou]

What happened?

Try to follow the Quickstart, when come to step 9 Approve request, fail to do that. The error on the page shows:
Failure unable to approve requestId 1002 Could not approve topic request. Please contact Administrator.

[aindriu-aiven]

Hi @luke-zhou apologies I didn't see this issue until now,

Would you be able to check the logs and add them here?

if you run the following command and then execute step 9 again gather those logs

docker logs --follow klaw-core

And repeat the same again for here:
docker logs --follow klaw-cluster-api

Apologies again for missing this post and not replying sooner.

[muralibasani]

@luke-zhou I just followed the same steps on mac os, and it was ok.

docker exec -it klaw-kafka /opt/bitnami/kafka/bin/kafka-topics.sh --list --bootstrap-server localhost:9092
__consumer_offsets
_schemas
testtopic

[luke-zhou]

Hi, I have attached the log here, and also a screenshot. BTW I am running this on ubuntu, not sure this info is related or not, just provided to you guys in case.
api.log
core.log

[aindriu-aiven]

Hi, I have attached the log here, and also a screenshot. BTW I am running this on ubuntu, not sure this info is related or not, just provided to you guys in case. api.log core.log

Thank you for the logs, at the moment it looks like docker is not able to resolve the docker host to be able to call the cluster api and make the change.

I do know the docker routing on Linux is different then on Mac and Windows and I also run ubuntu so I will try to reproduce and resolve this issue this morning.

Thanks, and I'll get back to you soon!

[aindriu-aiven]

@luke-zhou Thanks a million again for raising this. The below changes the networking for linux.
I tested before and after and it did work for me so please let me know if that fixes the issue for you.

docker run -d -t -i --network=host \
-e KLAW_CLUSTERAPI_ACCESS_BASE64_SECRET="dGhpcyBpcyBhIHNlY3JldCB0byBhY2Nlc3MgY2x1c3RlcmFwaQ==" \
-p 9343:9343 \
--name klaw-cluster-api aivenoy/klaw-cluster-api:nightly --add-host localhost:host-gateway \
&& docker run -d -t -i --network=host \
-e KLAW_UIAPI_SERVERS=http://localhost:9097 -e KLAW_CLUSTERAPI_ACCESS_BASE64_SECRET="dGhpcyBpcyBhIHNlY3JldCB0byBhY2Nlc3MgY2x1c3RlcmFwaQ==" \
-e KLAW_QUICKSTART_ENABLED=true \
-e SPRING_DATASOURCE_URL="jdbc:h2:file:/klaw/klawprodb;DB_CLOSE_ON_EXIT=FALSE;DB_CLOSE_DELAY=-1;MODE=MySQL;CASE_INSENSITIVE_IDENTIFIERS=TRUE;" \
-p 9097:9097 --name klaw-core aivenoy/klaw-core:nightly \
&& docker run -d -t -i --network=host -p 2181:2181 --add-host localhost:host-gateway \
-e ALLOW_ANONYMOUS_LOGIN=yes --name klaw-zookeeper bitnami/zookeeper:3.8 \
&& docker run -d -t -i --network=host -p 9092:9092 --add-host localhost:host-gateway \
-e KAFKA_CFG_ZOOKEEPER_CONNECT=localhost:2181 -e ALLOW_PLAINTEXT_LISTENER=yes \
-e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 \
--name klaw-kafka bitnami/kafka:3.3 \
&& docker run -d -t -i --network=host -p 8081:8081 --add-host localhost:host-gateway \
-e SCHEMA_REGISTRY_KAFKA_BROKERS=PLAINTEXT://localhost:9092 \
--name klaw-schema-registry bitnami/schema-registry:latest

For additional context, host.docker.internal was not supported on docker for linux, the networking changes for this were raised in this PR in 2019, it looks like it was closed in 2023, I am going to update the documentation to show a specific Linux setup to try it out.

Also we have added a number of cool features to the Klaw UI since this demo was created so I am going to create an issue to update the current version we are using as well.

[aindriu-aiven]

Related Klaw-docs PR created
Aiven-Open/klaw-docs#233

[luke-zhou]

I am afraid the issue is still existing. I have attached the log for core.
core.log

[aindriu-aiven]

I am afraid the issue is still existing. I have attached the log for core. core.log

Hey @luke-zhou did you delete the containers that already existed before copying and pasting the linux version of the docker command?

As I can see in the logs it is still looking for host.docker.internal but that is not defined in the linux version anymore?

[luke-zhou]

what i did is docker container prune. is that ok?

[aindriu-aiven]

what i did is docker container prune. is that ok?

That should be ok as long as the containers were stopped before you ran the command.

can you run "docker container ls" and see how the ports are set up? if its properly taken the network = host the ports should all be blank

[luke-zhou]

Please see attached screenshot

[aindriu-aiven]

Please see attached screenshot

Ok it looks like the deployment is working but if we are still seeing the old host.docker.internal in the api calls I think we need to update the setting as superadmin.
If you follow the instructions on this page under verify the installation, you should be able to see the url being used to contact the cluster api. (my bet is it is host.docker.internal)
https://www.klaw-project.io/docs/setup-configuration/klaw-installation/run-docker#5-verify-the-installation

You want to login as superadmin/welcometoklaw (or possibly kwsuperadmin123$$ for this deployment I'm not 100% sure)
change host.docker.internal to localhost
Test the connection
Save if it works.

This will hopefully resolve the issue you were having.

[luke-zhou]

@aindriu-aiven , I tried to change the setting from host.docker.internal to localhost. After I changed, tested connection was ok, no issue. Then I retested the approve-topic issue, the issue was still existing. I thought I may need to restart the server. So I stopped the container, and tried to restart them, but container klaw-kafka failed to start

[aindriu-aiven]

@aindriu-aiven , I tried to change the setting from host.docker.internal to localhost. After I changed, tested connection was ok, no issue. Then I retested the approve-topic issue, the issue was still existing. I thought I may need to restart the server. So I stopped the container, and tried to restart them, but container klaw-kafka failed to start

Hey @luke-zhou thanks for following up on this can you send me the logs of the failed to start instance?

This is very strange I have never seen anything like this before.

[luke-zhou]

@aindriu-aiven sorry, i am not familiar with docker, can you tell me how can I retrieve that log?

[aindriu-aiven]

@aindriu-aiven sorry, i am not familiar with docker, can you tell me how can I retrieve that log?

Hey, no worries at all.

If you could send the klaw-core and klaw-cluster-api logs again after the previous changes we made.

docker logs --follow klaw-core
docker logs --follow klaw-cluster-api

[luke-zhou]

api.log
core.log

[aindriu-aiven]

api.log core.log

Hey @luke-zhou good news is we now see that the core is able to communicate correctly with the API and it is now the api that is having trouble connecting to the kafka instance!

If you sign in as superadmin again and go to clusters and edit cluster and change the bootstrap server to "localhost:9092" from the current "host.docker.internal:9092" I think we will finally see it working, you'll also be able to test that connectivity on the environment page.

Because the instance was already setup the config was saved in the database meaning that this piece of config from the original deployment was kept.

I think this should resolve the last bottleneck you have though to getting the flow end 2 end :)

[luke-zhou]

Yes, it is working now, thank you for your help @aindriu-aiven

[aindriu-aiven]

Yes, it is working now, thank you for your help @aindriu-aiven

Super delighted that it got working!