-
Notifications
You must be signed in to change notification settings - Fork 3.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
if there is Event/notify when all brokers are down or one of them is up #1801
Comments
What you describe is the task of the client, not the application. Instead, the idea is to give librdkafka a couple of contraints when it comes to producing messages, namely:
More info here: https://github.com/edenhill/librdkafka/blob/master/INTRODUCTION.md#message-reliability |
Thanks for your response. |
Is there any way that the application can detect when the brokers become available? |
Apart from the ERR__ALL_BROKERS_DOWN error event there is currently no signalling for brokers coming up again, and what is really interesting is the availability of the leader for a partition. In the meantime I suggest you do a combination of listening to ERR__ALL_BROKERS_DOWN (error_cb) and monitoring the per-broker state in statistics (stats_cb, statistics.interval.ms=...): |
Glad to know that. I also see someone using metadata to monitor the brokers. By query the metadata, we can know the available count of brokers. Is query metadata a good choice? Is there impact for performance? rd_kafka_resp_err_t typedef struct rd_kafka_metadata {
} rd_kafka_metadata_t; |
metadata query will tell you what brokers exist in the cluster, but wont tell you if they are available. |
OK, great thanks! I studied the statistics you provided and the call back function can return all brokers information including the broker state. How may states the broker have? For my case, if the state is "UP' or "UPDATE", it means the broker is available, right? |
Yep. |
great!Thank you very much! |
please tell me how to get broker[].state?? i can't get it by function rd_kafka_metadata(..), |
i think the test connection operation should exist in our app application level. such as i use golang to send kafka message by librdkafka, i will start a new go routine to test if the broker is ok. |
Read the FAQ first: https://github.com/edenhill/librdkafka/wiki/FAQ
Description
Hi, I am sorry that I don't know where to ask questions. So I create an issue here to get your help. I am using the librdkafka using C language. The producer wants to know if there is available brokers. If all brokers are down, the producer can fire alarm. If at least one of the brokers is up, the alarm is cleared. I read the kafka code and see there is error call back function conf->error_cb which will tell the producer the error code. For all brokers down case, there is error code RD_KAFKA_RESP_ERR__ALL_BROKERS_DOWN.
So my questions are:
Appreciate you response and help.
The text was updated successfully, but these errors were encountered: