jaeger-agent can directly connect to kafka ？

[wuyupengwoaini]

We are preparing to use jaeger's tracing data for real-time analysis and offline analysis. So we use jaeger-collector to store the data in kafka, then jaeger-agent is still connected to jaeger-collecor.

But I found that the link is too long, and there is a memory queue inside each jaeger component, so the analysis will have a relatively large delay. But if jaeger-agent can connect directly to kafka, it can not only improve real-time performance, but also eliminate the need to deploy jaeger-collector separately.

So, does Jaeger have plans to support jaeger-agent to connect directly to kafka?

[yurishkuro]

I don't think the memory queue introduces any delay unless the collectors are under-provisioned.

Nobody asked for this feature so far, so we don't have any plans around it.

[wuyupengwoaini]

I may not be very clear about what I said. Jaeger-agent can connect directly to kafka, which can save the cost of deployment (no collector is needed), because now jaeger-agent is connected to the load balancer (haproxy) by domain name, and then the load balancer is responsible for distributing the request to jaeger-collector (currently two machines).

If the jaeger-agent can connect directly to kafka, then for us to save at least 3 machines and actually a small time delay

[pavolloffay]

@wuyupengwoaini did you consider sending data directly to the collector instead of the agent? Client libraries should support HTTP sender which is able to send data directly to the collector.

[wuyupengwoaini]

@pavolloffay languages ​​with asynchronous capabilities can theoretically support direct delivery to jaeger-collector. A small part of our company's core system uses asynchronous support very poor php language, so we can only use jaeger-agent to reduce performance loss.

[william-tran]

I think given #2216 and #2221 jaeger-client -> UDP -> agent -> kafka could be cobbled together

[pavolloffay]

Correct, the OTEL components will allow you to configure any combinations of the receivers/exporters.

I am closing this as done.