Add option to set TCP_NODELAY

[rfalke]

An MQTT broker can be run as a local message queue where the clients are running on the same host. In such a case the latency is quite large because of Nagle's Algorithm. It would be nice if paho allows has a config option which sets TCP_NODELAY to reduce this latency.

[icraggs]

I deliberately avoided setting TCP_NODELAY as some people have used it gratuitously to fix coding problems. Allowing it as an option seems reasonable.

[Lahrach]

Is there any work around to avoid the latency induced by Nagle's Algorithm when the clients are running on the same host?

[vortex314]

Hi, it's unclear to me if this has been implemented ?

[jumoog]

Hi, it's unclear to me if this has been implemented ?

no its not implemented

[icraggs]

This is not top of my list to implement - as evidenced by it being on the backlog milestone.

If anyone wants to make it happen quicker, then they could open a PR (remembering documentation and ideally some sort of test) or give me some sponsorship on Github sponsors.

[vortex314]

Ian , thanks for the swift answer. Maybe I'm trying to use MQTT in the wrong context. The purpose was to have MQTT as a central message broker with low latency : https://github.com/vortex314/serial2mqtt ( see picture on the page )
The purpose is driving a robotic distributed system. I noticed that loopback's across MQTT can range from 1 msec to 100 msec, just on the local system. With the strange effect that QOS1 is faster than QOS0, as it forces the same socket to flush faster.
.As Mosquitto supports TCP_NODELAY as a broker, my expectation was to find this also in the MQTT client.
I guess for real robotic work, I need something with more predictable latency. At the base MQTT is for IoT not robotics.
Food for thought. Will have to look at something like : https://www.researchgate.net/publication/332077763_ORC-A_Lightweight_Lightning-Fast_Middleware
Regards
Lieven

[icraggs]

There are many functions in this client which you probably don't need. For your application, which presumably means small packet sizes too, another MQTT client might be better. The Paho embedded C client, or the mosquitto C library (I just checked and libmosquitto has a TCP_NODELAY option).

It's true that the main priority of MQTT implementations tends not to be real time or low latency. Have you tried adding TCP_NODELAY to this client to see if it does actually behave as you expect?

[vortex314]

Thanks for the pointer, I will check it out.
Indeed , it's a drastic difference in a first preliminary test ( for what it's worth ) : 2-3 msec becomes 0.2->0.3 msec.
This with libmosquitto, need to evaluate further.

[pjcha]

Hey @vortex314 do you have evaluation results? Would be interesting to know if you found any performance issues with TCP_NODELAY?

[vortex314]

@PiyushJadhav I felt that MQTT is maybe not the right protocol or broker for something like robotics, where you really look for a minimal latency. MQTT looks to address sensors that are sporadic on line. So I'm actually evaluating Zenoh, Iceoryx, Redis, keydb, Ros2 where I am going to settle on Redis as it stands. Redis gives some features besides low-latency publish-subscribe , things like time-series and streams. Packed in a single product like a swiss army knife. In my case the actuators and sensors communicate instantly across the broker with the central brain. See : https://github.com/vortex314/robotSpine

[fpagliughi]

Wow I totally forgot about this one. I meant to leapfrog it by implementing UNIX sockets to replace localhost when possible. But this is a simpler first step that would be helpful either way, and should be fairly trivial to implement. (Famous last words).

Initially I was wondering if this should be a build or runtime option, but after thinking on it for a second or two, clearly it should be set at runtime on a per-client basis. I have an aggregating local-to-cloud bridge on an IoT gateway, and the local connection should be nodelay, but not the cloud connection. That’s my guess.

So perhaps add a nodelay field to the connect options? And pass it all the way down to the socket creation function - Socket_new() IIRC).

I was also wondering if it should be added as an individual option, or might there be other socket options in the future. If so, perhaps start a sock options sub-struct? I don’t want to over complicate, so I’ll start as a single option.

Does that all make sense?

[icraggs]

@fpagliughi have you tried this out to see what difference it makes?

Another solution could be to pass the socket back to a new callback function called from Socket_new(), then the application can set other options if it wants. Might be the simplest option. I think there's another request on here which wants access to the socket, I'll see if I can find it.

[fpagliughi]

I have no problem doing a few options in the library if they seem to make sense for an MQTT client. Especially since doing it manually could be tricky for the user to get right across platforms. (Windows always makes this kind of thing difficult!)

TCP_NODELAY certainly seems to make sense for MQTT, in theory, for small messages. Going the other way, I could also see wanting to set the socket buffer sizes to send or receive large messages. Maybe? I still have a number of open issues to chase down regarding problems working with large payloads.

But, no, I didn't do an extensive test of this yet. I'm planning on it (something that sends a bunch of small messages, with timing), but got distracted by the C++ client release... which should be this week!