Missing a way to get local IP address for received UDP message

[net147]

server.address().address returns 0.0.0.0 for a UDP server with multiple network interfaces listening on all interfaces. It would be useful to be able to determine the local IP address of the interface a UDP message is received on.

[patrakov]

Duplicate of issue #6589

[indutny]

I'm afraid this won't work in a general case, and this is the reason we haven't implemented it yet.

For example, if the socket is bound to :: address instead of 0.0.0.0 - it'll get only IPv6 addresses of other side, and nothing if the packet came through IPv4 (on many platforms).

[patrakov]

@indutny your comment missed the point. You are talking about addresses of the otherside, and this bug is about local ones.

Let's imagine that a server has adresses 1.2.3.4 (primary), 5.6.7.8 (secondary, but clients still have the right to send queries there, and I have the right to react to such queries using some different business logic), and others may come and go. Suppose that someone from 9.10.11.12 sends a UDP packet:

9.10.11.12:678 -> 5.6.7.8:345

In C, on the server, I could use IP_PKTINFO (linux) or IP_RECVDSTADDR (FreeBSD) to get the "5.6.7.8" and produce a reply going from the correct address (5.6.7.8, not 1.2.3.4). In node.js, there is no way to get that.

[indutny]

@patrakov ouch, good catch! Actually I didn't meant it, what I meant that IP_RECVDSTADDR won't work for dualstack sockets on OSX and probably other BSDs.

[patrakov]

Unfortunately, this bug caused rather severe tensions here due to requirements put forward by systemd proponents. Basically, they say: it is absolutely mandatory that your network applications do not rely on the network (other than the loopback interface) being up when they start. In fact, there is no non-deprecated language to express dependency on all the necessary IP addresses existing.

The conflict arises because there are three good ways to support multihomed systems in UDP servers, and none of them work with systemd in languages other than C, Ruby and Python >= 3.4.

Way 1: bind to 0.0.0.0 (which always succeeds), use IP_PKTINFO or IP_RECVDSTADDR or equivalents to get the local address, use it explicitly when sending replies. This fails due to this bug (lack of IP_PKTINFO-like functionality in node.js on systems where it exists in C).

Way 2: bind to IP addresses that the application needs when it starts. Use IP_FREEBIND functionality to convince the kernel to allow this binding even though these addresses do not yet exist in the system. Problem: lack of IP_FREEBIND support in node.js.

Way 3: subscribe to netlink notifications about newly-added and removed IP addresses. When an IP address appears that an application is interested in, create a UDP socket and bind to this address. Problem: no netlink support in node.js.

Way 4 (bad): loop and sleep and retry binding to the interesting set of IP addresses until it succeeds.

General problem: writing correct UDP servers for multihomed systems with a dynamic set of IP addresses is impossible in a portable way without busy-looping. It may even be impossible at all in some operating systems. So, if node.js is going to rely on portable functionality only, then any task of writing a multihomed UDP server is unsolvable, and this must be reflected in the documentation so that people don't fall into this trap when choosing an implementation language for a project that is to be run on linux servers.

[net147]

A workaround for UDP broadcasts that might work for some people is to check each interface in os.networkInterfaces() and see which interface has a subnet that the source address is part of. Then you would take the IP address of that interface as the destination address.

[sam-github]

Way 1: bind to 0.0.0.0 (which always succeeds), use IP_PKTINFO or IP_RECVDSTADDR or equivalents to get the local address, use it explicitly when sending replies. This fails due to this bug (lack of IP_PKTINFO-like functionality in node.js on systems where it exists in C).

Could you elaborate on why it is necessary to use the local host's receiving address explicitly as the source address when replying via dgram? This strikes me as a networking anti-pattern, the system is fully capable of routing, over any valid network interface, a packet with a known destination. There is no requirement that the local source be any particular address.

It could be that systemd is adding some special requirement to the mix here, but I don't see it. It sounds like you may have an unworkable intersection of your protocol's very specific desires about source addresses and systemd and the fact that the receive address cannot portably be known for UDP, but I don't see (from your description) a general problem for using node to implement dgram services started by systemd.

[sam-github]

And more practically... if I was absolutely desperate to write this in node on systemd, I'd probably either write my own special purpose dgram addon, or a simple c wrapper that forwarded to a node backend the packets and addressing info. The addon might be easier.

[patrakov]

Could you elaborate on why it is necessary to use the local host's receiving address explicitly as the source address when replying via dgram? ... the system is fully capable of routing, over any valid network interface, a packet with a known destination

Sure. Suppose that you have 192.168.1.144 (primary) and 192.168.1.145 (secondary) on your network interface, and think that you have a default route through 192.168.1.1. However here is what the machine thinks when it needs to send a packet to, say, 192.168.3.123:

# ip route get 192.168.3.123
192.168.3.123 via 192.168.1.1 dev eno1  src 192.168.1.144

It says: I will go through 192.168.1.1 (that's the normal routing), and (that's the unexpected part), unless something says otherwise, use 192.168.1.144 as the source address. It's that "use 192.168.1.144 as the source address" decision that we need to override.

When the application is sending UDP packets that are, logically, replies, the kernel does not, in fact, know (and cannot know) that they are replies! They are just UDP packets. So yes, when normally reaching 192.168.3.123, it makes total sense to use 192.168.1.144 as the source address. However, when replying to a packet that was previously sent to 192.168.1.145, the default source address is wrong, and the reply from it would be rejected on the other end, because it comes from an IP (192.168.1.144) different from the one expected (192.168.1.145).

For TCP, the kernel knows about replies and applies the source address selection correctly and automatically for them.

[sam-github]

the reply from it would be rejected on the other end, because it comes from an IP (192.168.1.144) different from the one expected (192.168.1.145).
For TCP, the kernel knows about replies and applies the source address selection correctly and automatically for them.

Indeed, and by using UDP instead of TCP, it becomes the application protocols responsibility to (re)implement those features of TCP it needs. In this case, correlation of requests and responses, which your protocol apparently doesn't support.

In your example, it seems that this is a problem caused by putting multiple addresses on the same interface.

Anyhow, more to the point, given IP_RECVDSTADDR can only work on Linux, what would you imagine the API to be? How would you hope that node makes this generally not supported feature available?

[patrakov]

Indeed, and by using UDP instead of TCP, it becomes the application protocols responsibility to (re)implement those features of TCP it needs. In this case, correlation of requests and responses, which your protocol apparently doesn't support.

It does support that, but clients actively verify (by using connect() on a UDP socket) that the replies come from the IP address that the request was sent to. Which is what I need to ensure. Normally, I would just create a separate socket for each IP address returned in os.networkInterfaces() or specified in my application's config file, but this fails because os.networkInterfaces() result can legitimately change after the application start (and is almost certainly incomplete during boot, which is not a bug), and systemd guys want my application to deal with it.

Anyhow, more to the point, given IP_RECVDSTADDR can only work on Linux, what would you imagine the API to be? How would you hope that node makes this generally not supported feature available?

I would like to see the rinfo in the "message" event of dgram.Socket extended with a new field. As the feature is indeed not portable to some operating systems, that field would have to be undefined/nonexistent on such systems, and a new boolean has to be added somewhere so that I can know beforehand whether the OS supports this feature. Also, a new version of send() that accepts the desired source address in addition to the existing parameters (and either does not exist or always fails on "bad" operating systems) would be needed.

[davequick]

Another scenario where this is applicable:

Dual nic'd machine: 1.2.3.4 = public side, 192.168.1.1 = rack local side. Write a DHCP server in node.js and run it on your dual nic'd machine - you have to accept broadcast packets but you'd like to know all of them that come in on the NIC related to your rack local side and issue addresses ONLY to those on your rack local side. You're screwed with the current implementation. You don't actually know which nic the incoming broadcast packet actually came from. I guess you could just respond to all incoming DHCP broadcasts to your private side and time out the conversations that don't respond back - but it feels sloppy. Just wanted to add some fuel to the fire. It seems like it would be ideal to find a way to expose the nic the broadcast came from up to the developer writing in node.js.

[bnoordhuis]

Closing, continues in nodejs/node#1649.