Broadcast Address Incorrectly Set on /31 Interfaces

[bensallen]

RancherOS Version: (ros os version) 1.5.2

Where are you running RancherOS? Bare-metal.

Due to #1863 and the upstream change vishvananda/netlink#248, interfaces with a /31 subnet have a broadcast address automatically calculated and assigned. Since /31 subnets are for point to point links, the network and broadcast addresses are used by the two end points.

With the auto-assigned broadcast address, /31 addressed interfaces do not successfully pass traffic.

It appears an upstream PR to correct the behavior has been proposed here: vishvananda/netlink#496.

/31 links are useful to minimize IP address usage, in a layer 3 topology from leaf switch to hosts.

Example of current behavior:

4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:83:52:f2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.98.2/31 brd 192.168.98.3 scope global eth2
       valid_lft forever preferred_lft forever

Example of correct config via:

$ ip addr del 192.168.98.2/31 brd 192.168.98.3 dev eth2
$ ip addr add 192.168.98.2/31 dev eth2

4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 52:54:00:83:52:f2 brd ff:ff:ff:ff:ff:ff
    inet 192.168.98.2/31 scope global eth2
       valid_lft forever preferred_lft forever

[kingsd041]

Tested with RancherOS v1.5.5-rc1

Use the following cloud-config.yaml config to install to disk:

#cloud-config
rancher:
  network:
    interfaces:
      eth0:
        dhcp: true
      eth1:
        dhcp: true
      eth2:
        dhcp: false
        address: 192.168.1.91/31

After waiting for Rancheros to restart, check the eth2 broadcast address

[root@rancher ~]# ifconfig eth2
eth2      Link encap:Ethernet  HWaddr 08:00:27:20:07:0D
          inet addr:192.168.1.91  Bcast:0.0.0.0  Mask:255.255.255.254
          inet6 addr: fe80::a00:27ff:fe20:70d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:47 errors:0 dropped:0 overruns:0 frame:0
          TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:4724 (4.6 KiB)  TX bytes:936 (936.0 B)

It can be seen from the results that eth2 does not automatically assign a broadcast address