Promiscuous mode on Raspberry Pi 3 B+ not seeing packets

[dgriswo]

On a Raspberry Pi model 3 B+, when the wired ethernet adapter is placed into promiscuous mode using "ifconfig eth0 promisc", and connected to a monitor port, the RX packet counters do not increase. Packet processing using Bro, iftop, or tcpdump does not see incoming packets. I have tested this on two different Raspberry Pi 3 B+ boards using both a Unifi switch and a Mikrotik switch.

This function works as expected on a Raspberry Pi Model 3 B. If fact, I can build an SD card for a 3 B+, verify functionality isn't working, then plug the SD card into a 3 B and verify functionality is working.

Steps to reproduce.

1. Install 2018-04-18-raspbian-stretch-lite on a Raspberry Pi 3 B+
2. Attach the Pi to a Switch monitor/mirror port, or a hub
3. Add the following to /etc/dhcpcd.conf

static
interface eth0
static ip_address=0.0.0.0

4. Add the following to /etc/sysctl.conf
   net.ipv6.conf.all.disable_ipv6 = 1
5. Reboot, or run sudo systemctl restart dhcpcd; sudo sysctl -p
6. Run "ifconfig eth0 promisc"
7. Run "tcpdump -i eth0"

[JamesH65]

That would indicate a problem in the firmware of the wireless chip itself. That will need to be reported to CYpress, I can do that when I get back in to the office.

[dgriswo]

This issue is with the Microchip LAN7515 wired network device, not the Broadcom/Cypress wireless device.

[pelwell]

Are you saying you receive nothing at all - not even broadcast packets? What does ethtool eth0 report?

[JamesH65]

@dgriswo Apologies, got the wrong adapter. We do use a different driver for that chip, so there could well be a driver issue, rather than firmware.

[dgriswo]

@pelwell The network points that I am capturing are between core and Internet routers; the only broadcast I would expect to see is ARP. The majority of the packets are unicast to/from the Internet. IIRC, the counter was zero. Not sure how long the ARP cache is.

I’ll try to retest capture on a port with SMB broadcasts.

[pelwell]

Testing promiscuity is more difficult in an age where hubs are hard to find, but by capturing a packet trace on one machine and replaying it from a Pi over a point-to-point network cable to a 3B+ I'm fairly happy that promiscuous mode is working. sudo tcpdump -p ... receives roughly half as many packets as sudo tcpdump ..., and the packets absent from the non-promiscuous capture appear to be between other hosts or to unwanted multicast addresses.

[JamesH65]

@dgriswo Can you double check that promiscuous mode is not working for you? Can you try with the latest kernel? Been a few changes in networking.

[dgriswo]

@JamesH65 I was able to get a 3B swapped for a 3B+ to test again. I believe I've isolated it down to a dhcpcd configuration.

in /etc/dhcpcd.conf, I use the following:

static
interface eth0
static ip_address=0.0.0.0

on the 3B, no IP is assigned to the interface and packets are processed.

eth0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 9000
        ether b8:27:eb:5c:95:ab  txqueuelen 1000  (Ethernet)
        RX packets 8939  bytes 5170687 (4.9 MiB)
        RX errors 0  dropped 7  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

on the 3B+, no IP is assigned to the interface, but packets are not processed.

eth0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 9000
        ether b8:27:eb:97:35:ee  txqueuelen 1000  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 0  bytes 0 (0.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

commenting out the lines in /etc/dhcpcd.conf on the 3B+, an APIPA address is assigned to eth0 (the switch port is egress only.) and packets are processed.

eth0: flags=4419<UP,BROADCAST,RUNNING,PROMISC,MULTICAST>  mtu 9000
        inet 169.254.72.132  netmask 255.255.0.0  broadcast 169.254.255.255
        ether b8:27:eb:97:35:ee  txqueuelen 1000  (Ethernet)
        RX packets 46795  bytes 45341125 (43.2 MiB)
        RX errors 0  dropped 6  overruns 0  frame 0
        TX packets 19  bytes 3396 (3.3 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

Is this a dhcpcd issue or a driver issue?

[JamesH65]

Why do you use static ip_address=0.0.0.0? That a specially defined address, so I am not sure what you intention is here. It's possible the implementation of DHCP has changed the way it works when using the 0.0.0.0 address.

[dgriswo]

dhcpcd manages the interfaces in Raspbian Stretch. I'm trying to bring the interface up without an IP address. Traditional networking used for promiscuous devices was ifconfig $IFACE 0.0.0.0 up; ifconfig $IFACE promisc I guess I'm trying to use the distribution standard instead of falling back to traditional network.

On a Pi 3B, I get no IP address and RX packets only. On the Pi 3B+, I either get no IP Address and no packets or an apipa address and both TX and RX packets.

[pelwell]

I'm investigating.

[pelwell]

I can confirm the report - with ipv6 disabled (ipv6.disabled=1 in cmdline.txt), and an invalid static IP address, the Ethernet interface on a 3B receives packets but not on a 3B+.

[pelwell]

This is baffling. Tracing the writes to the LAN7800 and PHY registers I can compare the final state when booting with and without a valid IP address, and apart from an additional multicast MAC address (mDNS) with a valid IP address I can't any difference, apart from a read-only status bit indicating that a pending transmission.

With a bit of luck Microchip will be able to shed some light.

[dgriswo]

@pelwell any news from Microchip?

[pelwell]

No, nothing. I've prodded them again.

[pelwell]

I think this issue has the same underlying cause as #2617, i.e. that the bottom half tasklet isn't scheduled until the first packet is transmitted. The next rpi-update release should include a fix.

[popcornmix]

Latest rpi-update has the fixes @pelwell refers too. Please update and test.

[aviramha]

I ran rpi-update and the issue still occurs for me. Doesn't happen on the other ethernet interface I have (USB one)

[dgriswo]

@aviramha The rpi-update from #2617 corrected the original issue. The latest raspbian stretch kernel is 4.19.42-v7+ and includes the above fix. (I think 4.17+ includes it.)

If you are on the latest version, I'd suggest opening a new issue, as this one is resolved.