RFC: batman-adv: directed multicast support

[T-X]

The following changes and patches add multicast-to-multi-unicast support between nodes. It supports IPv6 link-local multicast with up to 16 target nodes and if exceeded, either falls back to flooding (e.g. for important ones like ICMPv6 Neighbor Solicitations) or drops them if the gluon-ebtables-filter-multicast is installed.

This should fix our current issue of larger mesh networks having a significant amount of and overhead through ICMPv6 Neighbor Solicitations.

Recap of what we already have:

• Multicast listener awareness: A node knows which client device is interested in which multicast transmissions (bridge IGMP/MLD snooping, IGMP/MLD querier per node, multicast-capable batman-adv TT). -> Since current Gluon master
• Multicast-to-unicast from node to client: A node delivers multicast packets via individual unicast transmissions (only) to clients interested in it instead of broadcasting. -> Since Gluon v2016.2

This adds the missing piece to have directed, brodcast-less multicast for the whole path from one client to another.

For batman-adv:

• It adds some patches to batman-adv to allow not just multicast-to-single-unicast but also multicast-to-multi-unicast forwarding for one thing.
• For another, adds a patch to batman-adv to allow it to forward a frame via multicast-to-unicast but not via flooding. This is indicated to batman-adv via a "noflood" packet mark.

For gluon-ebtables-filter-multicast:

• Relaxes the filtering rules to allow a packet which would previously be dropped to pass if it were transsmitted via up to 16 unicast transmissions (so to up to 16 nodes in the mesh).

For gluon-mesh-batman-adv:

• Reenables the multicast_mode
• Sets multicast_fanout limit to 16.
• Assigns the marking number for the batman-adv noflood mark

This needs a good amount of (more) testing from other people, therefore sending it as RFC.

Once your community has fully migrated to a Gluon v2018.1 you can test this patchset: v2018.1 has the whole listener side ready. This patch only touches the multicast sender side, so you can add this patchset to dedicated nodes which have the multicast sending devices while receiving on any Gluon >=v2018.1 node.

[T-X]

So far I tested this patchset in the following two scenarios:

Test 1: Virtual environment

Scenario:

• 6 nodes (6 KVM instances)
• line topology

Tested:

• used ping6 to multicast addresses
• verified via tcpdump that multicast packets are transmitted correctly via individual batman-adv unicast packets
• verified that multicast_fanout works, that is that batman-adv correctly falls back to flooding if exceeded
• verified that the "noflood" mark works, that is that if a packet were to exceed multicast_fanout that such packets are correctly dropped instead of flooded.

Test 2: Video Streaming (non-virtual test setup)

Scenario:

• 3 nodes (top: TP-Link WDR4300, middle+bottom: WDR3600; meshed on 2.4GHz only as thick walls give no chance for 5GHz)
• 6 laptops (1x sender + 1x receiver in the cellar, 4x receivers at the top)
• Sender in the cellar transmitts three 640x320px videos simultaneously in a loop, using a dedicated IPv6 link-local multicast address for each video
• Receivers sign up for all three multicast addresses

Screenshot from a client device, running 3x VLC and H.O.R.S.T.

Commands:

• Sender:

$ vlc --loop ./video1.mp4 --sout="#rtp{dst=[ff12::23%enp0s25],port=5023,mux=ts,sap,name=foobar}" --sout-keep
$ vlc --loop ./video2.mp4 --sout="#rtp{dst=[ff12::42%enp0s25],port=5042,mux=ts,sap,name=foobar}" --sout-keep
$ vlc --loop ./video3.mp4 --sout="#rtp{dst=[ff12::123%enp0s25],port=5123,mux=ts,sap,name=foobar}" --sout-keep

• Receiver:

$ vlc "rtp://@[ff12::23%wlan0]:5023"
$ vlc "rtp://@[ff12::42%wlan0]:5042"
$ vlc "rtp://@[ff12::123%wlan0]:5123"

Observations

• Overall, videos were played smoothly during several hours of runtime.
• During midday and early afternoon videos would hang occasionally. "batctl ping" from the top to the bottom node only worked sometimes, batman-adv TQ values dropped severly. After about a minute, things would recover as if nothing had happened. Seemed like some wifi (chip? driver?) issues. Middle node usually had a signal strength of -70 to -80 dBm to the top and cellar one. But during that time the reported signal strength would jump wildly, even to values as good as -55 dBm which given the challenging topology did not seem to make sense at all
• Later only sporadic frame losses
• An "apt-get install" on a laptop 0.5m away from a multicast receving laptop using an "unrelated" wifi access point, even on a different 5GHz channel, would kill the multicasted video playback. Looks like channel 36 and 44 are still causing interference to each other. Also indicates the challenges of a TCP and UDP flow rivaling for and potentially congesting the same medium.

[T-X]

PS: In the video streaming test I also did a brief mobility/handover test. With one of the laptops at the top I moved down to the cellar and back up, briefly stopping at each node.

When moving around the lift before reaching the middle node the streams paused for a few seconds. As expected that metal beast left no chance to wpa_supplicant to perform any smooth handover. Other than that, video playback continued smoothly with only a few video frames lost. At each node I verified that the laptop and its wpa_supplicant had switched to the new AP.

[blocktrron]

In conjunction with Line 423, this seems to have the effect of not delivering the packet to nodes with the want-all flag set in case no other node has the multicast address in the transtable.

[T-X]

Urgh, you are absolutely right, line 423 should return NET_XMIT_SUCCESS instead of NET_XMIT_DROP... will fix this PR later today. Many thanks for spotting and identifying the issue, @blocktrron! And thanks to you and @mweinelt for testing this PR.

[T-X]

Changelog v2:

• Fixed return value in "batadv_mcast_forw_tt_send()" for an empty TT entry: the want-all lists still need to be tried. This fixes potential packetloss. - Discovered by @mweinelt and @blocktrron

[mweinelt]

I can confirm the issues we encountered are fixed in v2. Thanks!

[T-X]

Changelog v3:

• Updated to Gluon master branch (which is basically Gluon v2018.1 right now)

As soon as some community returns some positive feedback for this patchset in their otherwise Gluon v2018.1 based production network network then I will remove the "RFC".

As I said before, no need to apply this patchset on all nodes yet, just some test node(s) is fine as long as (most of) the rest of your network runs v2018.1.

[T-X]

Some useful commands to check whether your network is and which multicast addresses are or are not suitable for tests:

• Number of unsuportive nodes: $ batctl mf | grep " -"
• Number of receiver nodes for a specific multicast address: $ batctl tg -m -H | sed "s/...\([^ ]*\) .*/\1/" | sort | uniq -c | sort -n -r | less

Make sure that for the multicast address you are playing with has a total, summed unsupportive node count plus receiver node count matching your multicast address < 16.

Also, with the VLC example above your multicast receivers should show up in this "number of receivers" table. Also on your multicast receiving host running Linux have a look at $ ip maddr show to find the IPv6 multicast address to MAC multicast address mapping.

[christf]

we need to understand if #1468 is induced by this patch. Happily removing blocked when we know this is not the case.

[mweinelt]

It is unclear if the multicast patches are the cause of the crash in #1468, so this is not blocked but instead requires a decision on how to proceed.

We in Darmstadt have been testing this patch since its inception and had no issues with it in months.

[christf]

The patch is meant to appeal to a wider audience in order to gain momentum for an upstream merge. I find the approach to be valid and I am interested in the feature. It has undergone a review by blocktron and FFDA is running it for months.

As discussed on IRC I see two risks:

1. If merged everyone automatically has the patch. Maybe it would be beneficial to roll this out only when consciously selected
2. We have had the discussion previously whether to accept patches because they must be maintained and we have usually decided for upstreaming patches if possible. This is the goal all along.

I support merging this patch given (1) is addressed. @NeoRaider @rotanid @mweinelt what are your thoughts?

[rotanid]

i'm against merging a non-upstreamed patch like this.
if upstream doesn't want it even though they have been told about the FFDA test results, they will hopefully name reasons. but afaik no one submitted it yet.

[T-X]

Just some background information why I haven't submitted the patches to upstream yet:

Linux network subsystem maintainers are increasingly demanding a "new" interface for configurations (sysfs -> netlink). And have recently expressed this towards batman-adv which does not have a netlink interface for configuration options yet. Therefore any patch introducing new configuration options, including this one, will likely be rejected for now.

@ecsv has provided some initial patches to introduce netlink support for configuration options in batman-adv. However, this will probably need a bit more time. Until then I can't submit this patch upstream.

I'm also absolutely in favor of an "upstream-first" policy. I guess since many communities have switched to a multi-domain setup now, there is no hurry?

Otherwise, if multicast overhead is an urgent issue for some larger communities I guess we could also interpret this pull-request as a fix. And apply it regardless of upstreaming status (and I'm definitely going to upstream). But I currently have no overview of the urgency.

[T-X]

Changelog v4:

• rebased to master (new Patch numbers)
• fixed a double free bug in the batman-adv multicast-to-multi-unicast patch -> led to crashes, especially on netifd restart/shutdown (crash in nf_ct_del_from_dying_or_unconfirmed_list: "Kernel bug detected [...] #1468)
• changed the default multicast_fanout from 1 to 16 in the batman-adv multicast-to-multi-unicast patch (no functional change, as it was set to 16 by the netifd scripts, too; just to sync with the proposal that was sent to the batman-adv mailing list)

[T-X]

Changelog v5:

• fixed skb memory leaks in error cases

[T-X]

Changelog v6:

• rebased to master (no changes/conflicts)
• fixed the batadv_mcast_forw_tt_send() again, which was fixed in v2 but reintroduced by v4

[ecsv]

The fanout change will be part of the batman-adv 2019.2 release

[rotanid]

@T-X one of your commit messages should read "gluon-mesh-batman-adv:" instead of "batman-adv:"

@ecsv is "the fanout change" the whole upstream part of this PR?

[ecsv]

@rotanid No, there are also a noflood_mark which is not upstream. I am unsure about its usefulness in upstream batman-adv since it is rather tied to the gluon specific 355-mcast-drop. But we also have this ap_isolation which is rather similar.

[mweinelt]

batman-adv-legacy was dropped after v2019.1 was branched.

[mweinelt]

Superseded by #2206 and #2209.