Fixed group drift synchronization

[maxsharabayko]

When a group member updates its clock drift value and potentially the TSBPD base time, all other members need to be synchronized to use the same value.

Previously, the minimum drift value and the minimum TSBPD base time among all group member was taken as a reference, which resulted in bad behavior.
Having the improved drift tracer logic (PR #1965) now allows to do a proper synchronization of group members,

Experiment

Network: Win 10 <--> 1Gbps switch <--> CentOS 7.
Network impairments: RTT 40 ms, 1% packet loss.
SRT latency: 120 ms.

Both broadcast group transmission and single path transmission are running in parallel over the same local switch as described below.

Broadcast Group Transmission

(broadcast group sender)
srt-xtransmit generate srt://192.168.0.7:4200 srt://192.168.0.7:5200 --sendrate 10Mbps --enable-metrics

(group receiver)
srt-xtransmit receive srt://:4200 srt://:5200 --enable-metrics --reconnect

Single Path Transmission

(single path sender)
srt-xtransmit generate srt://192.168.0.7:1234 --sendrate 10Mbps --enable-metrics

(single path receiver)
srt-xtransmit receive srt://:1234 --enable-metrics

Results: Broadcast Receiver v1.4.4-rc.1

Experiment duration: ~4 hours.

Plots (click to expand/collapse)

Broadcast Group State

End-to-end Latency, us (min/max/avg): 2112032 / 2123588 / 2116079.
Jitter: 2051us.
Packets received 11717902
Reordered: 0 (dist 0)
Packets Lost (aka dropped): 13989 (0.12%) ❗

Single Socket State

End-to-end Latency, us (min/max/avg): 2235667 / 2235974 / 2235747
Jitter: 14us.
Packets received: 11844550
Reordered: 0 (dist 0)
Packets Lost (aka dropped): 0.

[maxsharabayko]

Results: Broadcast Receiver (This PR)

Experiment duration: ~4 hours.

Plots (click to expand/collapse)

Broadcast Group State

End-to-end Latency, us (min/max/avg): 2339203/2349529/2339264.
Jitter: 19 us.
Packets received 14174985
Reordered: 0 (dist 0)
Packets Lost (aka dropped): 6 (0.00004%) ✔️

Single Socket State

End-to-end Latency, us (min/max/avg): 2338819 / 2338960 / 2338904
Jitter: 10 us.
Packets received: 14175459,
Reordered: 0 (dist 0)
Packets Lost (aka dropped): 21 (0.00015%).

[J-Rogmann]

I did repeat the test with this PR and backup group enabled for a duration of 20h and in parallel a single socket SRT connection. Every hour the main link (enp7s0) with weight=51 was shutdown for 40s and brought up again afterwards.

Results:

Both SRT connection had the same amount of dropped packets at receiver side and performed equally good.

Backup Group

3 dropped packets in total and all appeared at 2021-09-29T01:55:02.236499+0200.

Single Connection

3 dropped packets in total, first at 2021-09-28T17:49:26.678357+0200, second at 2021-09-28T23:19:33.523074+0200 and third at 2021-09-29T12:03:33.217991+0200.

Network Impairments (LANforge)

2 links via enp7s0 and enp8s0 set to 100 Mbps bi-directional bandwidth, RTT=250ms (125 in each direction), 1% bi-directional packet loss on both links.

FLIP                 ---- LanForge  ----                 FLOP

enp7s0 (192.168.4.1) ---- eth2-eth3 ---- (192.168.4.2) enp7s0    (Main link)
enp8s0 (192.168.3.1) ---- eth4-eth5 ---- (192.168.3.2) enp8s0    (Backup link)

main/backup

sender

./srt-xtransmit generate "srt://192.168.4.2:4200?latency=1000&grouptype=backup&weight=51" "srt://192.168.3.2:4201?weight=50" --sendrate 10Mbps --duration 72000s --enable-metrics --statsfile group-snd-stats.csv --statsfreq 1s -v

receiver

./srt-xtransmit receive srt://:4200?latency=1000 srt://:4201 --enable-metrics --metricsfile group-rcv-metrics.csv --metricsfreq 1s --statsfile group-rcv-stats.csv --statsfreq 1s -v --reconnect

check for dropped packets at the receiver side:

cat group-rcv-metrics.csv | cut -d ',' -f7

single path

single path sender

./srt-xtransmit generate "srt://192.168.3.2:4222?latency=1000" --sendrate 10Mbps --duration 72000s --enable-metrics --statsfile single-snd-stats.csv --statsfreq 1s -v

single path receiver

./srt-xtransmit receive "srt://:4222?latency=1000" --enable-metrics --metricsfile single-rcv-metrics.csv --metricsfreq 1s --statsfile single-rcv-stats.csv --statsfreq 1s -v --reconnect

[maxsharabayko]

~26 hours run of broadcast streaming over two links (see the "Experiment" section here).

Packets received 83975852
Reordered: 0 (dist 0)
Packets Lost (aka dropped): 0 ✔️