RTMP acknowledge overflow causes the encoder to stop streaming

[winlinvip]

ACK SIZE refers to the number of bytes that the PEER should send an Acknowledge message after receiving. In other words, it indicates the number of bytes that the PEER has received.
RTMP is 32, which means it is 4 bytes long, but it does not explicitly state how overflow should be handled, whether it is int32 or uint32.

5.3. Acknowledgement (3)
   The client or the server sends the acknowledgment to the peer after
   receiving bytes equal to the window size. The window size is the
   maximum number of bytes that the sender sends without receiving
   acknowledgment from the receiver. The server sends the window size to
   the client after application connects. This message specifies the
   sequence number, which is the number of the bytes received so far.
0         1         2         3
01234567890123456789012345678901 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|    sequence number (4 byte)   | 
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
Figure 4 shows the payload for the protocol message 'Acknowledgement'.
   sequence number: 32 bits
     This field holds the number of bytes received so far.

Previously, the bitrate of videos was very low, such as 100kbps, so there would be no overflow issue with these 4 bytes. Now, the bitrate is 80 to 100 times higher, which easily leads to overflow. Some encoders have different rules for handling this overflow, causing them to receive incorrect values and mistakenly assume that the server has not acknowledged, resulting in re-streaming.

TRANS_BY_GPT3

[winlinvip]

Gao Sheng has encountered something similar, but did not explain it clearly: #598

TRANS_BY_GPT3

[winlinvip]

Play a stream with a bitrate of 8Mbps using Flash player, set the ack size to 10*1024*1024, and print out the acknowledge messages sent by the Flash player.

`ack=1` indicates an acknowledge message.
`seq` is the sequence number, which represents the number of bytes received by the client.
`sent` is the number of bytes displayed as transmitted by the server.

[2017-01-06 11:23:30.321][warn][13607][114][35] ack=1, seq=0x500ac2, sent=0x5766fb, diff=482361
[2017-01-06 11:23:35.493][warn][13607][114][35] ack=1, seq=0xa0192a, sent=0xa0f73a, diff=56848
[2017-01-06 11:23:39.153][warn][13607][114][35] ack=1, seq=0xf0229e, sent=0xf4e6bb, diff=312349
[2017-01-06 11:23:43.197][warn][13607][114][35] ack=1, seq=0x140241e, sent=0x140499e, diff=9600

The ack size is set to 10*1024*1024, which means an acknowledge message should be sent after the player has played 10MB of data.
For a flash player, the interval between sending messages, which is the difference in seq, is 0xa0192a-0x500ac2=5246568=5*1024*1024, which means it sends approximately half of the ack size.
Subsequently, it sends acknowledgements after receiving 5.002307891845703MB and 5.0003662109375MB, which is roughly half of the window size.

The following messages are:

[2017-01-06 11:50:16.010][warn][13607][114][35] ack=1, seq=0x68ba6930, sent=0x68c14790, diff=450144
[2017-01-06 11:50:20.063][warn][13607][114][35] ack=1, seq=0x690a7297, sent=0x690d58ae, diff=189975
[2017-01-06 11:50:23.727][warn][13607][114][35] ack=1, seq=0x695a79a2, sent=0x695dafa8, diff=210438
[2017-01-06 11:50:28.507][warn][13607][114][35] ack=1, seq=0x69aa7fe2, sent=0x69b1d0f1, diff=479503

The respective values are: 5.00229549407959, 5.0017194747924805, 5.00152587890625.

Therefore, it can be confirmed that the behavior of the flash player is to send an acknowledge confirmation message to the server when it receives more than 1/2 of the ack size (window size).

TRANS_BY_GPT3

[winlinvip]

Regarding the server side, my colleague has tested two of them:

FMS: 0xee77d0a7
NGINX-RTMP: 0xf0000000

Once these two thresholds are exceeded, the server will reset the counter to 0.

NGINX-RTMP code: https://github.com/arut/nginx-rtmp-module/blob/c0bf381d10de05c135f913921c58272838d5e1ee/ngx_rtmp_handler.c#L275

            if (s->in_bytes >= 0xf0000000) {
                ngx_log_debug0(NGX_LOG_DEBUG_RTMP, c->log, 0,
                               "resetting byte counter");
                s->in_bytes = 0;
                s->in_last_ack = 0;
            }

This number has no basis and there is no comment explaining its source.
In nginx-rtmp, the ack is sent when it exceeds the ack window (unlike Flash, which sends it when it exceeds half the window).

            if (s->ack_size && s->in_bytes - s->in_last_ack >= s->ack_size) {

                s->in_last_ack = s->in_bytes;

                ngx_log_debug1(NGX_LOG_DEBUG_RTMP, c->log, 0,
                        "sending RTMP ACK(%uD)", s->in_bytes);

                if (ngx_rtmp_send_ack(s, s->in_bytes)) {
                    ngx_rtmp_finalize_session(s);
                    return;
                }
            }

TRANS_BY_GPT3

[winlinvip]

About the handling of flash player overflow, the log is as follows:

[2017-01-06 12:24:16.129][warn][13607][114][35] ack=1, seq=0xed47b9cd, sent=0xed49da95, diff=139464
[2017-01-06 12:24:20.549][warn][13607][114][35] ack=1, seq=0xed97c0e6, sent=0xed992be4, diff=92926
[2017-01-06 12:24:26.044][warn][13607][114][35] ack=1, seq=0xede7c793, sent=0xeded7128, diff=371093
[2017-01-06 12:24:29.706][warn][13607][114][35] ack=1, seq=0xee37cff6, sent=0xee3d82a9, diff=373427
[2017-01-06 12:24:34.113][warn][13607][114][35] ack=1, seq=0xee87dca3, sent=0xee90a823, diff=576384
[2017-01-06 12:24:37.781][warn][13607][114][35] ack=1, seq=0xeed7e36a, sent=0xeedd0cbf, diff=338261
[2017-01-06 12:24:42.201][warn][13607][114][35] ack=1, seq=0xef27ed1a, sent=0xef2e1b6d, diff=405075
[2017-01-06 12:24:46.589][warn][13607][114][35] ack=1, seq=0xef77f9ad, sent=0xef7d5424, diff=350839
[2017-01-06 12:24:51.016][warn][13607][114][35] ack=1, seq=0xefc806b8, sent=0xefc8896a, diff=33458
[2017-01-06 12:24:55.432][warn][13607][114][35] ack=1, seq=0x500eb7, sent=0xf01a1a8a, diff=-271971373
[2017-01-06 12:25:00.178][warn][13607][114][35] ack=1, seq=0xa01113, sent=0xf06da792, diff=-271739265
[2017-01-06 12:25:04.577][warn][13607][114][35] ack=1, seq=0xf0186e, sent=0xf0bc6d49, diff=-271821605
[2017-01-06 12:25:09.373][warn][13607][114][35] ack=1, seq=0x140210b, sent=0xf10fe9e5, diff=-271595302
[2017-01-06 12:25:12.673][warn][13607][114][35] ack=1, seq=0x19021c6, sent=0xf15a736a, diff=-271953500
[2017-01-06 12:25:17.435][warn][13607][114][35] ack=1, seq=0x1e0287d, sent=0xf1aa3d1b, diff=-271969122
[2017-01-06 12:25:22.213][warn][13607][114][35] ack=1, seq=0x2302f34, sent=0xf1f9d01b, diff=-271998745
[2017-01-06 12:25:26.996][warn][13607][114][35] ack=1, seq=0x2803a9b, sent=0xf24c3340, diff=-271845211
[2017-01-06 12:25:31.382][warn][13607][114][35] ack=1, seq=0x2d03ed4, sent=0xf299ecd1, diff=-271995395
[2017-01-06 12:25:36.534][warn][13607][114][35] ack=1, seq=0x3204cee, sent=0xf2f33cf8, diff=-271388662
[2017-01-06 12:25:40.566][warn][13607][114][35] ack=1, seq=0x37053d6, sent=0xf33de12b, diff=-271741611
[2017-01-06 12:25:45.332][warn][13607][114][35] ack=1, seq=0x3c05a44, sent=0xf38b265b, diff=-271922153

The overflowing data is padded with either 32 bits or 31 bits, which is significantly different from what flash sends. If we directly set it to zero, it can match, just like in nginx-rtmp, where it is set to zero when it exceeds 0xf0000000.

TRANS_BY_GPT3

[winlinvip]

This problem has only two solutions:

1. If the accumulation of acknowledge exceeds 0xf0000000, it is set to zero.
2. Send an acknowledge message if it exceeds half of the ack size.

Supported in version 2.0.225.

TRANS_BY_GPT3

[winlinvip]

In addition, some encoders do not send an ack size (usually indicating that they do not need to send an acknowledge confirmation message), but they still want the server to send a confirmation message. Therefore, it is necessary to support configuration that can change the ack size for both input and output.

Supported in 3.0.13.

TRANS_BY_GPT3