Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HLS: SRS2 processes audio-only streams into HLS format, experiencing lag and static noise issues on Apple systems. #512

Closed
lewang321 opened this issue Oct 26, 2015 · 50 comments
Assignees
Labels
Bug It might be a bug. TransByAI Translated by AI/GPT.
Milestone

Comments

@lewang321
Copy link

lewang321 commented Oct 26, 2015

When slicing the pure audio stream with srs_2.0 into HLS, if the configuration option hls_vcodec vn; hls_acodec aac; is selected, there will be no video information in the PMT of ts. However, when playing the audio using Apple iOS system, there will be stuttering and noise, while it plays normally on Android system.

hls.conf configuration file:

listen              1935;
max_connections     1000;
vhost __defaultVhost__ {
    hls {
        enabled         on;
        hls_fragment    5;
        hls_td_ratio    1.5;
        hls_aof_ratio  1.0;
        hls_on_error    ignore;
        hls_storage     disk;
#        hls_ts_floor    off;
        hls_acodec      aac;
        hls_vcodec      vn;
#        hls_cleanup     on;
#        hls_nb_notify   64;
#        hls_wait_keyframe       off;
        hls_window      60;
        hls_path        ./objs/nginx/html;
        hls_m3u8_file   [app]/[stream].m3u8;
        hls_ts_file     [app]/[stream]-[seq].ts;
    }

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 26, 2015

Topology is RTMP ----> SRS -----> IOS.

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 26, 2015

The RTMP stream is pushed to SRS through FMLE. It is in the pure audio AAC format.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Oct 27, 2015

What is the sampling rate of the SRS logs and audio?

TRANS_BY_GPT3

@winlinvip winlinvip added the Bug It might be a bug. label Oct 27, 2015
@winlinvip winlinvip added this to the srs 2.0 release milestone Oct 27, 2015
@lewang321
Copy link
Author

[2015-10-27 10:08:08.715][trace][26340][0] srs checking config...
[2015-10-27 10:08:08.715][trace][26340][0] detect intranet address: 127.0.0.1, ifname=lo
[2015-10-27 10:08:08.715][trace][26340][0] retrieve local ipv4 ip=10.12.4.61, index=0
[2015-10-27 10:08:08.716][trace][26340][0] detect intranet address: 10.12.4.61, ifname=eth0
[2015-10-27 10:08:08.716][trace][26340][0] retrieve local ipv4 ip=172.17.42.1, index=1
[2015-10-27 10:08:08.716][trace][26340][0] detect intranet address: 172.17.42.1, ifname=docker0
[2015-10-27 10:08:08.716][warn][26340][0][0] stats network use index=0, ip=10.12.4.61
[2015-10-27 10:08:08.716][warn][26340][0][0] stats disk not configed, disk iops disabled.
[2015-10-27 10:08:08.716][trace][26340][0] write log to file ./objs/srs.log
[2015-10-27 10:08:08.716][trace][26340][0] you can: tailf ./objs/srs.log
[2015-10-27 10:08:08.716][trace][26340][0] @see: https://github.com/simple-rtmp-server/srs/wiki/v1_CN_SrsLog
[2015-10-27 10:08:08.716][trace][26340][0] srs(simple-rtmp-server) 2.0.195
[2015-10-27 10:08:08.716][trace][26340][0] license: The MIT License (MIT), Copyright (c) 2013-2015 SRS(simple-rtmp-server)
[2015-10-27 10:08:08.716][trace][26340][0] primary/master: SRS/1.0release
[2015-10-27 10:08:08.716][trace][26340][0] authors: winlin,wenjie.zhao
[2015-10-27 10:08:08.716][trace][26340][0] contributors: winlin<winlin@vip.126.com> wenjie.zhao<740936897@qq.com> xiangcheng.liu<liuxc0116@foxmail.com> naijia.liu<youngcow@youngcow.net> alcoholyi<alcoholyi@qq.com> byteman<wangchen2011@gmail.com> chad.wang<chad.wang.cn@gmail.com> suhetao<suhetao@gmail.com> Johnny<fengjihu@163.com> karthikeyan<keyanmca@gmail.com> StevenLiu<lq@chinaffmpeg.org> zhengfl<zhengfl_1989@126.com> tufang14<breadbean1449@gmail.com> allspace<allspace@gmail.com> niesongsong<nie950@gmail.com> rudeb0t<nimrod@themanxgroup.tw> 
[2015-10-27 10:08:08.716][trace][26340][0] uname: Linux MediaWiki 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[2015-10-27 10:08:08.716][trace][26340][0] build: 2015-10-26 16:49:02, little-endian
[2015-10-27 10:08:08.716][trace][26340][0] configure:  --disable-all --with-ssl --with-hls --with-nginx --with-ffmpeg --with-transcode
[2015-10-27 10:08:08.716][trace][26340][0] features: --prefix=/usr/local/srs --with-hls --without-hds --without-dvr --with-nginx --with-ssl --with-ffmpeg --with-transcode --without-ingest --without-stat --without-http-callback --without-http-server --without-stream-caster --without-http-api --without-librtmp --without-research --without-utest --without-gperf --without-gmc --without-gmp --without-gcp --without-gprof --without-arm-ubuntu12 --without-mips-ubuntu12 --log-trace
[2015-10-27 10:08:08.716][trace][26340][0] conf: conf/hls.conf, limit: 1000
[2015-10-27 10:08:08.716][trace][26340][0] writev limits write 1024 iovs a time
[2015-10-27 10:08:08.716][warn][26340][0][0] SRS 2.0.195 is not stable, please use stable branch 1.0release instead
[2015-10-27 10:08:08.716][trace][26340][0] cpu is amd64, glibc 2.12
[2015-10-27 10:08:08.716][trace][26340][0] cpu is x86_64, glibc 2.12
[2015-10-27 10:08:08.716][trace][26340][0] check feature rtmp handshake: on
[2015-10-27 10:08:08.716][trace][26340][0] check feature hls: on
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature hds: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature http callback: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature http api: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature http server: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature http parser: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature dvr: off
[2015-10-27 10:08:08.716][trace][26340][0] check feature transcode: on
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature ingest: off
[2015-10-27 10:08:08.716][warn][26340][0][0] check feature system stat: off
[2015-10-27 10:08:08.716][trace][26340][0] check feature compile nginx: on
[2015-10-27 10:08:08.716][trace][26340][0] check feature compile ffmpeg: on
[2015-10-27 10:08:08.716][warn][26340][0][0] stream caster: off
[2015-10-27 10:08:08.716][trace][26340][0] MR(merged-read): on, @see https://github.com/simple-rtmp-server/srs/issues/241
[2015-10-27 10:08:08.716][trace][26340][0] MR(merged-read) default 0 sleep 350
[2015-10-27 10:08:08.716][trace][26340][0] MW(merged-write) default sleep 350
[2015-10-27 10:08:08.716][trace][26340][0] read chunk stream cache cid [0, 16)
[2015-10-27 10:08:08.716][trace][26340][0] default gop cache 1, play queue 30s
[2015-10-27 10:08:08.716][trace][26340][0] complex send algorithm enabled.
[2015-10-27 10:08:08.716][warn][26340][0][0] TCP_NODELAY enabled, may hurts performance.
[2015-10-27 10:08:08.716][trace][26340][0] auto guess socket send buffer by merged write
[2015-10-27 10:08:08.716][trace][26340][0] system default latency in ms: mw(0-350) + mr(0-350) + play-queue(0-30000)
[2015-10-27 10:08:08.716][trace][26340][0] start deamon mode...
[2015-10-27 10:08:08.716][trace][26341][0] father process exit. ret=0
[2015-10-27 10:08:08.716][trace][26342][0] son(deamon) process running.
[2015-10-27 10:08:08.716][trace][26342][0] st_set_eventsys to epoll
[2015-10-27 10:08:08.717][trace][26342][0] st_init success, use epoll
[2015-10-27 10:08:08.717][trace][26340][0] grandpa process exit.
[2015-10-27 10:08:08.717][trace][26342][100] server main cid=100
[2015-10-27 10:08:08.717][trace][26342][100] write pid=26342 to ./objs/srs.pid success!
[2015-10-27 10:08:08.727][trace][26342][100] RTMP listen at tcp://0.0.0.0:1935, fd=8
[2015-10-27 10:08:08.727][trace][26342][100] signal installed
[2015-10-27 10:08:10.605][trace][26342][104] RTMP client ip=10.16.0.3
[2015-10-27 10:08:10.605][trace][26342][104] srand initialized the random.
[2015-10-27 10:08:10.607][trace][26342][104] simple handshake success.
[2015-10-27 10:08:10.607][trace][26342][104] connect app, tcUrl=rtmp://10.12.4.61/live, pageUrl=, swfUrl=rtmp://10.12.4.61/live, schema=rtmp, vhost=__defaultVhost__, port=1935, app=live, args=null
[2015-10-27 10:08:10.607][trace][26342][104] out chunk size to 60000
[2015-10-27 10:08:10.845][trace][26342][104] client identified, type=fmle-publish, stream_name=mytest1, duration=-1.00
[2015-10-27 10:08:10.855][trace][26342][104] source url=/live/mytest1, ip=10.16.0.3, cache=1, is_edge=0, source_id=-1[-1]
[2015-10-27 10:08:11.041][trace][26342][104] hls: win=60.00, frag=5.00, prefix=, path=./objs/nginx/html, m3u8=[app]/[stream].m3u8, ts=[app]/[stream]-[seq].ts, aof=1.00, floor=0, clean=1, waitk=1, dispose=0
[2015-10-27 10:08:11.052][trace][26342][104] start publish mr=0/350, p1stpt=20000, pnt=20000, tcp_nodelay=0, rtcid=106
[2015-10-27 10:08:11.382][trace][26342][104] got metadata
[2015-10-27 10:08:11.382][trace][26342][104] 4B audio sh, codec(10, profile=LC, 2channels, 0kbps, 44100HZ), flv(16bits, 2channels, 44100HZ)
[2015-10-27 10:08:11.382][trace][26342][104] input chunk size to 366
[2015-10-27 10:08:11.386][trace][26342][104] input chunk size to 398
[2015-10-27 10:08:11.386][trace][26342][104] audio drop ts segment, sequence_no=0, uri=mytest1-0.ts, duration=59839.19, start=0
[2015-10-27 10:08:11.391][trace][26342][104] input chunk size to 410
[2015-10-27 10:08:11.396][trace][26342][104] input chunk size to 501
[2015-10-27 10:08:14.896][trace][26342][104] input chunk size to 505
[2015-10-27 10:08:15.399][trace][26342][104] input chunk size to 514
[2015-10-27 10:08:16.394][trace][26342][104] input chunk size to 528
[2015-10-27 10:08:22.397][trace][26342][104] input chunk size to 539
[2015-10-27 10:08:32.885][trace][26342][104] input chunk size to 545
[2015-10-27 10:08:51.052][trace][26342][104] <- CPB time=20001, okbps=0,0,0, ikbps=129,129,0, mr=0/350, p1stpt=20000, pnt=20000

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

FMLE configuration is as follows: format: AAC, Channels: Stereo, Sample Rate: 44100 Hz, Bit Rate: 128 Kbps.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Oct 27, 2015

Try reducing the bitrate to 64. Also, please send me the slices and m3u8 file so that I can package and transfer them to the cloud storage.

TRANS_BY_GPT3

@lewang321
Copy link
Author

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

After adjustment, modify the FMLE configuration to:

Format: AAC
Channels: Stereo
Sample Rate: 44100 Hz
Bit Rate: 48 Kbps

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

There is still a regular noise 'pupu' when playing pure audio on Apple systems.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Oct 27, 2015

RTMP to TS audio explosion, the reason is because the TBN of RTMP is 1/1000, which is in milliseconds, but TS is 1/90000, which is in 1/90 milliseconds. Therefore, TS has a higher resolution.
To convert from lower resolution to higher resolution, the timestamps need to be recalculated. This is available in SRS1/nginx-rtmp, but the calculation seems to be incomplete. SRS2 does not calculate it and directly uses the RTMP timestamps.
This may cause audio explosion issues, typically occurring once per slice. Therefore, if the slice can be increased to 10 or 30 seconds, the impact will be minimal.

However, I will fix it and see how to solve it most effectively. You can increase the size of the slice to confirm if this is the issue.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Oct 27, 2015

I listened carefully, and there is a slight pause every 5 seconds, but the static noise from the microphone is constant.
Can you also record the RTMP stream and send me both the RTMP and corresponding TS files?

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Oct 27, 2015

I found out that there has been too much emphasis on PCR, actually it is not necessary.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Oct 27, 2015

Every PCR causes a timestamp error between RTMP and TS, resulting in crackling sound and significant gaps between segments.

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

C:\Desktop\rtmpdump-2.4-git-010913-windows>rtmpdump.exe -r rtmp://10.12.4.61:1935/live/mytest -v -B 10 -o out.flv
RTMPDump v2.4
(c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL
Connecting ...
INFO: Connected...
Starting Live Stream
For duration: 10.000 sec
INFO: Metadata:
INFO:   presetname            Custom
INFO:   creationdate          Tue Oct 27 11:01:12 2015
INFO:   audiodevice           IN  1+2 (Digigram PCX924e)
INFO:   audiosamplerate       44100.00
INFO:   audiochannels         2.00
INFO:   audioinputvolume      59.00
INFO:   audiocodecid          mp4a
INFO:   audiodatarate         48.00
INFO:   server                SRS/2.0.195(ZhouGuowen)
INFO:   srs_primary           SRS/1.0release
INFO:   srs_authors           winlin,wenjie.zhao
INFO:   server_version        2.0.195
64.294 kB / 9.70 sec
ERROR: RTMP_ReadPacket, failed to read RTMP packet header
66.581 kB / 10.05 sec
Download complete

@lewang321
Copy link
Author

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

The above was recorded using the rtmpdump tool for rtmp streams.

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

I adjusted the hls.conf configuration as follows:

listen              1935;
max_connections     1000;
vhost **defaultVhost** {
    hls {
        enabled         on;
        hls_fragment    30;
        hls_window      60;
        hls_acodec      aac;
        hls_vcodec      vn;
        hls_aof_ratio 1.0;
        hls_path        ./objs/nginx/html;
        hls_m3u8_file   [app]/[stream].m3u8;
        hls_ts_file     [app]/[stream]-[seq].ts;
    }

Note: The hls_fragment parameter has been changed to 30.

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

Here are the current startup logs and environment:

[2015-10-27 11:59:25.257][trace][26675][0] srs checking config...
[2015-10-27 11:59:25.257][trace][26675][0] detect intranet address: 127.0.0.1, ifname=lo
[2015-10-27 11:59:25.257][trace][26675][0] retrieve local ipv4 ip=10.12.4.61, index=0
[2015-10-27 11:59:25.257][trace][26675][0] detect intranet address: 10.12.4.61, ifname=eth0
[2015-10-27 11:59:25.258][trace][26675][0] retrieve local ipv4 ip=172.17.42.1, index=1
[2015-10-27 11:59:25.258][trace][26675][0] detect intranet address: 172.17.42.1, ifname=docker0
[2015-10-27 11:59:25.258][warn][26675][0][0] stats network use index=0, ip=10.12.4.61
[2015-10-27 11:59:25.258][warn][26675][0][0] stats disk not configed, disk iops disabled.
[2015-10-27 11:59:25.258][trace][26675][0] write log to file ./objs/srs.log
[2015-10-27 11:59:25.258][trace][26675][0] you can: tailf ./objs/srs.log
[2015-10-27 11:59:25.258][trace][26675][0] @see: https://github.com/simple-rtmp-server/srs/wiki/v1_CN_SrsLog
[2015-10-27 11:59:25.258][trace][26675][0] srs(simple-rtmp-server) 2.0.195
[2015-10-27 11:59:25.258][trace][26675][0] license: The MIT License (MIT), Copyright (c) 2013-2015 SRS(simple-rtmp-server)
[2015-10-27 11:59:25.258][trace][26675][0] primary/master: SRS/1.0release
[2015-10-27 11:59:25.258][trace][26675][0] authors: winlin,wenjie.zhao
[2015-10-27 11:59:25.258][trace][26675][0] contributors: winlinwinlin@vip.126.com wenjie.zhao740936897@qq.com xiangcheng.liuliuxc0116@foxmail.com naijia.liuyoungcow@youngcow.net alcoholyialcoholyi@qq.com bytemanwangchen2011@gmail.com chad.wangchad.wang.cn@gmail.com suhetaosuhetao@gmail.com Johnnyfengjihu@163.com karthikeyankeyanmca@gmail.com StevenLiulq@chinaffmpeg.org zhengflzhengfl_1989@126.com tufang14breadbean1449@gmail.com allspaceallspace@gmail.com niesongsongnie950@gmail.com rudeb0tnimrod@themanxgroup.tw 
[2015-10-27 11:59:25.258][trace][26675][0] uname: Linux MediaWiki 2.6.32-504.el6.x86_64 #1 SMP Wed Oct 15 04:27:16 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
[2015-10-27 11:59:25.258][trace][26675][0] build: 2015-10-26 16:49:02, little-endian
[2015-10-27 11:59:25.258][trace][26675][0] configure:  --disable-all --with-ssl --with-hls --with-nginx --with-ffmpeg --with-transcode
[2015-10-27 11:59:25.258][trace][26675][0] features: --prefix=/usr/local/srs --with-hls --without-hds --without-dvr --with-nginx --with-ssl --with-ffmpeg --with-transcode --without-ingest --without-stat --without-http-callback --without-http-server --without-stream-caster --without-http-api --without-librtmp --without-research --without-utest --without-gperf --without-gmc --without-gmp --without-gcp --without-gprof --without-arm-ubuntu12 --without-mips-ubuntu12 --log-trace
[2015-10-27 11:59:25.258][trace][26675][0] conf: conf/hls.conf, limit: 1000
[2015-10-27 11:59:25.258][trace][26675][0] writev limits write 1024 iovs a time
[2015-10-27 11:59:25.258][warn][26675][0][0] SRS 2.0.195 is not stable, please use stable branch 1.0release instead
[2015-10-27 11:59:25.258][trace][26675][0] cpu is amd64, glibc 2.12
[2015-10-27 11:59:25.258][trace][26675][0] cpu is x86_64, glibc 2.12
[2015-10-27 11:59:25.258][trace][26675][0] check feature rtmp handshake: on
[2015-10-27 11:59:25.258][trace][26675][0] check feature hls: on
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature hds: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature http callback: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature http api: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature http server: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature http parser: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature dvr: off
[2015-10-27 11:59:25.258][trace][26675][0] check feature transcode: on
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature ingest: off
[2015-10-27 11:59:25.258][warn][26675][0][0] check feature system stat: off
[2015-10-27 11:59:25.258][trace][26675][0] check feature compile nginx: on
[2015-10-27 11:59:25.258][trace][26675][0] check feature compile ffmpeg: on
[2015-10-27 11:59:25.258][warn][26675][0][0] stream caster: off
[2015-10-27 11:59:25.258][trace][26675][0] MR(merged-read): on, @see https://github.com/simple-rtmp-server/srs/issues/241
[2015-10-27 11:59:25.258][trace][26675][0] MR(merged-read) default 0 sleep 350
[2015-10-27 11:59:25.258][trace][26675][0] MW(merged-write) default sleep 350
[2015-10-27 11:59:25.258][trace][26675][0] read chunk stream cache cid [0, 16)
[2015-10-27 11:59:25.258][trace][26675][0] default gop cache 1, play queue 30s
[2015-10-27 11:59:25.258][trace][26675][0] complex send algorithm enabled.
[2015-10-27 11:59:25.258][warn][26675][0][0] TCP_NODELAY enabled, may hurts performance.
[2015-10-27 11:59:25.258][trace][26675][0] auto guess socket send buffer by merged write
[2015-10-27 11:59:25.258][trace][26675][0] system default latency in ms: mw(0-350) + mr(0-350) + play-queue(0-30000)
[2015-10-27 11:59:25.258][trace][26675][0] start deamon mode...
[2015-10-27 11:59:25.258][trace][26676][0] father process exit. ret=0
[2015-10-27 11:59:25.259][trace][26675][0] grandpa process exit.
[2015-10-27 11:59:25.259][trace][26677][0] son(deamon) process running.
[2015-10-27 11:59:25.259][trace][26677][0] st_set_eventsys to epoll
[2015-10-27 11:59:25.259][trace][26677][0] st_init success, use epoll
[2015-10-27 11:59:25.259][trace][26677][100] server main cid=100
[2015-10-27 11:59:25.260][trace][26677][100] write pid=26677 to ./objs/srs.pid success!
[2015-10-27 11:59:25.270][trace][26677][100] RTMP listen at tcp://0.0.0.0:1935, fd=8
[2015-10-27 11:59:25.270][trace][26677][100] signal installed
[2015-10-27 11:59:25.547][trace][26677][104] RTMP client ip=192.168.11.53
[2015-10-27 11:59:25.547][trace][26677][104] srand initialized the random.
[2015-10-27 11:59:25.548][trace][26677][104] simple handshake success.
[2015-10-27 11:59:25.548][trace][26677][104] connect app, tcUrl=rtmp://10.12.4.61:1935/live, pageUrl=, swfUrl=rtmp://10.12.4.61:1935/live, schema=rtmp, vhost=**defaultVhost**, port=1935, app=live, args=null
[2015-10-27 11:59:25.548][trace][26677][104] out chunk size to 60000
[2015-10-27 11:59:28.795][trace][26677][106] RTMP client ip=10.16.0.3
[2015-10-27 11:59:28.796][trace][26677][106] simple handshake success.
[2015-10-27 11:59:28.796][trace][26677][106] connect app, tcUrl=rtmp://10.12.4.61/live, pageUrl=, swfUrl=rtmp://10.12.4.61/live, schema=rtmp, vhost=**defaultVhost**, port=1935, app=live, args=null
[2015-10-27 11:59:28.796][trace][26677][106] out chunk size to 60000
[2015-10-27 11:59:29.045][trace][26677][106] client identified, type=fmle-publish, stream_name=mytest, duration=-1.00
[2015-10-27 11:59:29.055][trace][26677][106] source url=/live/mytest, ip=10.16.0.3, cache=1, is_edge=0, source_id=-1[-1]
[2015-10-27 11:59:29.255][trace][26677][106] hls: win=60.00, frag=30.00, prefix=, path=./objs/nginx/html, m3u8=[app]/[stream].m3u8, ts=[app]/[stream]-[seq].ts, aof=1.00, floor=0, clean=1, waitk=1, dispose=0
[2015-10-27 11:59:29.265][trace][26677][106] start publish mr=0/350, p1stpt=20000, pnt=20000, tcp_nodelay=0, rtcid=108
[2015-10-27 11:59:29.982][trace][26677][106] got metadata
[2015-10-27 11:59:29.989][trace][26677][106] 4B audio sh, codec(10, profile=LC, 2channels, 0kbps, 44100HZ), flv(16bits, 2channels, 44100HZ)
[2015-10-27 11:59:29.989][trace][26677][106] input chunk size to 143
[2015-10-27 11:59:29.989][trace][26677][106] input chunk size to 154
[2015-10-27 11:59:29.989][trace][26677][106] audio drop ts segment, sequence_no=0, uri=mytest-0.ts, duration=3343.64, start=0
[2015-10-27 11:59:30.491][trace][26677][106] input chunk size to 158
[2015-10-27 11:59:32.483][trace][26677][106] input chunk size to 161
[2015-10-27 11:59:37.484][trace][26677][106] input chunk size to 163
[2015-10-27 11:59:38.484][trace][26677][106] input chunk size to 166
[2015-10-27 11:59:40.499][trace][26677][106] input chunk size to 193
[2015-10-27 11:59:41.996][trace][26677][106] input chunk size to 197
[2015-10-27 11:59:45.987][trace][26677][106] input chunk size to 199
[2015-10-27 11:59:55.500][trace][26677][106] input chunk size to 211
[2015-10-27 11:59:55.747][warn][26677][104][62] client disconnect peer. ret=1004
[2015-10-27 11:59:59.762][trace][26677][110] RTMP client ip=192.168.11.53
[2015-10-27 11:59:59.762][trace][26677][110] simple handshake success.
[2015-10-27 11:59:59.762][trace][26677][110] connect app, tcUrl=rtmp://10.12.4.61:1935/live, pageUrl=, swfUrl=rtmp://10.12.4.61:1935/live, schema=rtmp, vhost=**defaultVhost**, port=1935, app=live, args=null
[2015-10-27 11:59:59.763][trace][26677][110] out chunk size to 60000
[2015-10-27 12:00:09.265][trace][26677][106] <- CPB time=20000, okbps=0,0,0, ikbps=51,51,0, mr=0/350, p1stpt=20000, pnt=20000

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

This is the current 30S slice.
http://pan.baidu.com/s/1i3523lF

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

At present, the sound of puffing is not so obvious, but there are still occasional intermittent situations.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Oct 27, 2015

Unable to use rtmpdump for recording, use srs_rtmp_dump instead, which can be found in the objs directory compiled. Rtmpdump will alter the timestamps.

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

Okay.

TRANS_BY_GPT3

@lewang321
Copy link
Author

[root@ objs]# ./srs_rtmp_dump -r rtmp://127.0.0.1:1935/live/mytest -o output.flv

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

Take a screenshot of the RTMP stream through srs_rtmp_dump
http://pan.baidu.com/s/1eQrvxHC

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

Can the issue you just mentioned about the time stamp mismatch between RTMP and TS caused by each PCR, which leads to popping sound and large gaps between clips, be improved by modifying the configuration file parameters?

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Oct 27, 2015

It seems that there is no issue with accuracy. I previously figured it out, but unfortunately, I have forgotten now and I don't remember it in my notes either. It is possible that nginx-rtmp, in order to reduce HLS overload, combines multiple AAC frames and writes them into ts files, which requires calculating time differences and such. I will figure out and fix this issue.

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

Okay, and I just tried SRS_1.0 and found that it seems to not support hls_acodec and hls_vcodec parameters, right?

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Oct 27, 2015

I changed to write each audio package, but this will increase the overhead of HLS. This can be avoided by changing the audio bitrate. The recommended bitrate is:

For the example video doc/source.200kbps.768x320.flv, the video bitrate is 212kbps. The audio bitrate:

  • When it is 30kbps, the TS size is 358892, FLV size is 294177, and the overhead is 21.9%.
  • When it is 48kbps, the TS size is 361336, FLV size is 317177, and the overhead is 13.9%.
  • When it is 54kbps, the TS size is 382768, FLV size is 324831, and the overhead is 17.8%.

Therefore, the overhead is minimized when the audio bitrate is between 48 and 54. If the video bitrate increases, the impact of the audio will be smaller.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Oct 27, 2015

2.0.196

There is no problem with video+audio.
The audio is slightly better, sometimes there is still distortion, but it has improved a lot.
I can't solve this problem for now, I don't know the behavior of the IOS player.
Let's stick with SRS2 for now, and we will solve this problem properly with SRS3.

TRANS_BY_GPT3

@winlinvip winlinvip modified the milestones: srs 3.0 release, srs 2.0 release Oct 27, 2015
@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

Okay, I will continue to pay attention to the SRS product. Thank you very much for taking the time to reply to my question.

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

I will be downloading version 2.0.196 and testing it in the near future.

TRANS_BY_GPT3

@lewang321
Copy link
Author

lewang321 commented Oct 27, 2015

Yes, it was also found during the testing process that as long as VIDEO and AUDIO are enabled, the iOS playback is normal. Now I understand what's going on.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

refer to #547

@DyncEric
Copy link

DyncEric commented Feb 17, 2016

The DTS of the audio needs to be consistent. If it is inconsistent, there will be a popping sound (in the case of both audio and video), but I have not tried it with pure audio.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Sep 15, 2016

It is caused by the cumulative error of the audio. For example, for AV streams (which include both audio and video), if multiple RTMP audio packets are combined into one AAC packet (to save the payload of TS), the total length of these packets should be calculated again based on the AAC timestamp (calculated by the number of samples and the sampling rate, with AAC generally fixed at 1024 bytes per sample, which can be obtained after parsing).

Suppose there are 10 A packets: A0-A9, converting their original encoder timestamps to 1/1000th of RTMP precision leads to loss of accuracy. Therefore, combining the errors of the 10 packets will display their errors. For example, if the RTMP timestamp for A0 is 100 milliseconds and A9 is 300 milliseconds, the actual combined AAC frame may have a timestamp of 303 milliseconds. The next RTMP timestamp is 300 milliseconds, resulting in some gaps.

Therefore, when combining multiple RTMP AAC packets into TS packets, the timestamps must be recalculated. Using 1024 bytes as the sample length directly in NGINX-RTMP is inaccurate. SRS can handle AV streams without combining audio packets, but for pure audio, packet combination is necessary; otherwise, it cannot be played.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Sep 15, 2016

I will find time to fix it completely.

TRANS_BY_GPT3

@winlinvip winlinvip reopened this Sep 15, 2016
@meibaogit
Copy link

meibaogit commented Sep 16, 2016

This morning, I checked and it appears to be like this:

An AAC frame is encoded using 1024 sample samples (not 1024 Bytes).

The playback time of an audio frame (in seconds) = number of sample samples in an AAC frame / sample rate.

If the sample rate is 44100Hz, according to the above, the playback time of this audio frame = 1024/44100 = 0.02232s * 1000 = 22.32ms.

The "pts" base clock is calculated based on 1000ms and reused in "ts", where the base clock is 90k, so it should also be multiplied by 90.

So, I'm looking at https://github.com/arut/nginx-rtmp-module/blob/e08959247dc840bb42cdf3389b1f5edb5686825f/hls/ngx_rtmp_hls_module.c#L1809.
est_pts = ctx->aframe_base + ctx->aframe_num * 90 * 1000 * 1024 / codec_ctx->sample_rate;
It seems that merging multiple frames and calculating pts is not a problem, right?
@winlinvip

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Sep 16, 2016

Didn't you see that he annotated TODO?

/* TODO: We assume here AAC frame size is 1024
     *       Need to handle AAC frames with frame size of 960 */

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Sep 16, 2016

And it only explains how to calculate, but does not explain why it should be calculated this way (recalculating timestamps based on sampling instead of using rtmp timestamps). This is the biggest problem in my opinion, in fact, it is unnecessary to calculate in HLS, only necessary when merging audio frames. If both audio and video streams are available in HLS, an audio frame can be written without needing to calculate DTS, directly using rtmp is fine.

TRANS_BY_GPT3

@meibaogit
Copy link

meibaogit commented Sep 17, 2016

1024 or 960 for AAC-LC
2048 or 1920 for HE-AAC (v2)
512 or 480 for AAC-LD and AAC-ELD
What determines the number of sampling samples required for encoding?
Once this is understood, his "need to" is resolved.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Sep 17, 2016

Calculating DTS based on the number of AAC samples is determined by TBN, and this is clear. The TBN for FLV is 1/1000, while the TBN for TS is 1/90000. TS has higher precision, and in reality, the original encoder also loses precision when outputting FLV/RTMP.

TRANS_BY_GPT3

@meibaogit
Copy link

meibaogit commented Sep 17, 2016

AAC Audio - Encoder Delay and Synchronization
https://developer.apple.com/library/content/technotes/tn2258/_index.html

It's also unclear here. Perhaps it's necessary to carefully examine the source code of fdkaac in order to understand.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Jan 6, 2017

The issue of explosive sound in HLS pure audio (there is a popping sound between each segment), Apple's reply email states that AAC files should be used instead of TS files, and the timestamps should be recalculated.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Feb 10, 2017

If TBN is 44100, then converting from FLV with TBN=1000 will result in loss, for example, a 34-millisecond FLV audio converted to TBN44100 would be 34*44100/1000=1499, converting it back to FLV with TBN44100 would be 1499*1000/44100=33, resulting in a loss of 1 millisecond. However, TS with TBN90000 is exactly 90 times the FLV's TBN, so it doesn't have this issue.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

About PCR, read #311

@winlinvip winlinvip changed the title srs_2.0对纯音频的流进行HLS 苹果系统播放卡顿伴有杂音的问题 HLS: SRS2对纯音频的流进行HLS 苹果系统播放卡顿伴有杂音的问题 Apr 16, 2017
@winlinvip
Copy link
Member

winlinvip commented Apr 16, 2017

The issue of loud noise has been identified and it is caused by the sampling rate. Please refer to: #547 (comment).

TRANS_BY_GPT3

@winlinvip winlinvip self-assigned this Sep 25, 2021
@winlinvip winlinvip added the TransByAI Translated by AI/GPT. label Jul 25, 2023
@winlinvip winlinvip changed the title HLS: SRS2对纯音频的流进行HLS 苹果系统播放卡顿伴有杂音的问题 HLS: SRS2 processes audio-only streams into HLS format, experiencing lag and static noise issues on Apple systems. Jul 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug It might be a bug. TransByAI Translated by AI/GPT.
Projects
None yet
Development

No branches or pull requests

4 participants