Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WebRTC: AddressSanitizer: heap-use-after-free #3413

Closed
limjoe opened this issue Feb 13, 2023 · 11 comments · Fixed by #3455
Closed

WebRTC: AddressSanitizer: heap-use-after-free #3413

limjoe opened this issue Feb 13, 2023 · 11 comments · Fixed by #3455
Assignees
Labels
Bug It might be a bug. TransByAI Translated by AI/GPT. WebRTC WebRTC, RTC2RTMP or RTMP2RTC.

Comments

@limjoe
Copy link

limjoe commented Feb 13, 2023

Description

v6.0.22 code, after compiling with Docker, it was noticed that certain pods had restart records after running for a while. Upon checking the logs, there were error records indicating heap_buffer_overflow, which is a memory overflow issue.

Docker compilation command

docker build -t srs:v6.0.22 --build-arg CONFARGS='--with-transcode  --http-callback=on --ssl=on --hls=on --http-api=on --stat=on --http-server=on --ffmpeg-tool=on'  --build-arg INSTALLDEPENDS='YES'  --platform linux/amd64  -f trunk/Dockerfile  . 
  1. SRS Version: v6.0.22

  2. SRS Log:

[2023-02-10 09:51:17.677][INFO][12][o04636c9] ignore disabled exec for vhost=__defaultVhost__
[2023-02-10 09:51:17.678][INFO][12][o04636c9] set fd=23 TCP_NODELAY 0=>1
[2023-02-10 09:51:17.678][INFO][12][o04636c9] start publish mr=0/350, p1stpt=20000, pnt=5000, tcp_nodelay=1
[2023-02-10 09:51:17.788][INFO][12][o04636c9] 36B video sh, codec(7, profile=Baseline, level=5, 2560x1440, 0kbps, 0.0fps, 0.0s)
[2023-02-10 09:51:17.838][INFO][12][o04636c9] -> HLS time=363061965ms, sno=3, ts=testlive/1676022677677.ts, dur=0ms, dva=0p
=================================================================
==12==ERROR: AddressSanitizer: heap-use-after-free on address 0x6130000a4842 at pc 0x562f22e36210 bp 0x7f611f4c61d0 sp 0x7f611f4c5978
READ of size 327 at 0x6130000a4842 thread T1 (srs-hybrid-2)
AddressSanitizer:DEADLYSIGNAL
AddressSanitizer: nested bug in the same thread, aborting.
  1. SRS Config:
listen              1935;
max_connections     1000;
daemon              off;
srs_log_tank        console;
srs_log_level       info;

grace_start_wait    2300;
grace_final_wait    3200;
force_grace_quit    on;
inotify_auto_reload on;

http_api {
    enabled         on;
    listen          1985;
}

vhost __defaultVhost__ {
    min_latency     on;
    tcp_nodelay     on;

    publish {
        mr off;
    }

    cluster {
        origin_cluster  on;
        coworkers       origin-rtmp-cluster-0.origin-rtmp-cluster-headless:1985 origin-rtmp-cluster-1.origin-rtmp-cluster-headless:1985 origin-rtmp-cluster-2.origin-rtmp-clu
ster-headless:1985;
    }

    hls {
        enabled         on;
        hls_fragment    10;
        hls_window      40;
        hls_path        ./objs/nginx/html;
        hls_m3u8_file   [app]/[stream].m3u8;
        hls_ts_file     [app]/[stream]/[timestamp].ts;
        hls_cleanup     on;
        hls_dispose     180;
        hls_wait_keyframe       on;
    }
}

Replay

Please describe how to replay the bug?

Step 1: docker image build

docker build -t srs:v6.0.22 --build-arg CONFARGS='--with-transcode  --http-callback=on --ssl=on --hls=on --http-api=on --stat=on --http-server=on --ffmpeg-tool=on'  --build-arg INSTALLDEPENDS='YES'  --platform linux/amd64  -f trunk/Dockerfile  . 

Step 2: docker run and push rtmp stream

1. Deploy the compiled image and run it.
2. After running for a certain period of time, some pods crash and restart. The specific reason for this issue is currently unknown.

Expect

No memory overflow crash issue during normal operation.

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Feb 13, 2023

asan's stack is too small, why is there no complete stack? Can you check if there are any other logs available?

If not, the only option is to map the core file and check the stack.

TRANS_BY_GPT3

@winlinvip winlinvip self-assigned this Feb 13, 2023
@winlinvip winlinvip added the Bug It might be a bug. label Feb 13, 2023
@limjoe
Copy link
Author

limjoe commented Feb 20, 2023

Last week, I observed for a week but still couldn't find a way to reproduce it. The logs before the crash only have the logs recorded above. How is the core file mapped out?

TRANS_BY_GPT3

@xiangzi1
Copy link

xiangzi1 commented Feb 27, 2023

I also encountered the same problem. Use case: pushing the stream in RTMP protocol, converting it to WebRTC protocol through RTMP, and reproducing the problem after running for more than ten hours, as shown in the figure:
Enterprise WeChat Screenshot_16774606903458
The key stack log is as follows: [File: srs.log]

TRANS_BY_GPT3

@xiangzi1
Copy link

srs(1).log

@shuizz
Copy link

shuizz commented Feb 27, 2023

Just to add to the phenomenon of xiangzi1: when playing 27 videos through WebRTC on the browser, there is a continuous increase in memory usage in SRS. Every 30 seconds, it stops and then resumes playing. After running for more than ten hours, it eventually crashes with the error "heap-use-after-free". This issue was discovered in version V5.0.141@winlinvip.

TRANS_BY_GPT3

@shuizz
Copy link

shuizz commented Feb 27, 2023

SRS Config
listen 1935;
max_connections 1000;

srs_log_tank console;
daemon off;
http_api {
enabled on;
listen 1985;
raw_api {
enabled on;
allow_reload on;
}
}
http_server {
enabled on;
listen 8080;
dir ./objs/nginx/html;
}
rtc_server {
enabled on;
listen 8000;
# @see https://github.com/ossrs/srs/wiki/v4_CN_WebRTC#config-candidate
candidate 192.168.2.5;
}
vhost defaultVhost {
http_hooks {
enabled on;
on_close http://192.168.2.5:13002/api/v1/clients;
on_play http://192.168.2.5:13002/api/v1/sessions;
on_stop http://192.168.2.5:13002/api/v1/sessions;
}
http_remux {
enabled on;
mount [vhost]/[app]/[stream].flv;
}
rtc {
enabled on;
# @see https://github.com/ossrs/srs/wiki/v4_CN_WebRTC#rtmp-to-rtc
rtmp_to_rtc on;
# @see https://github.com/ossrs/srs/wiki/v4_CN_WebRTC#rtc-to-rtmp
rtc_to_rtmp off;
stun_timeout 5;
stun_strict_check on;
}
publish {
normal_timeout 60000;
}
}
vhost download {
dvr {
enabled on;
dvr_path ./objs/nginx/html/[app]/[2006][01][02]/[stream].mp4;
}
http_hooks {
enabled on;
on_dvr http://192.168.2.5:13002/api/v1/dvrs;
}
}

@Drengel1990
Copy link

[2023-03-03 10:55:33.355][INFO][1][r2h69g96] TCP: clear zombies=4 resources, conns=12, removing=0, unsubs=0
[2023-03-03 10:55:33.355][INFO][1][rv6k24w6] TCP: disposing #0 resource(HttpConn)(0x607000440200), conns=12, disposing=4, zombies=0
[2023-03-03 10:55:33.355][INFO][1][mu2316z9] TCP: disposing #1 resource(HttpConn)(0x6070004402e0), conns=11, disposing=4, zombies=0
[2023-03-03 10:55:33.355][INFO][1][31734a35] TCP: disposing #2 resource(HttpConn)(0x6070004549b0), conns=10, disposing=4, zombies=0
[2023-03-03 10:55:33.355][INFO][1][w097i06x] TCP: disposing #3 resource(HttpConn)(0x60700045b390), conns=9, disposing=4, zombies=0
[2023-03-03 10:55:33.429][INFO][1][4366822b] RTC: NACK ARQ seq=17617, ssrc=11971, ts=5624030, count=12312/12312, 1214 bytes
[2023-03-03 10:55:33.431][INFO][1][1898s4x6] RTC: NACK ARQ seq=21300, ssrc=11959, ts=9071730, count=26426/26426, 1214 bytes
[2023-03-03 10:55:33.434][INFO][1][6hz572f7] RTC: NACK ARQ seq=36644, ssrc=11663, ts=41893940, count=2296/2296, 1214 bytes
[2023-03-03 10:55:33.465][INFO][1][83970e88] RTC: NACK ARQ seq=996, ssrc=11531, ts=52889060, count=28029/28029, 1214 bytes
[2023-03-03 10:55:33.470][INFO][1][341s4171] RTC: NACK ARQ seq=16790, ssrc=12063, ts=5496210, count=3017/3017, 1214 bytes
[2023-03-03 10:55:33.470][INFO][1][kop4l0u1] RTC: NACK ARQ seq=16788, ssrc=12067, ts=5493240, count=1320/1320, 372 bytes
[2023-03-03 10:55:33.484][INFO][1][4983282z] RTC: NACK ARQ seq=754, ssrc=11527, ts=52877270, count=38846/38846, 1214 bytes
[2023-03-03 10:55:33.486][INFO][1][18103pqh] RTC: NACK ARQ seq=16728, ssrc=12033, ts=5469210, count=25976/25976, 1214 bytes
[2023-03-03 10:55:33.488][INFO][1][324zl69v] RTC: NACK ARQ seq=1889, ssrc=11295, ts=67017060, count=3956/3956, 1214 bytes
[2023-03-03 10:55:33.531][INFO][1][92614i0h] RTC: NACK ARQ seq=36665, ssrc=11731, ts=41855060, count=29064/29064, 772 bytes
[2023-03-03 10:55:33.544][INFO][1][7o42v905] RTC: NACK ARQ seq=940, ssrc=11461, ts=52949090, count=21350/21350, 1214 bytes
[2023-03-03 10:55:33.545][INFO][1][525tj1d8] RTC: NACK ARQ seq=36630, ssrc=11713, ts=41848940, count=11053/11053, 1214 bytes
[2023-03-03 10:55:33.549][INFO][1][7b28d182] RTC: NACK ARQ seq=523, ssrc=11579, ts=52803900, count=17069/17069, 1214 bytes
[2023-03-03 10:55:33.550][INFO][1][346x0q06] RTC: NACK ARQ seq=1365, ssrc=11403, ts=66524130, count=505540/505540, 745 bytes
[2023-03-03 10:55:33.554][INFO][1][wp3s8z52] DTLS: State Passive RECV, done=1, arq=0/0, r0=1, r1=0, len=581, cnt=22, size=301, hs=11
[2023-03-03 10:55:33.554][INFO][1][wp3s8z52] DTLS: State Passive SEND, done=1, arq=0/0, r0=-1, r1=2, len=554, cnt=22, size=466, hs=4
[2023-03-03 10:55:33.558][INFO][1][mu2316z9] RTC: session address init 217.65.137.196:52555
[2023-03-03 10:55:33.558][INFO][1][mu2316z9] RTC: session STUN done, waiting DTLS handshake.
[2023-03-03 10:55:33.591][INFO][1][t9d39w8r] CircuitBreaker: cpu=98.00%,819MB, break=1,1,0, cond=98.00%
[2023-03-03 10:55:33.659][INFO][1][68f0lv93] RTC: NACK ARQ seq=16809, ssrc=12037, ts=5502240, count=461/461, 1214 bytes
[2023-03-03 10:55:33.705][INFO][1][rv6k24w6] RTC: session address init 217.65.137.196:52413
[2023-03-03 10:55:33.705][INFO][1][rv6k24w6] RTC: session STUN done, waiting DTLS handshake.
[2023-03-03 10:55:33.706][INFO][1][ql6l752c] RTC: NACK ARQ seq=623, ssrc=11575, ts=52842600, count=559546/559546, 1214 bytes
[2023-03-03 10:55:33.718][INFO][1][1n5422eg] RTC: NACK ARQ seq=1235, ssrc=12209, ts=485640, count=1/1, 1214 bytes
[2023-03-03 10:55:33.719][INFO][1][61kgs0nk] RTC: NACK ARQ seq=36722, ssrc=11717, ts=41879090, count=26590/26590, 1214 bytes
[2023-03-03 10:55:33.733][INFO][1][77710383] RTC: NACK ARQ seq=49548, ssrc=11931, ts=23607000, count=54123/54123, 1214 bytes
[2023-03-03 10:55:33.733][INFO][1][vj564xw3] RTC: NACK ARQ seq=53558, ssrc=11933, ts=22292820, count=55035/55035, 1214 bytes
[2023-03-03 10:55:33.778][INFO][1][rv6k24w6] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=157, cnt=22, size=144, hs=1
[2023-03-03 10:55:33.779][INFO][1][rv6k24w6] DTLS: State Passive SEND, done=0, arq=0/0, r0=-1, r1=2, len=679, cnt=22, size=82, hs=2
[2023-03-03 10:55:33.901][INFO][1][9j512x67] RTC: NACK ARQ seq=15387, ssrc=11961, ts=6516000, count=8989/8989, 1214 bytes
[2023-03-03 10:55:33.920][INFO][1][yk125bay] RTC: NACK ARQ seq=20372, ssrc=11915, ts=34775910, count=390019/390019, 1214 bytes
[2023-03-03 10:55:33.928][INFO][1][aj95c452] RTC: NACK ARQ seq=23292, ssrc=11897, ts=35943210, count=441806/441806, 1214 bytes
[2023-03-03 10:55:33.950][INFO][1][31734a35] RTC: session address init 145.224.100.30:65332
[2023-03-03 10:55:33.950][INFO][1][31734a35] RTC: session STUN done, waiting DTLS handshake.
[2023-03-03 10:55:34.053][INFO][1][075r4553] RTC: NACK ARQ seq=1881, ssrc=11347, ts=67014090, count=1676/1676, 1214 bytes
[2023-03-03 10:55:34.056][INFO][1][y0y27spr] RTC: NACK ARQ seq=24027, ssrc=11849, ts=36231570, count=2857/2857, 1214 bytes
[2023-03-03 10:55:34.061][INFO][1][vkgdb358] RTC: NACK ARQ seq=1916, ssrc=11327, ts=67049910, count=405/405, 1214 bytes
[2023-03-03 10:55:34.064][INFO][1][1t55jng4] RTC: NACK ARQ seq=1870, ssrc=11395, ts=66986910, count=1559/1559, 1214 bytes
[2023-03-03 10:55:34.121][INFO][1][1741b163] RTC: NACK ARQ seq=1890, ssrc=11341, ts=67017240, count=47659/47659, 1214 bytes
[2023-03-03 10:55:34.122][INFO][1][rv6k24w6] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=578, cnt=22, size=299, hs=11
[2023-03-03 10:55:34.122][INFO][1][rv6k24w6] DTLS: State Passive SEND, done=1, arq=0/0, r0=1, r1=0, len=1233, cnt=22, size=82, hs=2
[2023-03-03 10:55:34.122][INFO][1][rv6k24w6] RTC: DTLS handshake done.
[2023-03-03 10:55:34.122][INFO][1][rv6k24w6] RTC: session pub=0, sub=1, to=5000ms connection established
[2023-03-03 10:55:34.122][INFO][1][rv6k24w6] RTC: Subscriber url=/live/G0CWP6EW established
[2023-03-03 10:55:34.123][INFO][1][rv6k24w6] create consumer, no gop cache
[2023-03-03 10:55:34.123][INFO][1][rv6k24w6] RTC: start play url=/live/G0CWP6EW, source_id=gd2034o9/41j56hrn, realtime=1, mw_msgs=0
[2023-03-03 10:55:34.124][INFO][1][g1gtk4d8] RTC: NACK ARQ seq=34798, ssrc=11745, ts=41363820, count=377721/377721, 1214 bytes
[2023-03-03 10:55:34.127][INFO][1][6o759303] RTC: NACK ARQ seq=24030, ssrc=11845, ts=36234450, count=2420/2420, 542 bytes
[2023-03-03 10:55:34.155][INFO][1][mu2316z9] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=157, cnt=22, size=144, hs=1
[2023-03-03 10:55:34.155][INFO][1][mu2316z9] DTLS: State Passive SEND, done=0, arq=0/0, r0=-1, r1=2, len=679, cnt=22, size=82, hs=2
[2023-03-03 10:55:34.170][INFO][1][95cv64k0] RTC: NACK ARQ seq=36869, ssrc=11711, ts=41956940, count=37492/37492, 1214 bytes
[2023-03-03 10:55:34.231][INFO][1][91c8e089] RTC: NACK ARQ seq=1794, ssrc=11365, ts=66920940, count=472938/472938, 760 bytes
[2023-03-03 10:55:34.242][INFO][1][0s4i6766] RTC: NACK ARQ seq=18248, ssrc=11913, ts=33881760, count=434724/434724, 1214 bytes
[2023-03-03 10:55:34.247][INFO][1][mu2316z9] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=578, cnt=22, size=299, hs=11
[2023-03-03 10:55:34.247][INFO][1][mu2316z9] DTLS: State Passive SEND, done=1, arq=0/0, r0=1, r1=0, len=1233, cnt=22, size=82, hs=2
[2023-03-03 10:55:34.247][INFO][1][mu2316z9] RTC: DTLS handshake done.
[2023-03-03 10:55:34.247][INFO][1][mu2316z9] RTC: session pub=0, sub=1, to=5000ms connection established
[2023-03-03 10:55:34.247][INFO][1][mu2316z9] RTC: Subscriber url=/live/8OZHQA6K established
[2023-03-03 10:55:34.247][INFO][1][mu2316z9] create consumer, no gop cache
[2023-03-03 10:55:34.247][INFO][1][mu2316z9] RTC: start play url=/live/8OZHQA6K, source_id=1z7h769f/u584x836, realtime=1, mw_msgs=0
[2023-03-03 10:55:34.249][INFO][1][6mph4a92] RTC: NACK ARQ seq=23750, ssrc=11895, ts=36140850, count=2087/2087, 1214 bytes
[2023-03-03 10:55:34.253][INFO][1][67a8k836] RTC: NACK ARQ seq=17815, ssrc=11957, ts=10199970, count=29841/29841, 1214 bytes
[2023-03-03 10:55:34.285][INFO][1][mu2316z9] RTC: Jitter init base=200, value=2294
[2023-03-03 10:55:34.285][INFO][1][mu2316z9] RTC: Jitter rebase value=2294, last=0, distance=-2294, pkt-base=2294/2294, correct-base=200/0
[2023-03-03 10:55:34.285][INFO][1][mu2316z9] RTC: Jitter init base=20000, value=896580
[2023-03-03 10:55:34.285][INFO][1][mu2316z9] RTC: Jitter rebase value=896580, last=0, distance=-896580, pkt-base=896580/896580, correct-base=20000/0
[2023-03-03 10:55:34.362][INFO][1][zi37a1nq] RTC: NACK ARQ seq=760, ssrc=11581, ts=52853580, count=554767/554767, 233 bytes
[2023-03-03 10:55:34.409][INFO][1][789q58a5] RTC: NACK ARQ seq=1159, ssrc=11519, ts=53048720, count=13714/13714, 1214 bytes
[2023-03-03 10:55:34.416][INFO][1][73z3769g] RTC: NACK ARQ seq=1942, ssrc=11313, ts=67076910, count=12852/12852, 1214 bytes
[2023-03-03 10:55:34.435][INFO][1][rv6k24w6] RTC: Jitter init base=200, value=1343
[2023-03-03 10:55:34.435][INFO][1][rv6k24w6] RTC: Jitter rebase value=1343, last=0, distance=-1343, pkt-base=1343/1343, correct-base=200/0
[2023-03-03 10:55:34.435][INFO][1][rv6k24w6] RTC: Jitter init base=20000, value=53306550
[2023-03-03 10:55:34.435][INFO][1][rv6k24w6] RTC: Jitter rebase value=53306550, last=0, distance=-53306550, pkt-base=53306550/53306550, correct-base=20000/0
[2023-03-03 10:55:34.591][INFO][1][t9d39w8r] CircuitBreaker: cpu=100.00%,820MB, break=1,1,0, cond=100.00%
[2023-03-03 10:55:34.609][INFO][1][05c2j29b] RTC: NACK ARQ seq=1218, ssrc=12223, ts=465300, count=1/1, 1214 bytes
[2023-03-03 10:55:34.620][INFO][1][656490z1] RTC: NACK ARQ seq=1268, ssrc=12193, ts=486180, count=1/1, 1214 bytes
[2023-03-03 10:55:34.620][INFO][1][69841232] RTC: NACK ARQ seq=1770, ssrc=12119, ts=662150, count=1/1, 139 bytes
[2023-03-03 10:55:34.633][INFO][1][7mz78g39] RTC: NACK ARQ seq=1791, ssrc=12153, ts=656820, count=1/1, 1214 bytes
[2023-03-03 10:55:34.636][INFO][1][643ye9h5] RTC: NACK ARQ seq=5630, ssrc=12101, ts=1833210, count=695/695, 1214 bytes
[2023-03-03 10:55:34.735][INFO][1][g5872l54] Hybrid cpu=100.00%,820MB, cid=306,3466, timer=61,0,15837, clock=0,42,4,1,1,1,1,1,0, free=9, objs=(pkt:71143,raw:1134,fua:69843,msg:71463,oth:164,buf:2)
[2023-03-03 10:55:34.736][INFO][1][w097i06x] RTC: session destroy by timeout, username=i148i286:Zepn
[2023-03-03 10:55:34.736][INFO][1][w097i06x] RTC: before dispose resource(RtcConn)(0x61d0003a3e80), conns=385, zombies=0, ign=0, inz=0, ind=0
[2023-03-03 10:55:34.736][INFO][1][w097i06x] RTC: session detach from [w097i06x](RtcConn), disposing=1
[2023-03-03 10:55:34.736][INFO][1][a7557lwr] RTC: session destroy by timeout, username=6fm20697:W5pk
[2023-03-03 10:55:34.736][INFO][1][a7557lwr] RTC: before dispose resource(RtcConn)(0x61d0003a7a80), conns=385, zombies=1, ign=0, inz=0, ind=0
[2023-03-03 10:55:34.736][INFO][1][a7557lwr] RTC: session detach from [a7557lwr](RtcConn), disposing=1
[2023-03-03 10:55:34.736][INFO][1][9b354583] RTC: session destroy by timeout, username=01967012:C175
[2023-03-03 10:55:34.736][INFO][1][9b354583] RTC: before dispose resource(RtcConn)(0x61d0003ba680), conns=385, zombies=2, ign=0, inz=0, ind=0
[2023-03-03 10:55:34.736][INFO][1][9b354583] RTC: session detach from [9b354583](RtcConn), disposing=1
[2023-03-03 10:55:34.736][INFO][1][g5872l54] RTC: Server conns=382, rpkts=(3399,rtp:0,stun:173,rtcp:3225), spkts=(109471,rtp:109300,stun:170,rtcp:0), rtcp=(pli:45,twcc:0,rr:0), rnk=(1948,1948,h:39271,m:55), loss=(r:10818,s:0), fid=(id:19,fid:3399,ffid:0,addr:165,faddr:3403)
[2023-03-03 10:55:34.741][INFO][1][2k71gg38] RTC: clear zombies=3 resources, conns=385, removing=0, unsubs=9
[2023-03-03 10:55:34.741][INFO][1][w097i06x] RTC: disposing #0 resource(RtcConn)(0x61d0003a3e80), conns=385, disposing=3, zombies=0
[2023-03-03 10:55:34.741][INFO][1][a7557lwr] RTC: disposing #1 resource(RtcConn)(0x61d0003a7a80), conns=384, disposing=3, zombies=0
[2023-03-03 10:55:34.741][INFO][1][9b354583] RTC: disposing #2 resource(RtcConn)(0x61d0003ba680), conns=383, disposing=3, zombies=0
[2023-03-03 10:55:34.742][WARN][1][9b354583][11][DTLS_HANG] DTLS: Hang, done=0, version=-1, arq=0
[2023-03-03 10:55:34.742][WARN][1][9b354583][11][DTLS_HANG] DTLS: Hang, done=0, version=-1, arq=0
[2023-03-03 10:55:34.742][WARN][1][9b354583][11][DTLS_HANG] DTLS: Hang, done=0, version=-1, arq=0
[2023-03-03 10:55:34.742][WARN][1][9b354583][11][DTLS_HANG] DTLS: Hang, done=0, version=-1, arq=0
[2023-03-03 10:55:34.742][WARN][1][9b354583][11][DTLS_HANG] DTLS: Hang, done=0, version=-1, arq=0
[2023-03-03 10:55:34.742][WARN][1][9b354583][11][DTLS_HANG] DTLS: Hang, done=0, version=-1, arq=0
[2023-03-03 10:55:34.750][INFO][1][993cpw25] RTC: NACK ARQ seq=581, ssrc=11585, ts=52842600, count=386459/386459, 1214 bytes
[2023-03-03 10:55:34.750][INFO][1][27w79834] RTC: NACK ARQ seq=16758, ssrc=12019, ts=5535180, count=31/31, 1214 bytes
[2023-03-03 10:55:34.758][INFO][1][02x7b88z] RTC: NACK ARQ seq=16605, ssrc=12087, ts=5442030, count=18025/18025, 1214 bytes
[2023-03-03 10:55:34.837][INFO][1][3497528i] RTC: NACK ARQ seq=37069, ssrc=11707, ts=41989970, count=480/480, 1214 bytes
[2023-03-03 10:55:35.028][INFO][1][j07z3534] RTC: NACK ARQ seq=33508, ssrc=11753, ts=40856940, count=417233/417233, 1214 bytes
[2023-03-03 10:55:35.031][INFO][1][28vi87af] RTC: NACK ARQ seq=1447, ssrc=12175, ts=539550, count=1/1, 1214 bytes
[2023-03-03 10:55:35.036][INFO][1][577639j5] RTC: NACK ARQ seq=1326, ssrc=11521, ts=53105510, count=2994/2994, 1214 bytes
[2023-03-03 10:55:35.043][INFO][1][j483c451] RTC: NACK ARQ seq=1458, ssrc=11465, ts=53108660, count=770/770, 1214 bytes
[2023-03-03 10:55:35.049][INFO][1][2o8ks3h7] RTC: NACK ARQ seq=16724, ssrc=12055, ts=5514120, count=45/45, 173 bytes
[2023-03-03 10:55:35.051][INFO][1][s385769h] RTC: NACK ARQ seq=1456, ssrc=12167, ts=554490, count=1/1, 1214 bytes
[2023-03-03 10:55:35.173][INFO][1][6s82sj16] RTC: NACK ARQ seq=36543, ssrc=11757, ts=41189940, count=7977/7977, 1214 bytes
[2023-03-03 10:55:35.180][INFO][1][op7j474l] RTC: NACK ARQ seq=1503, ssrc=11467, ts=53129720, count=423/423, 377 bytes
[2023-03-03 10:55:35.180][INFO][1][0027q895] RTC: NACK ARQ seq=1496, ssrc=11485, ts=53125040, count=1838/1838, 1214 bytes
[2023-03-03 10:55:35.195][INFO][1][00ue0143] RTC: NACK ARQ seq=35052, ssrc=11751, ts=41468760, count=464692/464692, 1214 bytes
[2023-03-03 10:55:35.197][INFO][1][0yx0rx79] RTC: NACK ARQ seq=1456, ssrc=11559, ts=53037360, count=1983/1983, 521 bytes
[2023-03-03 10:55:35.286][INFO][1][c165ll35] <- CPB time=745078652, okbps=0,0,0, ikbps=0,1136,1811, mr=0/350, p1stpt=20000, pnt=5000
[2023-03-03 10:55:35.352][INFO][1][4x3we391] RTC: NACK ARQ seq=1181, ssrc=11553, ts=53022960, count=6096/6096, 1214 bytes
[2023-03-03 10:55:35.352][INFO][1][rv6k24w6] RTC: NACK ARQ seq=212, ssrc=12235, ts=17550, count=1/1, 569 bytes
[2023-03-03 10:55:35.353][INFO][1][0485i8h6] RTC: NACK ARQ seq=1408, ssrc=11561, ts=53026920, count=5779/5779, 1214 bytes
[2023-03-03 10:55:35.353][INFO][1][48v3x83f] RTC: NACK ARQ seq=2115, ssrc=11301, ts=67128300, count=3754/3754, 1214 bytes
[2023-03-03 10:55:35.355][INFO][1][88wl1656] RTC: NACK ARQ seq=41779, ssrc=11595, ts=52695090, count=7131/7131, 345 bytes
[2023-03-03 10:55:35.355][INFO][1][2a640o41] RTC: NACK ARQ seq=1265, ssrc=11583, ts=53085600, count=13936/13936, 136 bytes
[2023-03-03 10:55:35.414][INFO][1][nj1lthzj] RTC: NACK ARQ seq=1424, ssrc=11549, ts=53026920, count=1967/1967, 1214 bytes
[2023-03-03 10:55:35.414][INFO][1][2k8482g0] RTC: NACK ARQ seq=16768, ssrc=12073, ts=5535180, count=8718/8718, 1214 bytes
[2023-03-03 10:55:35.415][INFO][1][62090762] RTC: NACK ARQ seq=1490, ssrc=11511, ts=53109920, count=2145/2145, 249 bytes
[2023-03-03 10:55:35.415][INFO][1][288j0130] RTC: NACK ARQ seq=1577, ssrc=11451, ts=53200190, count=1310/1310, 1214 bytes
[2023-03-03 10:55:35.417][INFO][1][7a85el30] RTC: NACK ARQ seq=1310, ssrc=11547, ts=53041860, count=37089/37089, 1214 bytes
[2023-03-03 10:55:35.417][INFO][1][36k9t776] RTC: NACK ARQ seq=1490, ssrc=11507, ts=53109920, count=2130/2130, 249 bytes
[2023-03-03 10:55:35.418][INFO][1][9f72y88d] RTC: NACK ARQ seq=24186, ssrc=11863, ts=36303030, count=2740/2740, 1039 bytes
[2023-03-03 10:55:35.419][INFO][1][qsuz2197] RTC: NACK ARQ seq=824, ssrc=11545, ts=52998660, count=32161/32161, 1214 bytes
[2023-03-03 10:55:35.419][INFO][1][5fqi9c89] RTC: NACK ARQ seq=1427, ssrc=11537, ts=53041860, count=28171/28171, 1214 bytes
[2023-03-03 10:55:35.421][INFO][1][y28vz100] RTC: NACK ARQ seq=16292, ssrc=12089, ts=5352390, count=19574/19574, 118 bytes
[2023-03-03 10:55:35.429][INFO][1][29o45eh4] RTC: NACK ARQ seq=24455, ssrc=11793, ts=36437580, count=1793/1793, 1214 bytes
[2023-03-03 10:55:35.429][INFO][1][d809bx75] RTC: NACK ARQ seq=1512, ssrc=11453, ts=53148980, count=43150/43150, 1214 bytes
[2023-03-03 10:55:35.431][INFO][1][1lo19271] RTC: NACK ARQ seq=1298, ssrc=11563, ts=53099010, count=1131/1131, 1214 bytes
[2023-03-03 10:55:35.431][INFO][1][67024288] RTC: NACK ARQ seq=1276, ssrc=11477, ts=53105510, count=18261/18261, 40 bytes
[2023-03-03 10:55:35.436][INFO][1][669e9974] RTC: NACK ARQ seq=1467, ssrc=11539, ts=53061390, count=28640/28640, 1214 bytes
[2023-03-03 10:55:35.439][INFO][1][2y40qwb9] RTC: NACK ARQ seq=1847, ssrc=11367, ts=66989880, count=510343/510343, 1214 bytes
[2023-03-03 10:55:35.440][INFO][1][r2wbm7c0] RTC: NACK ARQ seq=1423, ssrc=11475, ts=53105510, count=4437/4437, 1214 bytes
[2023-03-03 10:55:35.442][INFO][1][027962dy] RTC: NACK ARQ seq=1436, ssrc=11463, ts=53129450, count=7996/7996, 1214 bytes
[2023-03-03 10:55:35.472][INFO][1][1813x05e] RTC: NACK ARQ seq=1482, ssrc=11471, ts=53125040, count=2920/2920, 1214 bytes
[2023-03-03 10:55:35.473][INFO][1][c3u882a2] RTC: NACK ARQ seq=16480, ssrc=11963, ts=6604380, count=2975/2975, 1214 bytes
[2023-03-03 10:55:35.480][INFO][1][p6q77272] RTC: NACK ARQ seq=24401, ssrc=11807, ts=36411750, count=802/802, 1214 bytes
[2023-03-03 10:55:35.481][INFO][1][58m37w67] RTC: NACK ARQ seq=1438, ssrc=11513, ts=53069420, count=11607/11607, 1214 bytes
[2023-03-03 10:55:35.485][INFO][1][51z7536z] RTC: NACK ARQ seq=24508, ssrc=11787, ts=36461880, count=7672/7672, 1214 bytes
[2023-03-03 10:55:35.485][INFO][1][984021b7] RTC: NACK ARQ seq=24136, ssrc=11891, ts=36308970, count=1040/1040, 833 bytes
[2023-03-03 10:55:35.488][INFO][1][5h710y63] RTC: NACK ARQ seq=24526, ssrc=11795, ts=36467910, count=486/486, 1214 bytes
[2023-03-03 10:55:35.488][INFO][1][31e01ot7] RTC: NACK ARQ seq=24457, ssrc=11789, ts=36441450, count=2582/2582, 1214 bytes
[2023-03-03 10:55:35.489][INFO][1][sl65y724] RTC: session address change 91.102.183.167:8292 -> 91.102.183.167:8092, cached=1, nn_change=13/7, nn_address=2
[2023-03-03 10:55:35.533][INFO][1][yp1mbp65] RTC: NACK ARQ seq=2084, ssrc=11303, ts=67109850, count=40058/40058, 761 bytes
[2023-03-03 10:55:35.534][INFO][1][83h1pei6] RTC: NACK ARQ seq=2045, ssrc=11377, ts=67050000, count=1268/1268, 1214 bytes
[2023-03-03 10:55:35.562][INFO][1][342dv241] RTC: NACK ARQ seq=24454, ssrc=11805, ts=36432270, count=2164/2164, 1214 bytes
[2023-03-03 10:55:35.563][INFO][1][c585f342] RTC: NACK ARQ seq=16804, ssrc=12061, ts=5535180, count=70/70, 1214 bytes
[2023-03-03 10:55:35.591][INFO][1][t9d39w8r] CircuitBreaker: cpu=100.00%,822MB, break=1,1,0, cond=100.00%
[2023-03-03 10:55:35.696][INFO][1][3879vb7l] RTC: NACK ARQ seq=16623, ssrc=11993, ts=5499270, count=35743/35743, 1214 bytes
[2023-03-03 10:55:35.765][INFO][1][3t496w07] RTC: NACK ARQ seq=1847, ssrc=12229, ts=614970, count=1/1, 1214 bytes
[2023-03-03 10:55:35.892][INFO][1][053h1178] RTC: NACK ARQ seq=37207, ssrc=11619, ts=42134060, count=66060/66060, 1214 bytes
[2023-03-03 10:55:35.893][INFO][1][ttl88264] RTC: NACK ARQ seq=1053, ssrc=11573, ts=53022960, count=555682/555682, 1214 bytes
[2023-03-03 10:55:35.894][INFO][1][1c36905z] RTC: NACK ARQ seq=34387, ssrc=11743, ts=41180670, count=354778/354778, 1214 bytes
[2023-03-03 10:55:35.896][INFO][1][7qp08k71] RTC: NACK ARQ seq=16625, ssrc=12025, ts=5460390, count=37636/37636, 1214 bytes
[2023-03-03 10:55:35.897][INFO][1][571jkbu8] RTC: NACK ARQ seq=18195, ssrc=11975, ts=5885300, count=94/94, 1214 bytes
[2023-03-03 10:55:35.914][INFO][1][87m508ku] RTC: NACK ARQ seq=37460, ssrc=11621, ts=42238910, count=3245/3245, 1214 bytes
[2023-03-03 10:55:35.999][INFO][1][906b7g15] RTC: NACK ARQ seq=24724, ssrc=11779, ts=36531090, count=161/161, 1214 bytes
[2023-03-03 10:55:36.001][INFO][1][wd36v690] RTC: NACK ARQ seq=24026, ssrc=11909, ts=36248940, count=1295/1295, 1214 bytes
[2023-03-03 10:55:36.004][INFO][1][9679hr28] RTC: NACK ARQ seq=2116, ssrc=11325, ts=67101120, count=12204/12204, 1214 bytes
[2023-03-03 10:55:36.053][INFO][1][1r3b86kx] RTC: NACK ARQ seq=37155, ssrc=11669, ts=42082940, count=7329/7329, 550 bytes
[2023-03-03 10:55:36.067][INFO][1][t407gq2y] RTC: NACK ARQ seq=37482, ssrc=11649, ts=42245030, count=19531/19531, 1214 bytes
[2023-03-03 10:55:36.070][INFO][1][274ab65s] RTC: NACK ARQ seq=1441, ssrc=12115, ts=570240, count=1/1, 1214 bytes
[2023-03-03 10:55:36.196][INFO][1][e1932qlb] RTC: NACK ARQ seq=24174, ssrc=11905, ts=36299790, count=9629/9629, 1214 bytes
[2023-03-03 10:55:36.338][INFO][1][g50n4q35] <- RTC RECV #13, udp 36282, pps 1/3(k), schedule 36282
[2023-03-03 10:55:36.343][INFO][1][ml0n8683] RTC: NACK ARQ seq=16455, ssrc=12077, ts=5415300, count=98601/98601, 1214 bytes
[2023-03-03 10:55:36.577][INFO][1][31734a35] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=157, cnt=22, size=144, hs=1
[2023-03-03 10:55:36.577][INFO][1][31734a35] DTLS: State Passive SEND, done=0, arq=0/0, r0=-1, r1=2, len=678, cnt=22, size=82, hs=2
[2023-03-03 10:55:36.591][INFO][1][t9d39w8r] CircuitBreaker: cpu=100.00%,822MB, break=1,1,0, cond=100.00%
[2023-03-03 10:55:36.650][INFO][1][7j6bqm5u] RTC: NACK ARQ seq=10144, ssrc=11747, ts=31497120, count=279678/279678, 768 bytes
[2023-03-03 10:55:36.652][INFO][1][80l1256x] Process: cpu=100.00%,822MB, threads=2
[2023-03-03 10:55:36.653][INFO][1][8a13r442] RTC: NACK ARQ seq=24225, ssrc=11907, ts=36321120, count=308050/308050, 1214 bytes
[2023-03-03 10:55:36.660][INFO][1][7jdu8836] RTC: NACK ARQ seq=37456, ssrc=11703, ts=42178970, count=32734/32734, 1214 bytes
[2023-03-03 10:55:36.662][INFO][1][j6fu036j] RTC: NACK ARQ seq=24642, ssrc=11821, ts=36486450, count=3917/3917, 1214 bytes
[2023-03-03 10:55:36.749][INFO][1][9dgc01z9] RTC: NACK ARQ seq=17776, ssrc=11981, ts=5805180, count=1297/1297, 1214 bytes
[2023-03-03 10:55:36.757][INFO][1][78148k7c] RTC: NACK ARQ seq=2066, ssrc=11421, ts=67014000, count=63265/63265, 1214 bytes
[2023-03-03 10:55:37.034][INFO][1][0320r081] RTC: NACK ARQ seq=10627, ssrc=11967, ts=6803820, count=26/26, 1214 bytes
[2023-03-03 10:55:37.071][INFO][1][ty828p79] RTC: NACK ARQ seq=24224, ssrc=11901, ts=36320760, count=262916/262916, 1214 bytes
[2023-03-03 10:55:37.087][INFO][1][e0k36fpu] RTC: NACK ARQ seq=33948, ssrc=11777, ts=40069350, count=9856/9856, 281 bytes
[2023-03-03 10:55:37.193][INFO][1][u4ku932n] RTC: NACK ARQ seq=24534, ssrc=11875, ts=36455760, count=22019/22019, 1214 bytes
[2023-03-03 10:55:37.193][INFO][1][1k10z438] RTC: NACK ARQ seq=24609, ssrc=11867, ts=36468180, count=20685/20685, 1214 bytes
[2023-03-03 10:55:37.290][INFO][1][o7l7f1u8] RTC: NACK ARQ seq=15318, ssrc=12097, ts=5015970, count=91/91, 460 bytes
[2023-03-03 10:55:37.296][INFO][1][o1vyc020] RTC: NACK ARQ seq=2371, ssrc=11397, ts=67202910, count=2667/2667, 633 bytes
[2023-03-03 10:55:37.316][INFO][1][9o416087] RTC: NACK ARQ seq=17231, ssrc=12045, ts=5670000, count=3052/3052, 1214 bytes
[2023-03-03 10:55:37.364][INFO][1][188o245j] RTC: NACK ARQ seq=17719, ssrc=12039, ts=5787090, count=8344/8344, 1214 bytes
[2023-03-03 10:55:37.389][INFO][1][m32x48v0] RTC: NACK ARQ seq=2015, ssrc=11447, ts=53381180, count=430/430, 953 bytes
[2023-03-03 10:55:37.417][INFO][1][664401h1] RTC: NACK ARQ seq=2383, ssrc=11333, ts=67242960, count=1710/1710, 201 bytes
[2023-03-03 10:55:37.436][INFO][1][5vu723a9] RTC: NACK ARQ seq=24867, ssrc=11803, ts=36594360, count=3428/3428, 1214 bytes
[2023-03-03 10:55:37.590][INFO][1][bcz80k06] RTC: session address change 91.102.183.167:8183 -> 91.102.183.167:9986, cached=1, nn_change=7/4, nn_address=2
[2023-03-03 10:55:37.591][INFO][1][t9d39w8r] CircuitBreaker: cpu=100.00%,824MB, break=1,1,0, cond=100.00%
[2023-03-03 10:55:37.604][INFO][1][7447346f] RTC: NACK ARQ seq=24688, ssrc=11853, ts=36508050, count=9530/9530, 1214 bytes
[2023-03-03 10:55:37.609][INFO][1][52498u96] RTC: NACK ARQ seq=2433, ssrc=12137, ts=948350, count=93/93, 1214 bytes
[2023-03-03 10:55:37.612][INFO][1][0iw8905i] RTC: NACK ARQ seq=2482, ssrc=11345, ts=67284090, count=1520/1520, 1214 bytes
[2023-03-03 10:55:37.622][INFO][1][3t6zda12] RTC: NACK ARQ seq=1930, ssrc=11495, ts=53274800, count=1871/1871, 1214 bytes
[2023-03-03 10:55:37.631][WARN][1][g50n4q35][11] handle udp pkt, count=107022/107022, err: code=5008(RtcStun)(RTC do STUN or ICE failed) : size=100, data=[00 01 00 50 21 12 a4 42] : no session, stun username=i148i286:Zepn, peer_id=145.224.100.30:51074, fast=-9023243327903571823
thread [1][g50n4q35]: cycle() [./src/app/srs_app_listener.cpp:743][errno=11]
thread [1][ttl88264]: on_udp_packet() [./src/app/srs_app_rtc_server.cpp:415][errno=11]
[2023-03-03 10:55:37.767][INFO][1][57zs1kr9] RTC: NACK ARQ seq=37501, ssrc=11705, ts=42197150, count=112891/112891, 1214 bytes
[2023-03-03 10:55:37.771][INFO][1][s91824se] RTC: NACK ARQ seq=2490, ssrc=12107, ts=965180, count=1/1, 1214 bytes
[2023-03-03 10:55:37.776][INFO][1][8967x130] RTC: NACK ARQ seq=2515, ssrc=12105, ts=974090, count=257/257, 1214 bytes
[2023-03-03 10:55:37.809][INFO][1][7q258725] RTC: NACK ARQ seq=2230, ssrc=11425, ts=67107150, count=8031/8031, 1214 bytes
[2023-03-03 10:55:37.810][INFO][1][x88520q7] RTC: NACK ARQ seq=25048, ssrc=11785, ts=36671940, count=3822/3822, 1214 bytes
[2023-03-03 10:55:37.812][INFO][1][8v72j079] RTC: NACK ARQ seq=17297, ssrc=12001, ts=5685030, count=233/233, 1214 bytes
[2023-03-03 10:55:37.812][INFO][1][6843o916] RTC: NACK ARQ seq=2006, ssrc=11523, ts=53366240, count=5163/5163, 1214 bytes
[2023-03-03 10:55:37.814][INFO][1][t3m893z1] RTC: NACK ARQ seq=24397, ssrc=11889, ts=36402480, count=30583/30583, 731 bytes
[2023-03-03 10:55:37.817][INFO][1][3525l48m] RTC: NACK ARQ seq=1930, ssrc=11489, ts=53285510, count=2920/2920, 1214 bytes
[2023-03-03 10:55:37.819][INFO][1][342qf11g] RTC: NACK ARQ seq=17874, ssrc=12011, ts=5847120, count=292/292, 1214 bytes
[2023-03-03 10:55:37.820][INFO][1][c68m42j9] RTC: NACK ARQ seq=2522, ssrc=12111, ts=959670, count=17/17, 1214 bytes
[2023-03-03 10:55:37.830][INFO][1][637473u9] RTC: NACK ARQ seq=2426, ssrc=12121, ts=941780, count=259/259, 1214 bytes
[2023-03-03 10:55:37.832][INFO][1][6sl7h780] RTC: NACK ARQ seq=18763, ssrc=11973, ts=6068360, count=76/76, 1214 bytes
[2023-03-03 10:55:37.833][INFO][1][f17i1b34] RTC: NACK ARQ seq=18049, ssrc=12005, ts=5904270, count=149/149, 1214 bytes
[2023-03-03 10:55:37.854][INFO][1][c2f793o8] RTC: NACK ARQ seq=1529, ssrc=11587, ts=53235540, count=1796/1796, 889 bytes
[2023-03-03 10:55:37.868][INFO][1][i594s76v] RTC: NACK ARQ seq=18081, ssrc=11991, ts=5913270, count=189/189, 1214 bytes
[2023-03-03 10:55:37.870][INFO][1][58rt448w] RTC: NACK ARQ seq=2480, ssrc=12139, ts=941760, count=20/20, 1214 bytes
[2023-03-03 10:55:37.953][INFO][1][sl65y724] RTC: session address change 91.102.183.167:8092 -> 91.102.183.167:8292, cached=1, nn_change=14/7, nn_address=2
[2023-03-03 10:55:37.954][INFO][1][h19zv914] RTC: NACK ARQ seq=24828, ssrc=11835, ts=36564030, count=3079/3079, 1214 bytes
[2023-03-03 10:55:37.964][INFO][1][b5si3592] RTC: NACK ARQ seq=2060, ssrc=11499, ts=53377130, count=3294/3294, 1214 bytes
[2023-03-03 10:55:38.054][INFO][1][n2y6e623] RTC: NACK ARQ seq=37836, ssrc=11735, ts=42340970, count=2532/2532, 1214 bytes
[2023-03-03 10:55:38.060][INFO][1][4b7e53v4] RTC: NACK ARQ seq=2598, ssrc=12159, ts=969210, count=16/16, 1214 bytes
[2023-03-03 10:55:38.061][INFO][1][1oaoi3wq] RTC: NACK ARQ seq=2637, ssrc=12113, ts=1001180, count=18/18, 482 bytes
[2023-03-03 10:55:38.061][INFO][1][2s8ogou1] RTC: NACK ARQ seq=2562, ssrc=12143, ts=965970, count=37/37, 1214 bytes
[2023-03-03 10:55:38.061][INFO][1][24512314] RTC: NACK ARQ seq=18571, ssrc=11997, ts=6034610, count=142/142, 1214 bytes
[2023-03-03 10:55:38.069][INFO][1][569fq59t] RTC: NACK ARQ seq=2731, ssrc=12103, ts=1031420, count=90/90, 1214 bytes
[2023-03-03 10:55:38.099][INFO][1][9cg4e508] RTC: NACK ARQ seq=24238, ssrc=11921, ts=36339390, count=34251/34251, 543 bytes
[2023-03-03 10:55:38.108][INFO][1][31734a35] DTLS: State Passive RECV, done=0, arq=0/0, r0=1, r1=0, len=571, cnt=22, size=299, hs=11
[2023-03-03 10:55:38.108][INFO][1][31734a35] DTLS: State Passive SEND, done=1, arq=0/0, r0=1, r1=0, len=1224, cnt=22, size=82, hs=2
[2023-03-03 10:55:38.108][INFO][1][31734a35] RTC: DTLS handshake done.
[2023-03-03 10:55:38.109][INFO][1][31734a35] RTC: session pub=0, sub=1, to=5000ms connection established
[2023-03-03 10:55:38.109][INFO][1][31734a35] RTC: Subscriber url=/live/8OZHQA6K established
[2023-03-03 10:55:38.109][INFO][1][31734a35] create consumer, no gop cache
[2023-03-03 10:55:38.109][INFO][1][31734a35] RTC: start play url=/live/8OZHQA6K, source_id=1z7h769f/u584x836, realtime=1, mw_msgs=0
[2023-03-03 10:55:38.133][INFO][1][7155u39g] RTC: NACK ARQ seq=24881, ssrc=11871, ts=36576180, count=7025/7025, 1214 bytes
[2023-03-03 10:55:38.151][INFO][1][61v7j5vy] RTC: NACK ARQ seq=24857, ssrc=11837, ts=36564030, count=482/482, 1214 bytes
[2023-03-03 10:55:38.208][INFO][1][31734a35] RTC: Jitter init base=200, value=3291
[2023-03-03 10:55:38.208][INFO][1][31734a35] RTC: Jitter rebase value=3291, last=0, distance=-3291, pkt-base=3291/3291, correct-base=200/0
[2023-03-03 10:55:38.208][INFO][1][31734a35] RTC: Jitter init base=20000, value=1274490
[2023-03-03 10:55:38.208][INFO][1][31734a35] RTC: Jitter rebase value=1274490, last=0, distance=-1274490, pkt-base=1274490/1274490, correct-base=20000/0
[2023-03-03 10:55:38.227][INFO][1][w87321sh] RTC: NACK ARQ seq=16308, ssrc=12093, ts=5355450, count=87960/87960, 1214 bytes
[2023-03-03 10:55:38.228][INFO][1][lti0e5fr] RTC: NACK ARQ seq=2605, ssrc=12129, ts=992270, count=264/264, 1214 bytes
[2023-03-03 10:55:38.230][INFO][1][1176r960] RTC: NACK ARQ seq=2061, ssrc=12147, ts=807300, count=536/536, 1214 bytes
[2023-03-03 10:55:38.363][INFO][1][a7557lwr] http: on_play ok, client_id=a7557lwr, url=http://host.docker.internal:2022/api/v1/hook/verify/sessions, request={"server_id":"vid-45a98o4","action":"on_play","client_id":"a7557lwr","ip":"192.168.192.1","vhost":"__defaultVhost__","app":"live","stream":"8OZHQA6K","tcUrl":"webrtc://aciem-induravit.online/live","param":"?secret=43ab299d-4fa6-4a40-8f9d-264f5549d742","pageUrl":"","stream_url":"/live/8OZHQA6K","stream_id":"vid-tv31196"}, response={"code":0}
=================================================================
==1==ERROR: AddressSanitizer: heap-use-after-free on address 0x61d0003a7b90 at pc 0x5645e1c42f10 bp 0x7fddca105b30 sp 0x7fddca1052d8
READ of size 13 at 0x61d0003a7b90 thread T1 (srs-hybrid-2)
    #0 0x5645e1c42f0f in __interceptor_memcpy.part.0 (/usr/local/srs/objs/srs+0x438f0f)
    #1 0x5645e269fe85 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forward_iterator_tag) (/usr/local/srs/objs/srs+0xe95e85)
    #2 0x5645e22575af in SrsRtcConnection::username[abi:cxx11]() src/app/srs_app_rtc_conn.cpp:1855
    #3 0x5645e232cb28 in SrsGoApiRtcPlay::serve_http(ISrsHttpResponseWriter*, ISrsHttpMessage*, SrsRtcUserConfig*) src/app/srs_app_rtc_api.cpp:245
    #4 0x5645e232a043 in SrsGoApiRtcPlay::do_serve_http(ISrsHttpResponseWriter*, ISrsHttpMessage*, SrsJsonObject*) src/app/srs_app_rtc_api.cpp:167
    #5 0x5645e2326f87 in SrsGoApiRtcPlay::serve_http(ISrsHttpResponseWriter*, ISrsHttpMessage*) src/app/srs_app_rtc_api.cpp:49
    #6 0x5645e1e98fb6 in SrsHttpServeMux::serve_http(ISrsHttpResponseWriter*, ISrsHttpMessage*) src/protocol/srs_protocol_http_stack.cpp:764
    #7 0x5645e1e9b8f4 in SrsHttpCorsMux::serve_http(ISrsHttpResponseWriter*, ISrsHttpMessage*) src/protocol/srs_protocol_http_stack.cpp:923
    #8 0x5645e20fd81c in SrsHttpConn::process_request(ISrsHttpResponseWriter*, ISrsHttpMessage*, int) src/app/srs_app_http_conn.cpp:232
    #9 0x5645e20fcf35 in SrsHttpConn::process_requests(SrsRequest**) src/app/srs_app_http_conn.cpp:205
    #10 0x5645e20fc39b in SrsHttpConn::do_cycle() src/app/srs_app_http_conn.cpp:159
    #11 0x5645e20fb80c in SrsHttpConn::cycle() src/app/srs_app_http_conn.cpp:104
    #12 0x5645e1fd1609 in SrsFastCoroutine::cycle() src/app/srs_app_st.cpp:285
    #13 0x5645e1fd1759 in SrsFastCoroutine::pfn(void*) src/app/srs_app_st.cpp:300
    #14 0x5645e237addd in _st_thread_main /srs/trunk/objs/Platform-SRS5-Linux-5.15.0-GCC9.4.0-x86_64/st-srs/sched.c:380
    #15 0x5645e237b703 in st_thread_create /srs/trunk/objs/Platform-SRS5-Linux-5.15.0-GCC9.4.0-x86_64/st-srs/sched.c:666
    #16 0x604005ef310f  (<unknown module>)

0x61d0003a7b90 is located 272 bytes inside of 2016-byte region [0x61d0003a7a80,0x61d0003a8260)
freed by thread T1 (srs-hybrid-2) here:
    #0 0x5645e1cb7f6f in operator delete(void*) (/usr/local/srs/objs/srs+0x4adf6f)

previously allocated by thread T1 (srs-hybrid-2) here:
    #0 0x5645e1cb6fd7 in operator new(unsigned long) (/usr/local/srs/objs/srs+0x4acfd7)

Thread T1 (srs-hybrid-2) created by T0 here:
    #0 0x5645e1be2295 in pthread_create (/usr/local/srs/objs/srs+0x3d8295)
    #1 0x5645e2208229 in SrsThreadPool::execute(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SrsCplxError* (*)(void*), void*) src/app/srs_app_threads.cpp:676
    #2 0x5645e2379712 in run_in_thread_pool() src/main/srs_main_server.cpp:517
    #3 0x5645e237917a in run_directly_or_daemon() src/main/srs_main_server.cpp:456
    #4 0x5645e237623c in do_main(int, char**, char**) src/main/srs_main_server.cpp:245
    #5 0x5645e237655c in main src/main/srs_main_server.cpp:256
    #6 0x7fddd8c20082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082)

SUMMARY: AddressSanitizer: heap-use-after-free (/usr/local/srs/objs/srs+0x438f0f) in __interceptor_memcpy.part.0
Shadow bytes around the buggy address:
  0x0c3a8006cf20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3a8006cf30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3a8006cf40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x0c3a8006cf50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c3a8006cf60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c3a8006cf70: fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c3a8006cf80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c3a8006cf90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c3a8006cfa0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c3a8006cfb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
  0x0c3a8006cfc0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07 
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
  Shadow gap:              cc
[2023-03-03 10:55:38.402][ERROR][1][a7557lwr][0] =================================================================
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0] ==1==ERROR: AddressSanitizer: heap-use-after-free on address 0x61d0003a7b90 at pc 0x5645e1c42f10 bp 0x7fddca105b30 sp 0x7fddca1052d8
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0] READ of size 13 at 0x61d0003a7b90 thread T1 (srs-hybrid-2)
sh: 1: addr2line: not found
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #0 0x5645e1c42f0f in __interceptor_memcpy.part.0 (/usr/local/srs/objs/srs+0x438f0f), r0=1094
sh: 1: addr2line: not found
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #1 0x5645e269fe85 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forward_iterator_tag) (/usr/local/srs/objs/srs+0xe95e85), r0=1094
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #2 0x5645e22575af in SrsRtcConnection::username[abi:cxx11]() src/app/srs_app_rtc_conn.cpp:1855, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #3 0x5645e232cb28 in SrsGoApiRtcPlay::serve_http(ISrsHttpResponseWriter*, ISrsHttpMessage*, SrsRtcUserConfig*) src/app/srs_app_rtc_api.cpp:245, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #4 0x5645e232a043 in SrsGoApiRtcPlay::do_serve_http(ISrsHttpResponseWriter*, ISrsHttpMessage*, SrsJsonObject*) src/app/srs_app_rtc_api.cpp:167, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #5 0x5645e2326f87 in SrsGoApiRtcPlay::serve_http(ISrsHttpResponseWriter*, ISrsHttpMessage*) src/app/srs_app_rtc_api.cpp:49, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #6 0x5645e1e98fb6 in SrsHttpServeMux::serve_http(ISrsHttpResponseWriter*, ISrsHttpMessage*) src/protocol/srs_protocol_http_stack.cpp:764, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #7 0x5645e1e9b8f4 in SrsHttpCorsMux::serve_http(ISrsHttpResponseWriter*, ISrsHttpMessage*) src/protocol/srs_protocol_http_stack.cpp:923, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #8 0x5645e20fd81c in SrsHttpConn::process_request(ISrsHttpResponseWriter*, ISrsHttpMessage*, int) src/app/srs_app_http_conn.cpp:232, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #9 0x5645e20fcf35 in SrsHttpConn::process_requests(SrsRequest**) src/app/srs_app_http_conn.cpp:205, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #10 0x5645e20fc39b in SrsHttpConn::do_cycle() src/app/srs_app_http_conn.cpp:159, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #11 0x5645e20fb80c in SrsHttpConn::cycle() src/app/srs_app_http_conn.cpp:104, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #12 0x5645e1fd1609 in SrsFastCoroutine::cycle() src/app/srs_app_st.cpp:285, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #13 0x5645e1fd1759 in SrsFastCoroutine::pfn(void*) src/app/srs_app_st.cpp:300, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #14 0x5645e237addd in _st_thread_main /srs/trunk/objs/Platform-SRS5-Linux-5.15.0-GCC9.4.0-x86_64/st-srs/sched.c:380, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #15 0x5645e237b703 in st_thread_create /srs/trunk/objs/Platform-SRS5-Linux-5.15.0-GCC9.4.0-x86_64/st-srs/sched.c:666, r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #16 0x604005ef310f  (<unknown module>), r0=1093
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0] 0x61d0003a7b90 is located 272 bytes inside of 2016-byte region [0x61d0003a7a80,0x61d0003a8260)
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0] freed by thread T1 (srs-hybrid-2) here:
sh: 1: addr2line: not found
[2023-03-03 10:55:38.404][ERROR][1][a7557lwr][0]     #0 0x5645e1cb7f6f in operator delete(void*) (/usr/local/srs/objs/srs+0x4adf6f), r0=1094
[2023-03-03 10:55:38.404][ERROR][1][a7557lwr][0] previously allocated by thread T1 (srs-hybrid-2) here:
sh: 1: addr2line: not found
[2023-03-03 10:55:38.404][ERROR][1][a7557lwr][0]     #0 0x5645e1cb6fd7 in operator new(unsigned long) (/usr/local/srs/objs/srs+0x4acfd7), r0=1094
[2023-03-03 10:55:38.404][ERROR][1][a7557lwr][0] Thread T1 (srs-hybrid-2) created by T0 here:
sh: 1: addr2line: not found
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]     #0 0x5645e1be2295 in pthread_create (/usr/local/srs/objs/srs+0x3d8295), r0=1094
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]     #1 0x5645e2208229 in SrsThreadPool::execute(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, SrsCplxError* (*)(void*), void*) src/app/srs_app_threads.cpp:676, r0=1093
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]     #2 0x5645e2379712 in run_in_thread_pool() src/main/srs_main_server.cpp:517, r0=1093
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]     #3 0x5645e237917a in run_directly_or_daemon() src/main/srs_main_server.cpp:456, r0=1093
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]     #4 0x5645e237623c in do_main(int, char**, char**) src/main/srs_main_server.cpp:245, r0=1093
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]     #5 0x5645e237655c in main src/main/srs_main_server.cpp:256, r0=1093
sh: 1: addr2line: not found
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]     #6 0x7fddd8c20082 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x24082), r0=1094
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0] SUMMARY: AddressSanitizer: heap-use-after-free (/usr/local/srs/objs/srs+0x438f0f) in __interceptor_memcpy.part.0
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0] Shadow bytes around the buggy address:
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   0x0c3a8006cf20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   0x0c3a8006cf30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   0x0c3a8006cf40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   0x0c3a8006cf50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   0x0c3a8006cf60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0] =>0x0c3a8006cf70: fd fd[fd]fd fd fd fd fd fd fd fd fd fd fd fd fd
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   0x0c3a8006cf80: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   0x0c3a8006cf90: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   0x0c3a8006cfa0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   0x0c3a8006cfb0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   0x0c3a8006cfc0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0] Shadow byte legend (one shadow byte represents 8 application bytes):
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Addressable:           00
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Partially addressable: 01 02 03 04 05 06 07 
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Heap left redzone:       fa
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Freed heap region:       fd
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Stack left redzone:      f1
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Stack mid redzone:       f2
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Stack right redzone:     f3
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Stack after return:      f5
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Stack use after scope:   f8
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Global redzone:          f9
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Global init order:       f6
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Poisoned by user:        f7
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Container overflow:      fc
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Array cookie:            ac
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Intra object redzone:    bb
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   ASan internal:           fe
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Left alloca redzone:     ca
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Right alloca redzone:    cb
[2023-03-03 10:55:38.405][ERROR][1][a7557lwr][0]   Shadow gap:              cc
==1==ABORTING

@xiaozhihong
Copy link
Collaborator

xiaozhihong commented Mar 7, 2023

During the htp_hook_on_play period, the rtc connection timed out and was deleted, resulting in the use of a wild pointer.

[2023-03-03 10:55:34.741][INFO][1][a7557lwr] RTC: disposing #1 resource(RtcConn)(0x61d0003a7a80), conns=384, disposing=3, zombies=0
[2023-03-03 10:55:38.363][INFO][1][a7557lwr] http: on_play ok, client_id=a7557lwr, url=http://host.docker.internal:2022/api/v1/hook/verify/sessions, request={"server_id":"vid-45a98o4","action":"on_play","client_id":"a7557lwr","ip":"192.168.192.1","vhost":"__defaultVhost__","app":"live","stream":"8OZHQA6K","tcUrl":"webrtc://aciem-induravit.online/live","param":"?secret=43ab299d-4fa6-4a40-8f9d-264f5549d742","pageUrl":"","stream_url":"/live/8OZHQA6K","stream_id":"vid-tv31196"}, response={"code":0}
[2023-03-03 10:55:38.402][ERROR][1][a7557lwr][0] =================================================================
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0] ==1==ERROR: AddressSanitizer: heap-use-after-free on address 0x61d0003a7b90 at pc 0x5645e1c42f10 bp 0x7fddca105b30 sp 0x7fddca1052d8
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0] READ of size 13 at 0x61d0003a7b90 thread T1 (srs-hybrid-2)
sh: 1: addr2line: not found
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #0 0x5645e1c42f0f in __interceptor_memcpy.part.0 (/usr/local/srs/objs/srs+0x438f0f), r0=1094
sh: 1: addr2line: not found
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #1 0x5645e269fe85 in void std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >::_M_construct<char*>(char*, char*, std::forward_iterator_tag) (/usr/local/srs/objs/srs+0xe95e85), r0=1094
[2023-03-03 10:55:38.403][ERROR][1][a7557lwr][0]     #2 0x5645e22575af in SrsRtcConnection::username[abi:cxx11]() src/app/srs_app_rtc_conn.cpp:1855, r0=1093

TRANS_BY_GPT3

@winlinvip
Copy link
Member

winlinvip commented Mar 7, 2023

@limjoe Please ask a question. I noticed that when you start Docker, SRS logs are printed to the console. I remember that after the container restarts, the logs cannot be found, right? I wonder how you find the logs of previous containers? Do you record the container ID?

TRANS_BY_GPT3

@winlinvip winlinvip changed the title AddressSanitizer: heap-use-after-free WebRTC: AddressSanitizer: heap-use-after-free Mar 7, 2023
@winlinvip winlinvip added the WebRTC WebRTC, RTC2RTMP or RTMP2RTC. label Mar 7, 2023
@limjoe
Copy link
Author

limjoe commented Mar 10, 2023

@winlinvip In k8s, using the command "kubectl logs -f origin-rtmp-cluster-1 --tail=100 -p" allows you to view the logs before the container crashes.

TRANS_BY_GPT3

@xiaozhihong
Copy link
Collaborator

xiaozhihong commented Mar 23, 2023

@limjoe, can you make this log above more complete? Or you can stop logging to the console and directly output it to a file.

TRANS_BY_GPT3

@winlinvip winlinvip added the TransByAI Translated by AI/GPT. label Jul 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug It might be a bug. TransByAI Translated by AI/GPT. WebRTC WebRTC, RTC2RTMP or RTMP2RTC.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants