Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Should not crash if no more available ports #222

Closed
ibc opened this issue Nov 12, 2018 · 4 comments
Closed

Should not crash if no more available ports #222

ibc opened this issue Nov 12, 2018 · 4 comments
Assignees
Labels
Milestone

Comments

@ibc
Copy link
Member

ibc commented Nov 12, 2018

This must be confirmed, but it seems that, if there are no more available RTP ports, the mediasoup-worker process crashes. Not sure if this is intentional. AFAIR it's not. Instead, the JS promise should just be rejected.

2018-11-12T17:31:15.228Z mediasoup:ERROR:mediasoup-worker [id:zuggrvev#9] RTC::UdpSocket::GetRandomPort() | throwing MediaSoupError | no more available ports for IP 'X.X.X.X'
2018-11-12T17:31:15.230Z mediasoup:ERROR:mediasoup-worker [id:zuggrvev#9] RTC::WebRtcTransport::WebRtcTransport() | error adding IPv4 UDP socket: no more available ports for IP 'X.X.X.X'
2018-11-12T17:31:15.230Z mediasoup:ERROR:mediasoup-worker [id:zuggrvev#9] RTC::WebRtcTransport::WebRtcTransport() | throwing MediaSoupError | could not open any IP:port
2018-11-12T17:31:15.411Z mediasoup:ERROR:Worker child process exited [id:zuggrvev#9, code:null, signal:SIGSEGV]
2018-11-12T17:31:15.541Z mediasoup:ERROR:Peer "router.createWebRtcTransport" request failed: InvalidStateError: Channel closed
2018-11-12T17:31:15.546Z mediasoup:ERROR:Peer receiveRequest() failed [method:createTransport]: InvalidStateError: Channel closed at klass (/home/ubuntu/mediasoup-demo-cosmo/server/node_modules/mediasoup/lib/errors.js:7:4) at Object.close (/home/ubuntu/mediasoup-demo-cosmo/server/node_modules/mediasoup/lib/Channel.js:247:14) at Channel.close (/home/ubuntu/mediasoup-demo-cosmo/server/node_modules/mediasoup/lib/Channel.js:170:9) at Worker.close (/home/ubuntu/mediasoup-demo-cosmo/server/node_modules/mediasoup/lib/Worker.js:132:17) at ChildProcess.Worker._child.on (/home/ubuntu/mediasoup-demo-cosmo/server/node_modules/mediasoup/lib/Worker.js:101:9) at emitTwo (events.js:106:13) at ChildProcess.emit (events.js:191:7) at Process.ChildProcess._handle.onexit (internal/child_process.js:219:12)
@ibc ibc added the bug label Nov 12, 2018
@ibc ibc added this to the v2 updates milestone Nov 12, 2018
@ibc ibc assigned ibc and jmillan Nov 12, 2018
@ibc
Copy link
Member Author

ibc commented Nov 16, 2018

Better logs showing an assertion error in libuv that produces the real crash:

  mediasoup:Channel request() [method:router.createWebRtcTransport, id:60685604] +660ms
  mediasoup:ERROR:mediasoup-worker [id:oqagohae#1] RTC::UdpSocket::GetRandomPort() | throwing MediaSoupError | no more available ports for IP '192.168.1.36' +0ms
  mediasoup:ERROR:mediasoup-worker [id:oqagohae#1] RTC::WebRtcTransport::WebRtcTransport() | error adding IPv4 UDP socket: no more available ports for IP '192.168.1.36' +0ms
  mediasoup:ERROR:mediasoup-worker [id:oqagohae#1] RTC::WebRtcTransport::WebRtcTransport() | throwing MediaSoupError | could not open any IP:port +1ms
mediasoup-worker's stderr: Assertion failed: (!uv__is_closing(handle)), function uv_close, file ../deps/libuv/src/unix/core.c, line 117.
  mediasoup:Channel channel ended by the other side +5ms
  mediasoup:ERROR:Worker child process exited [id:oqagohae#1, code:null, signal:SIGABRT]

Core dump:

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_CRASH (SIGABRT)
Exception Codes:       0x0000000000000000, 0x0000000000000000
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
Assertion failed: (!uv__is_closing(handle)), function uv_close, file ../deps/libuv/src/unix/core.c, line 117.
 

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   libsystem_kernel.dylib        	0x00007fff7b1feb86 __pthread_kill + 10
1   libsystem_pthread.dylib       	0x00007fff7b2b4c50 pthread_kill + 285
2   libsystem_c.dylib             	0x00007fff7b1681c9 abort + 127
3   libsystem_c.dylib             	0x00007fff7b130868 __assert_rtn + 320
4   mediasoup-worker              	0x000000010e91422a uv_close + 278
5   mediasoup-worker              	0x000000010e8f304f Timer::Destroy() + 25 (Timer.cpp:49)
6   mediasoup-worker              	0x000000010e8c040e RTC::Transport::~Transport() + 206 (vector:439)
7   mediasoup-worker              	0x000000010e8c56fa RTC::WebRtcTransport::WebRtcTransport(RTC::Transport::Listener*, Channel::Notifier*, unsigned int, RTC::WebRtcTransport::Options&) + 3930 (WebRtcTransport.cpp:179)
8   mediasoup-worker              	0x000000010e8aa7c1 RTC::Router::HandleRequest(Channel::Request*) + 3907 (Router.cpp:228)
9   mediasoup-worker              	0x000000010e893166 Channel::UnixStreamSocket::UserOnUnixStreamRead() + 216 (UnixStreamSocket.cpp:295)
10  mediasoup-worker              	0x000000010e91b9dc uv__stream_io + 1265 (stream.c:1260)
11  mediasoup-worker              	0x000000010e922397 uv__io_poll + 1735
12  mediasoup-worker              	0x000000010e914531 uv_run + 314 (core.c:371)
13  mediasoup-worker              	0x000000010e88e73d Worker::Worker(Channel::UnixStreamSocket*) + 301 (Worker.cpp:37)
14  mediasoup-worker              	0x000000010e8f5c02 main + 1474 (main.cpp:94)
15  libdyld.dylib                 	0x00007fff7b0c008d start + 1

@ibc
Copy link
Member Author

ibc commented Nov 16, 2018

Ok, the problem is the following:

  • When there are no more ports available, the WebRtcTransport constructor fails so it calls delete this and, after that, it throws an exception.
  • The ~WebRtcTransport destructor is called and then the parent ~Transport destructor is called.
  • The ~Transport destructor destroys the RTCP timer by calling this->rtcpTimer->Destroy().
  • After the destructors are called, the WebRtcTransport constructor throws an exception.
  • Somehow, this exception causes that the ~Transport destructor is called again, so it calls this->rtcpTimer->Destroy() again and here the obvious crash (the timer was already deleted).

Custom logs to show the problem:

  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] RTC::Transport::Transport() | Transport::Transport() +160ms
  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] Timer::Timer() | Timer::Timer() +0ms
  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] RTC::WebRtcTransport::WebRtcTransport() | WebRtcTransport::WebRtcTransport() +0ms
  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] RTC::UdpSocket::GetRandomPort() | throwing MediaSoupError | no more available ports for IP '192.168.1.36' +0ms
  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] RTC::WebRtcTransport::WebRtcTransport() | error adding IPv4 UDP socket: no more available ports for IP '192.168.1.36' +0ms
  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] RTC::WebRtcTransport::~WebRtcTransport() | WebRtcTransport::~WebRtcTransport() +0ms
  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] RTC::Transport::~Transport() | Transport::~Transport() +0ms
  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] Timer::Destroy() | Timer::Destroy() +1ms
  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] RTC::WebRtcTransport::WebRtcTransport() | throwing MediaSoupError | could not open any IP:port +0ms
  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] RTC::Transport::~Transport() | Transport::~Transport() +0ms
  mediasoup:ERROR:mediasoup-worker [id:cayzcbno#1] Timer::Destroy() | Timer::Destroy() +0ms
mediasoup-worker's stderr: Assertion failed: (!uv__is_closing(handle)), function uv_close, file ../deps/libuv/src/unix/core.c, line 117.

Some related info:

https://isocpp.org/wiki/faq/exceptions#selfcleaning-members

@ibc
Copy link
Member Author

ibc commented Nov 16, 2018

So, theoretically, when a C++ class constructor throws an exception, its destructor is NOT called. The problem we have is that, after throwing the exception in the WebRtcTransport constructor, the destructor of its parent class Transport is being called! But it was already called before when calling delete this in the WebRtcTransport constructor.

And it seems to be the expected behavior:

https://stackoverflow.com/questions/3759270/will-the-destructor-of-the-base-class-called-if-an-object-throws-an-exception-in

So there our bug.

@ibc
Copy link
Member Author

ibc commented Nov 16, 2018

@jmillan this will also affect the PlainRtpTransport class. In fact, it will leak if for any reason SetRemoteParameters() fails in the constructor.

@ibc ibc closed this as completed in de3d89f Nov 16, 2018
lavarsicious pushed a commit to lavarsicious/mediasoup that referenced this issue Feb 5, 2019
- The destructor of ~Transport was being called twice
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

No branches or pull requests

2 participants