[7.13.0-RC3] OSError in SocksServersComponent: [Errno 10048] error while attempting to bind on address

[kozlovsky]

While initializing the Socks5 server in Tribler 7.13.0-RC3, an OSError occurs due to a potential race condition between port selection and its utilization:

Traceback (most recent call last):
  File "tribler\core\components\component.py", line 39, in start
  File "tribler\core\components\socks_servers\socks_servers_component.py", line 26, in run
  File "tribler\core\components\socks_servers\socks5\server.py", line 31, in start
  File "asyncio\base_events.py", line 1463, in create_server
OSError: [Errno 10048] error while attempting to bind on address ('127.0.0.1', 63843):
only one usage of each socket address (protocol/network address/port) is normally permitted

The reason for the error is a time gap between the moment Tribler selects a random free port and the moment it starts using the port (in socks_server_component.py):

socks_server = Socks5Server(port=default_network_utils.get_random_free_port())
...
await socks_server.start()

When the gap is too wide, another process can grab the port before Socks5Server starts actually using it. When the gap is too small, I suspect it is possible that the operating system still believes sometimes that the port is occupied after it was temporarily taken inside the get_random_free_port() call.

The whole approach of using the default_network_utils.get_random_free_port() looks incorrect, as having a race condition with it is always possible.

The correct approach to grab a free port is to specify the value 0 for the port when starting the server. In this case, the operating system should automatically choose and assign the free port. Then we can retrieve the number of the given port and remember it for later usage.

As I understand it, the current logic with default_network_utils.get_random_free_port() was implemented this way because historically, Tribler consisted of parts with an order of initialization different from the order of dependencies, so it was necessary to have the port value before the server is actually started.

Now Tribler consists of components with clear dependencies, and their initialization order corresponds with dependency order. That allows us to retire the error-prone default_network_utils.get_random_free_port() usage and ask the OS for the port directly when starting the server.

We have similar logic in several places:

1. When starting Socks5Server (see OSError asyncio.base_events in create_server: [Errno 10048] error while attempting to bind on address ('127.0.0.1', 8307) #6149, Address already in use (now with international flavor!) #5437)
2. When starting Libtorrent
3. When starting the REST API (see the issues Weird bug  #6767, [7.5.2] OSError: [Errno 10048] error while attempting to bind on address ('127.0.0.1', 8085) #5547). In this latter case, the free port is first searched by Tribler GUI, and the found value then passed to the Tribler Core. After that, the Tribler Core tries to grab the port in a range by performing a loop. After that, the port value is reported back to Tribler GUI. All this complicated machinery is vulnerable to race conditions and is unnecessary now: it is possible for Tribler Core to request a free port from OS, start the REST API server and pass the port value to Tribler GUI via the processes database.

To do a correct fix we should stop rely on default_network_utils.get_random_free_port() and instead pass the port value zero when opening any listening port to allow OS to choose a free port without any race condition.

[sentry-for-tribler]

Sentry issue: TRIBLER-19G