You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I use aresponses, but essentially any server might work e.g. RawTestServer.
a request hander that sleeps long enough (5 seconds) before returning an empty response ({}).
an aiohttp client that is forced to time out fast enough (0.1 seconds) AND that closes its own connections also fast enough, before the server begins the shutdown (<0.2s).
Because this happens in a test suite, everything typically happens very fast: the server spin up, the request & handling, and the server shutdown via context manager — there is no time for "idle" connection closing.
The client behaves as expected. The server, however, leaves 2 tasks pending after the server itself has "exited" via the content manager:
Task was destroyed but it is pending!
task: <Task pending name='Task-10' coro=<RequestHandler._handle_request() running at …/site-packages/aiohttp/web_protocol.py:442> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[Task.task_wakeup()]>
Task was destroyed but it is pending!
task: <Task pending name='Task-8' coro=<RequestHandler.start() running at …/site-packages/aiohttp/web_protocol.py:521> wait_for=<Task pending name='Task-10' coro=<RequestHandler._handle_request() running at …/site-packages/aiohttp/web_protocol.py:442> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[Task.task_wakeup()]>>
After some deep dive, I figured out that the task in RequestHandler._task_handler is created but never cancelled nor awaited when the connection is lost. More on that, the field is set to None and is not shut down later when the server shuts down (via [conn.shutdown() for conn in self._connections]) — because the self._connections is empty by that time.
If I do the manual hacking in RequestHandler.connection_lost() to add cancelling:
… then it helps to avoid pending/orphan tasks in the end.
I am not sure what would be a proper fix — all these connection_made/connection_lost methods are sync, so I cannot use await there to properly cancel-and-wait for the task.
Can you please hint where/how to fix this the best way?
To Reproduce
The timing is only for convenicence — to see the difference between 0.1s timeout and 5.0s shutdown.
The repro is standalone and works in Python 3.11.4, 3.7.9, and basically everywhere where I tested. The aiohttp is the most recent one.
fromaiohttp.webimport*fromaiohttp.test_utilsimport*t0=Noneasyncdefhandler(request):
awaitasyncio.sleep(5)
returnjson_response({})
asyncdefclient_side(host, port):
globalt0# Simulate the typical client-side behaviour:session=aiohttp.ClientSession(connector=aiohttp.TCPConnector(limit=0))
timeout=aiohttp.ClientTimeout(total=0.1)
asyncwithsession:
# t1 = asyncio.get_running_loop().time()# print(f"⏰ starting the request = {t1-t0:.6f}")withcontextlib.suppress(asyncio.TimeoutError):
awaitsession.get(f'http://{host}:{port}/', timeout=timeout)
# t2 = asyncio.get_running_loop().time()# print(f"⏰ finished the request = {t2-t0:.6f} / {t2-t1:.6f}")# t3 = asyncio.get_running_loop().time()# print(f"⏰ exited the inner ctx mgr = {t3-t0:.6f}")asyncdefmain():
globalt0, t1before=asyncio.all_tasks()
t0=asyncio.get_running_loop().time()
asyncwithRawTestServer(handler) asserver:
awaitclient_side(host=server.host, port=server.port)
# Let the client-side session close its connections (server-side: "connection lost"),# but stay within the request handler's time before server shutdown (0.1+0.2 < 5).awaitasyncio.sleep(0.2)
# t4 = asyncio.get_running_loop().time()# print(f"⏰ exited the outer mgr = {t4-t0:.6f}")after=asyncio.all_tasks()
fortinafter-before:
print(f"💥Unattended asyncio task: {t!r}")
if__name__=='__main__':
asyncio.run(main())
Expected behavior
No unattended tasks are left after exiting the server.
In the logs below, I've added a few manual prints in aiohttp's code — notice that the CONN LOST happens BEFORE the SERVER SHUTDOWN. That is important. Without the 0.2s sleep, they happen in the proper order: server shutdown, then the client goes away — and there is no issue.
Logs/tracebacks
>>>> SERVER CONN MADE len(self._connections)=1 handler=<RequestHandler connected>
>>>> SERVER CONN LOST handler=<RequestHandler connected> exc=None
>>> ReqHand cancelling the task
>>>> SERVER shutdown len(self._connections)=0
💥Unattended asyncio task: <Task pending name='Task-6' coro=<RequestHandler._handle_request() running at /Users/nolar/.pyenv/versions/kopf3114/lib/python3.11/site-packages/aiohttp/web_protocol.py:443> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[Task.task_wakeup()]>
💥Unattended asyncio task: <Task pending name='Task-4' coro=<RequestHandler.start() running at /Users/nolar/.pyenv/versions/kopf3114/lib/python3.11/site-packages/aiohttp/web_protocol.py:522> wait_for=<Task pending name='Task-6' coro=<RequestHandler._handle_request() running at /Users/nolar/.pyenv/versions/kopf3114/lib/python3.11/site-packages/aiohttp/web_protocol.py:443> wait_for=<Future pending cb=[Task.task_wakeup()]> cb=[Task.task_wakeup()]>>
Describe the bug
The setup:
aresponses
, but essentially any server might work e.g.RawTestServer
.{}
).Because this happens in a test suite, everything typically happens very fast: the server spin up, the request & handling, and the server shutdown via context manager — there is no time for "idle" connection closing.
The client behaves as expected. The server, however, leaves 2 tasks pending after the server itself has "exited" via the content manager:
After some deep dive, I figured out that the task in
RequestHandler._task_handler
is created but never cancelled nor awaited when the connection is lost. More on that, the field is set toNone
and is not shut down later when the server shuts down (via[conn.shutdown() for conn in self._connections]
) — because theself._connections
is empty by that time.If I do the manual hacking in
RequestHandler.connection_lost()
to add cancelling:… then it helps to avoid pending/orphan tasks in the end.
I am not sure what would be a proper fix — all these
connection_made
/connection_lost
methods are sync, so I cannot useawait
there to properly cancel-and-wait for the task.Can you please hint where/how to fix this the best way?
To Reproduce
The timing is only for convenicence — to see the difference between 0.1s timeout and 5.0s shutdown.
The repro is standalone and works in Python 3.11.4, 3.7.9, and basically everywhere where I tested. The aiohttp is the most recent one.
Expected behavior
No unattended tasks are left after exiting the server.
In the logs below, I've added a few manual prints in aiohttp's code — notice that the CONN LOST happens BEFORE the SERVER SHUTDOWN. That is important. Without the 0.2s sleep, they happen in the proper order: server shutdown, then the client goes away — and there is no issue.
Logs/tracebacks
Python Version
3.11.4
aiohttp Version
multidict Version
yarl Version
OS
MacOS
Related component
Server
Additional context
No response
Code of Conduct
The text was updated successfully, but these errors were encountered: