ClientSession keep transferring data after closed

[dmmartins]

Long story short

After a ClientSession is closed it keeps connected to the server and keeps transferring data.

Steps to reproduce

import asyncio
import aiohttp

async def download():
    url = 'http://example.com/large_file.iso'
    async with aiohttp.ClientSession() as http:
        async with http.get(url) as response:
            pass
    while True:
        await asyncio.sleep(1)

loop = asyncio.get_event_loop()
loop.run_until_complete(download())

When I run this code the process cpu usage goes to 90+% and the memory usage keeps increasing.
Also I can see the process is still connected to the server.

I tried explicitly closing client session but it has same behavior.

[webknjaz]

@asvetlov has accidentally closed this with misleading commit reference.

[wumpus]

The streaming interface does the right thing: if you call response.content.read(1000) and then response.close(), looks like only the first 256k of the response is read (a single read). I observed this using strace, if you're on Linux it's worth using strace instead of looking at the process size etc.

I used strace with your example and it behaved the same as the streaming interface (at least in Linux), a single system call read that got 256k of the file. Here's what it looks like with timestamps:

12:39:13 recvfrom(7, "HTTP/1.1 200 OK\r\nDate: Tue, 26 D"..., 262144, 0, NULL, NULL) = 262144
12:39:13 brk(NULL)                      = 0x1cfe000
12:39:13 brk(0x1d3f000)                 = 0x1d3f000
12:39:13 epoll_ctl(4, EPOLL_CTL_DEL, 7, 0x7ffdbdb76c20) = 0
12:39:13 epoll_wait(4, [], 1, 0)        = 0
12:39:13 epoll_wait(4, [], 1, 0)        = 0
12:39:13 close(7)                       = 0
12:39:13 epoll_wait(4, [], 1, 1000)     = 0
12:39:14 epoll_wait(4, [], 1, 0)        = 0
12:39:14 epoll_wait(4, [], 1, 1000)     = 0
12:39:15 epoll_wait(4, [], 1, 0)        = 0
12:39:15 epoll_wait(4, [], 1, 1000)     = 0
12:39:16 epoll_wait(4, [], 1, 0)        = 0
12:39:16 epoll_wait(4, [], 1, 1000)     = 0
12:39:17 epoll_wait(4, [], 1, 0)        = 0
12:39:17 epoll_wait(4, [], 1, 1000)     = 0
12:39:18 epoll_wait(4, [], 1, 0)        = 0
12:39:18 epoll_wait(4, [], 1, 1000)     = 0
12:39:19 epoll_wait(4, [], 1, 0)        = 0
12:39:19 epoll_wait(4, [], 1, 1000)     = 0
12:39:20 epoll_wait(4, [], 1, 0)        = 0
12:39:20 epoll_wait(4, [], 1, 1000)     = 0
12:39:21 epoll_wait(4, [], 1, 0)        = 0
12:39:21 epoll_wait(4, [], 1, 1000)     = 0
12:39:22 epoll_wait(4, [], 1, 0)        = 0
12:39:22 epoll_wait(4, [], 1, 1000)     = 0
12:39:23 epoll_wait(4, [], 1, 0)        = 0
12:39:23 epoll_wait(4, [], 1, 1000)     = 0
12:39:24 epoll_wait(4, [], 1, 0)        = 0
12:39:24 epoll_wait(4, 0x7fd9e74cc620, 1, 1000) = -1 EINTR (Interrupted system call)
12:39:24 --- SIGINT {si_signo=SIGINT, si_code=SI_KERNEL} ---

The EINTR is when I hit ^C