_SelectorSocketTransport.writelines is missing a flow control check allowing writes to fill memory until exhausted

[bdraco]

Bug report

Bug description:

This is the public issue for GHSA-fw89-6wjj-8j95

CPython versions tested on:

3.12

Operating systems tested on:

Linux, macOS

Linked PRs

• gh-127655: Ensure _SelectorSocketTransport.writelines pauses the protocol if needed #127656
• [3.13] gh-127655: Ensure _SelectorSocketTransport.writelines pauses the protocol if needed (GH-127656) #127663
• [3.12] gh-127655: Ensure _SelectorSocketTransport.writelines pauses the protocol if needed (GH-127656) #127664

[picnixz]

@bdraco I put all security labels but does it affect 3.9/3.10/3.11? (can't check for now)

[bdraco]

Its 3.12+ only since writelines was implemented in asyncio selector transport in 3.12

[kumaraditya303]

Thanks @bdraco

I assume aiohttp would be affected by this too if it uses writelines. If possible do share any performance numbers you have of how much impact the zero copy writes have made to aiohttp.

[bdraco]

Thanks @bdraco

I assume aiohttp would be affected by this too if it uses writelines. If possible do share any performance numbers you have of how much impact the zero copy writes have made to aiohttp.

@kumaraditya303

aio-libs/aiohttp#10126 is the PR from a few hours ago that turned off zero copy writes which was released as part of aiohttp 3.11.10

The codspeed report shows the degraded performances however its not so great because we were pushing the CI pretty hard at the time and it wasn't keeping up. codspeed is set to only show changes >8% on GH so you'd have to look at the report directly to get a better idea.

This run https://codspeed.io/aio-libs/aiohttp/runs/675241d947498039ef29339c is probably the best one to look at for aiohttp 3.11.x. For aiohttp master branch https://codspeed.io/aio-libs/aiohttp/branches/disable_zero_copy as its going to show the difference the memcpy makes.

The relevant benchmarks that show a real difference are (keep in mind we are benchmarking disabling zero copy writes):

test_one_hundred_get_requests_with_1024_chunked_payload +3% (writelines is a bit worse for tiny payloads)
test_one_hundred_get_requests_with_512kib_chunked_payload -21% (writelines is a lot better for large payloads)
test_one_hundred_get_requests_with_30000_chunked_payload -4% (writelines is a slightly better for medium size payloads)

Note that the margin of error is ~1-3% with the benchmarking (but if you have experience and a bit of wetware memory of previous runs, you can look at the flame-graphs you can usually tell if its error or actual change). Based on that, I'd say the above seems accurate and the error rate here is likely <1% between runs.

Also benchmarks above were run with 3.12. We switched to benchmarking with 3.13 now that the first patch release, 3.13.1 is out a few hours after these runs. https://codspeed.io/aio-libs/aiohttp/runs/67524ceb47498039ef2933c9 is the first run with Python 3.13 for aiohttp 3.11 and shows a few unrelated (to this issue) improvements. More details in aio-libs/aiohttp#10131

[bdraco]

Closing issue as linked PRs are merged