#4587 Always check transport is not closing before reuse

[Dahuage]

What do these changes do?

Always check transport is not closing before reuse a connection.

Reuse a protocol based on keepalive in headers is unreliable.
For example, uWSGI will not support keepalive even it serves a
http1.1 request, except explicitly configure uWSGI with a
--http-keepalive option.

Servers designed like uWSGI would cause aiohttp intermittently
raise a ConnectionResetException when the protocol poll runs
out and some protocol is reused.

Though it can be bypassed by properly configuring either in server like uWSGI or aiohttp side, this exception is confusing and annoying for new guys to aiohttp.
Yet more importantly, sometimes it can not be triggered in a development env, but it is a potential disaster when in a production env where it has high concurrence.

Are there changes in behavior for the user?

No

Related issue number

#4587

Checklist

• I think the code is well written
• Unit tests for the changes exist
• Documentation reflects the changes
• If you provide code modification, please add yourself to CONTRIBUTORS.txt
  • The format is <Name> <Surname>.
  • Please keep alphabetical order, the file is sorted by names.
• Add a new news fragment into the CHANGES folder
  • name it <issue_id>.<type> for example (588.bugfix)
  • if you don't have an issue_id change it to the pr id after creating the pr
  • ensure type is one of the following:
    • .feature: Signifying a new feature.
    • .bugfix: Signifying a bug fix.
    • .doc: Signifying a documentation improvement.
    • .removal: Signifying a deprecation or removal of public API.
    • .misc: A ticket has been closed, but it is not of interest to users.
  • Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."

[codecov-commenter]

Codecov Report

Merging #4778 into master will decrease coverage by 0.03%.
The diff coverage is 55.55%.

@@            Coverage Diff             @@
##           master    #4778      +/-   ##
==========================================
- Coverage   97.60%   97.57%   -0.04%     
==========================================
  Files          43       43              
  Lines        8932     8940       +8     
  Branches     1406     1408       +2     
==========================================
+ Hits         8718     8723       +5     
- Misses         95       97       +2     
- Partials      119      120       +1     

Impacted Files	Coverage Δ
aiohttp/connector.py	95.85% <50.00%> (-0.44%)	⬇️
aiohttp/client_proto.py	96.66% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ffa4bf2...b343169. Read the comment docs.

[kkonevets]

@Dahuage Hi, you said one can configure properly on aiohttp side to avoid the error, how can I do it?

[Dahuage]

@Dahuage Hi, you said one can configure properly on aiohttp side to avoid the error, how can I do it?

if the error is caused by the server you requested disable keep-alive.

then config the aiohttp send http1.0 request to the server may avoid this problem.

just init the session like this:

async def main():
    async with aiohttp.ClientSession(version = aiohttp.http.HttpVersion10) as sess:
        await do_sth_with_sess(sess)

by the way, if the server you requested is under your control, make sure your server enable HTTP/1.1 keepalive connections.

hope it helps!

[kkonevets]

@Dahuage thanks for the answer. The server is not under my control, so I just enabled http1.0, looks like it works

[IsaacFlos]

It's too cool！

[945941192]

Thank you for your contribution to help me solve the problem for a long time.

[mncu]

wow ! cool !

[WoodyFleurant]

Seems that one test fail

[aorumbayev]

Any updates on this ?

[cdeboeser]

Would also be very interested in this fix! :) And thank you for the contribution!

[codecov-io]

Codecov Report

Merging #4778 into master will decrease coverage by 0.02%.
The diff coverage is 55.55%.

@@            Coverage Diff             @@
##           master    #4778      +/-   ##
==========================================
- Coverage   97.58%   97.56%   -0.03%     
==========================================
  Files          43       43              
  Lines        8960     8968       +8     
  Branches     1414     1416       +2     
==========================================
+ Hits         8744     8750       +6     
- Misses         97       98       +1     
- Partials      119      120       +1     

Flag	Coverage Δ
#unit	97.56% <55.55%> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
aiohttp/connector.py	96.05% <50.00%> (-0.43%)	⬇️
aiohttp/client_proto.py	96.66% <100.00%> (ø)
aiohttp/web_fileresponse.py	98.38% <0.00%> (+0.53%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cc7c4dd...9fda0e1. Read the comment docs.

[asvetlov]

Thanks!

[github-actions]

💚 Backport successful

The PR was backported to the following branches:

• ✅ 3.7