Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable cleanup_closed for aiohttp.TCPConnector with cpython 3.11.1+ #93013

Merged
merged 2 commits into from
May 14, 2023

Conversation

bdraco
Copy link
Member

@bdraco bdraco commented May 13, 2023

Proposed change

There is currently a relatively fast memory leak when using cpython 3.11.1+ and cleanup_closed with aiohttp

For my production instance it was leaking ~450MiB per day of MemoryBIO, SSLProtocol, SSLObject, _SSLProtocolTransport memoryview, and managedbuffer objects

see aio-libs/aiohttp#7252 see python/cpython#98540

We should be able to replace this with aio-libs/aiohttp#7280 in the future once 3.8.5 is released but that's months away at this point aio-libs/aiohttp#7252 (comment)

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

There is currently a relatively fast memory leak when using
cpython 3.11.2+ and cleanup_closed with aiohttp

For my production instance it was leaking ~450MiB per day
of `MemoryBIO`, `SSLProtocol`, `SSLObject`, `_SSLProtocolTransport`
`memoryview`, and `managedbuffer` objects

see aio-libs/aiohttp#7252
see python/cpython#98540
@home-assistant home-assistant bot added bugfix cla-signed core small-pr PRs with less than 30 lines. labels May 13, 2023
@bdraco bdraco added this to the 2023.5.3 milestone May 13, 2023
@bdraco bdraco changed the title Disable cleanup_closed for aiohttp.TCPConnector with cpython 3.11.2+ Disable cleanup_closed for aiohttp.TCPConnector with cpython 3.11.1+ May 13, 2023
bdraco added a commit that referenced this pull request May 13, 2023
@bdraco bdraco marked this pull request as ready for review May 13, 2023 20:36
@bdraco bdraco requested a review from a team as a code owner May 13, 2023 20:36
@balloob balloob merged commit 7d371a3 into dev May 14, 2023
@balloob balloob deleted the disable_cleanup_closed_newer_python branch May 14, 2023 00:15
balloob pushed a commit that referenced this pull request May 14, 2023
…93013)

* Disable cleanup_closed for aiohttp.TCPConnector with cpython 3.11.2+

There is currently a relatively fast memory leak when using
cpython 3.11.2+ and cleanup_closed with aiohttp

For my production instance it was leaking ~450MiB per day
of `MemoryBIO`, `SSLProtocol`, `SSLObject`, `_SSLProtocolTransport`
`memoryview`, and `managedbuffer` objects

see aio-libs/aiohttp#7252
see python/cpython#98540

* Update homeassistant/helpers/aiohttp_client.py
@balloob balloob mentioned this pull request May 14, 2023
@github-actions github-actions bot locked and limited conversation to collaborators May 15, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants