Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch to using the AsyncResolver with aiohttp #114529

Merged
merged 18 commits into from
Apr 6, 2024
Merged

Switch to using the AsyncResolver with aiohttp #114529

merged 18 commits into from
Apr 6, 2024

Conversation

bdraco
Copy link
Member

@bdraco bdraco commented Mar 31, 2024 •
edited
Loading

Needs some tests fixed

Proposed change

AsyncResolver avoids creating executor jobs to do DNS resolution. This is key to further reducing startup time, as the DNS cache is not built yet, and integrations are heavily competing for the executor. Additionally, having to create executor jobs to make async HTTP connections negatively impacts performance and latency.

https://docs.aiohttp.org/en/stable/#library-installation

For speeding up DNS resolving by client API you may install aiodns as well. This option is highly recommended:

AsyncResolver was not usable before aio-libs/aiohttp#8270 because it did not fallback to A records and only returned AAAA records in most cases when IPv6 was available.

This is a backport of aio-libs/aiohttp#8270

Making aiodns a top level dep should have very minimal impact as aiodns is already required by dhcp, which is required by default config

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Deprecation (breaking change to happen in the future)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • I have followed the perfect PR recommendations
  • The code has been formatted using Ruff (ruff format homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

To help with the load of incoming pull requests:

@bdraco
Bump aiodns to 3.2.0
1467d29 
changelog: aio-libs/aiodns@v3.1.1...v3.2.0
@bdraco
Switch to using the AsyncResolver with aiohttp
601e682 
This avoids creating executor jobs to do DNS resolution

AsyncResolver was not usable before aio-libs/aiohttp#8270
because it did not fallback to fallback to A records and only returned AAAA records
in most cases when IPv6 was available

This is a backport of aio-libs/aiohttp#8270
@bdraco
Switch to using the AsyncResolver with aiohttp
7102d99 
This avoids creating executor jobs to do DNS resolution

AsyncResolver was not usable before aio-libs/aiohttp#8270
because it did not fallback to fallback to A records and only returned AAAA records
in most cases when IPv6 was available

This is a backport of aio-libs/aiohttp#8270
@bdraco
Switch to using the AsyncResolver with aiohttp
7227ff8 
This avoids creating executor jobs to do DNS resolution

AsyncResolver was not usable before aio-libs/aiohttp#8270
because it did not fallback to fallback to A records and only returned AAAA records
in most cases when IPv6 was available

This is a backport of aio-libs/aiohttp#8270
@bdraco
Switch to using the AsyncResolver with aiohttp
1b1abfa 
This avoids creating executor jobs to do DNS resolution

AsyncResolver was not usable before aio-libs/aiohttp#8270
because it did not fallback to fallback to A records and only returned AAAA records
in most cases when IPv6 was available

This is a backport of aio-libs/aiohttp#8270
@home-assistant
Copy link

Hey there @gjohansson-ST, mind taking a look at this pull request as it has been labeled with an integration (dnsip) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of dnsip can trigger bot actions by commenting:

  • @home-assistant close Closes the pull request.
  • @home-assistant rename Awesome new title Renames the pull request.
  • @home-assistant reopen Reopen the pull request.
  • @home-assistant unassign dnsip Removes the current integration label and assignees on the pull request, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the pull request.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the pull request.

@bdraco
Copy link
Member Author

bdraco commented Mar 31, 2024

Test failures are likely unmocked http calls in integrations. Since it ran in the executor before the executor would be shut down and it wouldn't linger, but since dns resolution is shielded, they're waiting to finish

bdraco added a commit that referenced this pull request Mar 31, 2024
@bdraco
Fix flakey sonos test test_creating_entry_sets_up_media_player
62275c7 
extracted from #114529
@bdraco bdraco mentioned this pull request Mar 31, 2024
Merged
20 tasks
This was referenced Mar 31, 2024
Merged
Merged
@bdraco bdraco marked this pull request as ready for review March 31, 2024 21:23
@bdraco bdraco requested a review from jjlawren as a code owner March 31, 2024 21:23
@bdraco bdraco requested review from bieniu and a team as code owners March 31, 2024 21:23
@MartinHjelmare
Copy link
Member

MartinHjelmare commented Apr 3, 2024 •
edited
Loading

Since the upstream PR isn't merged yet, I suggest we draft this PR. If we want to add it before the upstream PR is merged, it's not a backport so at minimum things need to be renamed.

@MartinHjelmare MartinHjelmare marked this pull request as draft April 3, 2024 10:28
@bdraco
Copy link
Member Author

bdraco commented Apr 3, 2024

Will get the upstream PR merged soon. Got distracted with other issues and didn't come back to it. It should be ready to merge, but I'm double checking that nobody else has review feedback before I do that.

@bdraco bdraco marked this pull request as ready for review April 5, 2024 04:12
@bdraco
Copy link
Member Author

bdraco commented Apr 5, 2024

Got busy with release issues so I didn't get to finish double checking the aiohttp PR until today. Its merged now so this one should be good to go now

@balloob balloob merged commit f497c46 into dev Apr 6, 2024
38 checks passed
@balloob balloob deleted the aiodns_320_async_resolver branch April 6, 2024 03:15
@github-actions github-actions bot locked and limited conversation to collaborators Apr 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@MartinHjelmare MartinHjelmare MartinHjelmare left review comments

@balloob balloob balloob approved these changes

@jjlawren jjlawren Awaiting requested review from jjlawren

@bieniu bieniu Awaiting requested review from bieniu

Assignees

@gjohansson-ST gjohansson-ST

Labels
cla-signed code-quality core Quality Scale: No score
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@bdraco @MartinHjelmare @balloob @gjohansson-ST