-
-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix incorrect rejection of ws:// and wss:// urls #8482
Conversation
Ref #6722 (comment) |
This should probably not modify the constant but be in a separate one so the initial connect and redirect might be treated differently.. |
Yeah that is true as well. |
Does this look better? |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #8482 +/- ##
=======================================
Coverage 97.61% 97.61%
=======================================
Files 107 107
Lines 33159 33191 +32
Branches 3895 3898 +3
=======================================
+ Hits 32369 32401 +32
Misses 571 571
Partials 219 219
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Yeah, I think so. Though, we won't see a proof until the tests are in. |
Based on the comments we had on that previous PR, this looks like the right change. Can we get a regression test though, to be sure it doesn't get broken again in future? |
sure, I will need to crab an wss and ws url that does not need any json format query params though as I am not sure if the discord one can be used or not for the test here. |
Make sure that the test does not depend on the internet access. |
uh do I edit |
Whichever makes sense for the test you write (comparing to the tests in each file). I think the first tests small details using mocks, while the second is closer to E2E tests against a running application. |
This is because libraries like discord.py needs to use a wss type of url to make a socket to an api service, and it cannot be done outside of aiohttp as it can block the async event loop which can result in undefined behavior. Test will arrive soon.
for more information, see https://pre-commit.ci
Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua>
tests/conftest.py
Outdated
@@ -1,5 +1,7 @@ | |||
# type: ignore | |||
import asyncio | |||
import base64 | |||
import hashlib |
Check notice
Code scanning / CodeQL
Module is imported with 'import' and 'import from' Note test
Co-authored-by: Sam Bull <aa6bs0@sambull.org>
Backport to 3.10: 💔 cherry-picking failed — conflicts found❌ Failed to cleanly apply 62173be on top of patchback/backports/3.10/62173bea1e4e3538d46e6e9b94d727d870d86879/pr-8482 Backporting merged PR #8482 into master
🤖 @patchback |
Thanks @AraHaan |
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) <wk.cvs.github@sydorenko.org.ua> Co-authored-by: J. Nick Koston <nick@koston.org> Co-authored-by: Sam Bull <aa6bs0@sambull.org> (cherry picked from commit 62173be)
…d wss:// urls (#8511) Co-authored-by: pre-commit-ci[bot] Co-authored-by: Sviatoslav Sydorenko (Святослав Сидоренко) Co-authored-by: J. Nick Koston <nick@koston.org> Co-authored-by: Sam Bull <aa6bs0@sambull.org> Co-authored-by: AraHaan <seandhunt_7@yahoo.com>
@@ -0,0 +1,2 @@ | |||
Fixed the incorrect rejection of ``ws://`` and ``wss://`` urls |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It'd be useful to improve this text with context as requested earlier in #8482 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry I missed the there was more to that request.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I only got to take a look again and realized that this description is rather vague. It refers to an incorrect behavior without really getting into what's incorrect about it and where/how it's happening, what's the visible effect for the end-users.
I know this is hard to write and hard to explain, so I'm thinking of ways to lint it better. I was hoping to try integrating https://vale.sh and see if that would help people / give better hints on how to address such things.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought the "incorrect" behaviour referenced here is not in any release of aiohttp, so probably a changelog entry is unneeded anyway.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@Dreamsorcerer yep! I actually brought it up in the follow-up and we dropped this note in favor of linking the original one and crediting the contribution to more people, mentioning more PRs/issues.
# if the connection wasn't already closed | ||
for c in connections: | ||
c.close() | ||
del c |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI, __del__()
is more or less predictable in CPython, but in PyPy it's not really guaranteed to be called at a specific moment in time. Calling gc.collect()
might improve it, especially in tests, but might need several invocations.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you think it better to switch it back to __del__
? codeql complained about it
Code scanning
/ CodeQL
__del__
is called explicitly
The del special method is called explicitly.
Show more details
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, linters would normally complain and you'd have to ignore. However, I'd probably just stick a few calls to gc.collect()
, maybe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@@ -0,0 +1,2 @@ | |||
Fixed the incorrect rejection of ``ws://`` and ``wss://`` urls | |||
-- by :user:` AraHaan`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is because libraries like discord.py needs to use a wss type of url to make a socket to an api service, and it cannot be done outside of aiohttp as it can block the async event loop which can result in undefined behavior.
Test will arrive soon.
What do these changes do?
This fixes the breaking change that breaks the usage of wss style urls inside of api wrappers that requires them in order to start up at all.
Are there changes in behavior for the user?
The only change is that the wss style urls that previously did not throw in 3.9.5 will now still work in 3.10.0 and 4.0.0 when this gets merged. This is because this change special cases wss links.
Is it a substantial burden for the maintainers to support this?
I think this is not a substantial burden as this is both a simple change, and prevents breaking applications that expect this entirely while also allowing them to get the latest updates on both features and bug fixes (and possibly also security fixes) vs pinning them forever to 3.9.5 being the last version they can ever use and them eating a security bug that they can never get fixed.
Related issue number
Fixes #8481
Checklist
Will finish the checklist as I work on this.
CONTRIBUTORS.txt
CHANGES/
foldername it
<issue_or_pr_num>.<type>.rst
(e.g.588.bugfix.rst
)if you don't have an issue number, change it to the pull request
number after creating the PR
.bugfix
: A bug fix for something the maintainers deemed animproper undesired behavior that got corrected to match
pre-agreed expectations.
.feature
: A new behavior, public APIs. That sort of stuff..deprecation
: A declaration of future API removals and breakingchanges in behavior.
.breaking
: When something public is removed in a breaking way.Could be deprecated in an earlier release.
.doc
: Notable updates to the documentation structure or buildprocess.
.packaging
: Notes for downstreams about unobvious side effectsand tooling. Changes in the test invocation considerations and
runtime assumptions.
.contrib
: Stuff that affects the contributor experience. e.g.Running tests, building the docs, setting up the development
environment.
.misc
: Changes that are hard to assign to any of the abovecategories.
Make sure to use full sentences with correct case and punctuation,
for example:
Use the past tense or the present tense a non-imperative mood,
referring to what's changed compared to the last released version
of this project.