Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

configure warnings as errors in the test suite #128770

Open
graingert opened this issue Jan 13, 2025 · 3 comments
Open

configure warnings as errors in the test suite #128770

graingert opened this issue Jan 13, 2025 · 3 comments
Labels
tests Tests in the Lib/test dir type-feature A feature request or enhancement

Comments

@graingert
Copy link
Contributor

graingert commented Jan 13, 2025 •
edited by bedevere-app bot
Loading

Feature or enhancement

Proposal:

It's very easy (especially when deprecating something) to introduce a lot of noisy test output with new warnings, eg: #127949 (comment)

I think it would be useful to either configure warnings as errors in the test suite (and then capture unraisable/threadexcept errors using a mechanism like pytest has) or tee any warnings produced, and fail CI if any have been produced that were not captured with catch_warnings or assertWarns etc like https://pypi.org/project/pytest-max-warnings/

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://discuss.python.org/t/its-too-easy-to-introduce-noise-in-cpythons-test-suite-output-resourcewarnings-and-deprecationwarnings/75234

Linked PRs
@graingert graingert added the type-feature A feature request or enhancement label Jan 13, 2025
@bedevere-app bedevere-app bot mentioned this issue Jan 13, 2025
Merged
@erlend-aasland erlend-aasland added the tests Tests in the Lib/test dir label Jan 13, 2025
@sobolevn
Copy link
Member

See #110012 test_socket raises warnings right now.

@graingert
Copy link
Contributor Author

See #128816 test_doctest also has a warning

@graingert
Copy link
Contributor Author

See #110012 test_socket raises warnings right now.

I'm thinking we mark these as default with catch_warnings in the test for now?

@bedevere-app bedevere-app bot mentioned this issue Jan 16, 2025
Merged
hugovk pushed a commit that referenced this issue Jan 17, 2025
@graingert @sobolevn @brettcannon
gh-128770: raise warnings as errors in test suite - except for test_s…
7807b40 
…ocket which still logs warnings (#128726)

Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Brett Cannon <brett@python.org>
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jan 17, 2025
@graingert @sobolevn
@brettcannon @miss-islington
pythongh-128770: raise warnings as errors in test suite - except for …
128244d 
…test_socket which still logs warnings (pythonGH-128726)

(cherry picked from commit 7807b40)

Co-authored-by: Thomas Grainger <tagrain@gmail.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Brett Cannon <brett@python.org>
@bedevere-app bedevere-app bot mentioned this issue Jan 17, 2025
Draft
hugovk added a commit to hugovk/cpython that referenced this issue Jan 17, 2025
@hugovk
Revert "pythongh-128770: raise warnings as errors in test suite - exc…
0342073 
…ept for test_socket which still logs warnings (python#128726)"

This reverts commit 7807b40.
@bedevere-app bedevere-app bot mentioned this issue Jan 17, 2025
Merged
hugovk added a commit that referenced this issue Jan 17, 2025
@hugovk
Revert "gh-128770: raise warnings as errors in test suite - except fo…
76856ae 
…r test_socket which still logs warnings (#128726)" (#128936)
graingert added a commit to graingert/cpython that referenced this issue Jan 18, 2025
@graingert
Reapply "pythongh-128770: raise warnings as errors in test suite - ex…
a6bb15e 
…cept for test_socket which still logs warnings (python#128726)" (python#128936)

This reverts commit 76856ae.
@bedevere-app bedevere-app bot mentioned this issue Jan 18, 2025
Open
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Jan 21, 2025
@graingert @sobolevn
@brettcannon @srinivasreddy
pythongh-128770: raise warnings as errors in test suite - except for …
f12e00e 
…test_socket which still logs warnings (python#128726)

Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Brett Cannon <brett@python.org>
srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Jan 21, 2025
@hugovk @srinivasreddy
Revert "pythongh-128770: raise warnings as errors in test suite - exc…
076d0eb 
…ept for test_socket which still logs warnings (python#128726)" (python#128936)
pablogsal pushed a commit that referenced this issue Jan 22, 2025
@graingert
gh-128770: fix ResourceWarning in test_pyrepl (#128906)
24c84d8
miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jan 22, 2025
@graingert @miss-islington
pythongh-128770: fix ResourceWarning in test_pyrepl (pythonGH-128906)
79bba48 
(cherry picked from commit 24c84d8)

Co-authored-by: Thomas Grainger <tagrain@gmail.com>
@bedevere-app bedevere-app bot mentioned this issue Jan 22, 2025
Merged
pablogsal pushed a commit that referenced this issue Jan 22, 2025
@miss-islington
[3.13] gh-128770: fix ResourceWarning in test_pyrepl (GH-128906) (#12…
67971cd 
…9179)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
tests Tests in the Lib/test dir type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@graingert @sobolevn @erlend-aasland