Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

return_value and not_a_test_method health checks strict error fix #3581

Merged
merged 25 commits into from
Mar 15, 2023
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
4aa13d7
add note_deprecation
reaganjlee Feb 13, 2023
676aa1e
modify all()
reaganjlee Feb 13, 2023
f327a3c
add deprecated label
reaganjlee Feb 13, 2023
b3a171f
add deprecated label
reaganjlee Feb 13, 2023
c04f465
take out previous mentions of attributes
reaganjlee Feb 13, 2023
ec8613a
Merge branch 'master' of https://github.com/reaganjlee/hypothesis
reaganjlee Feb 13, 2023
29c55b4
change __iter__/__getattr__
reaganjlee Feb 15, 2023
53182a9
create release.rst and add name to authors
reaganjlee Feb 15, 2023
16f83c8
undo previous commits
reaganjlee Feb 15, 2023
840ae59
Merge branch 'strict-error-second'
reaganjlee Feb 15, 2023
bbd1ac3
Rephrase messages + add tests
Zac-HD Feb 16, 2023
53196b4
Fix tests
Zac-HD Feb 16, 2023
5dc6d40
add different tests
reaganjlee Mar 6, 2023
9463229
change all() and remove previous
reaganjlee Mar 6, 2023
eaa0695
remove extraneous comments + add __iter__
reaganjlee Mar 6, 2023
335d8c2
add deprecation when validate suppressions
reaganjlee Mar 7, 2023
8a9c35e
remove return_value test
reaganjlee Mar 7, 2023
71a1b77
add enum class for __iter__
reaganjlee Mar 11, 2023
ee09b9e
add enum metaclass for __iter__
reaganjlee Mar 11, 2023
00dd980
Merge branch 'master' of https://github.com/reaganjlee/hypothesis
reaganjlee Mar 11, 2023
431d331
remove cls arg
reaganjlee Mar 11, 2023
af8f317
Avoid HealthCheck.all()
Zac-HD Mar 12, 2023
f515c8f
Tweak formatting etc
Zac-HD Mar 12, 2023
e717840
Apply suggestions from code review
Zac-HD Mar 12, 2023
c29e219
add support for graalpy
timfel Jan 30, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions hypothesis-python/docs/changes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1735,7 +1735,7 @@ in the `Array API standard <https://data-apis.org/>`__ (see :pull:`3065`).
This release makes :doc:`stateful testing <stateful>` more likely to tell you
if you do something unexpected and unsupported:

- The :obj:`~hypothesis.HealthCheck.return_value` health check now applies to
- The ``hypothesis.HealthCheck.return_value`` health check now applies to
:func:`~hypothesis.stateful.rule` and :func:`~hypothesis.stateful.initialize`
rules, if they don't have ``target`` bundles, as well as
:func:`~hypothesis.stateful.invariant`.
Expand Down Expand Up @@ -1833,7 +1833,7 @@ Thanks to Ruben Opdebeeck for this contribution!
This release emits a more useful error message when :func:`@given() <hypothesis.given>`
is applied to a coroutine function, i.e. one defined using ``async def`` (:issue:`3054`).

This was previously only handled by the generic :obj:`~hypothesis.HealthCheck.return_value`
This was previously only handled by the generic ``hypothesis.HealthCheck.return_value``
health check, which doesn't direct you to use either :ref:`a custom executor <custom-function-execution>`
or a library such as :pypi:`pytest-trio` or :pypi:`pytest-asyncio` to handle it for you.

Expand Down Expand Up @@ -8981,7 +8981,7 @@ with the standard library :mod:`python:unittest` module:

- Applying :func:`@given <hypothesis.given>` to a non-test method which is
overridden from :class:`python:unittest.TestCase`, such as ``setUp``,
raises :attr:`a new health check <hypothesis.HealthCheck.not_a_test_method>`.
raises a new health check ``<hypothesis.HealthCheck.not_a_test_method>``.
(:issue:`991`)
- Using :meth:`~python:unittest.TestCase.subTest` within a test decorated
with :func:`@given <hypothesis.given>` would leak intermediate results
Expand Down
2 changes: 1 addition & 1 deletion hypothesis-python/docs/healthchecks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Using a value of ``HealthCheck.all()`` will disable all health checks.
.. autoclass:: hypothesis.HealthCheck
:undoc-members:
:inherited-members:
:exclude-members: all
:exclude-members: all, return_value, not_a_test_method


.. _deprecation-policy:
Expand Down
20 changes: 17 additions & 3 deletions hypothesis-python/src/hypothesis/_settings.py
Original file line number Diff line number Diff line change
Expand Up @@ -457,7 +457,17 @@ def __repr__(self):

@classmethod
def all(cls) -> List["HealthCheck"]:
return list(HealthCheck)
attributes = list(HealthCheck)
strict_errors = [HealthCheck.return_value, HealthCheck.not_a_test_method]

for strict_error in strict_errors:
attributes.remove(strict_error)

return attributes

# @classmethod
# def __iter__(cls) -> Iterator["HealthCheck"]:
# return iter(cls.all())
Zac-HD marked this conversation as resolved.
Show resolved Hide resolved

data_too_large = 1
"""Checks if too many examples are aborted for being too large.
Expand All @@ -479,14 +489,18 @@ def all(cls) -> List["HealthCheck"]:
testing."""

return_value = 5
"""Checks if your tests return a non-None value (which will be ignored and
"""Deprecated, now a strict error.

Checks if your tests return a non-None value (which will be ignored and
is unlikely to do what you want)."""
Zac-HD marked this conversation as resolved.
Show resolved Hide resolved

large_base_example = 7
"""Checks if the natural example to shrink towards is very large."""

not_a_test_method = 8
"""Checks if :func:`@given <hypothesis.given>` has been applied to a
"""Deprecated, now a strict error.

Checks if :func:`@given <hypothesis.given>` has been applied to a
method defined by :class:`python:unittest.TestCase` (i.e. not a test)."""

function_scoped_fixture = 9
Expand Down
6 changes: 4 additions & 2 deletions hypothesis-python/src/hypothesis/internal/healthcheck.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,14 +9,16 @@
# obtain one at https://mozilla.org/MPL/2.0/.

from hypothesis.errors import FailedHealthCheck

from hypothesis._settings import HealthCheck, note_deprecation

def fail_health_check(settings, message, label):
# Tell pytest to omit the body of this function from tracebacks
# https://docs.pytest.org/en/latest/example/simple.html#writing-well-integrated-assertion-helpers
__tracebackhide__ = True

if label in settings.suppress_health_check:
if label in [HealthCheck.return_value, HealthCheck.not_a_test_method]:
note_deprecation("The return_value and not_a_test_method health checks are deprecated, and are now strict errors.", since="2023-02-13", has_codemod=False)
elif label in settings.suppress_health_check: # Put if the label == (the two words I'm looking out for)
Zac-HD marked this conversation as resolved.
Show resolved Hide resolved
return
message += (
"\nSee https://hypothesis.readthedocs.io/en/latest/health"
Expand Down