-
-
Notifications
You must be signed in to change notification settings - Fork 674
Use pytest for more TestSuite tests #40971
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from all commits
2aa9e4e
277fa29
66b496f
09bca86
7211af2
d5a6af0
75ec8a6
82bba0d
b9988a7
0f42046
546c87f
24162b3
73f297f
a424b8e
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
import pytest | ||
|
||
|
||
def test_positive_integer_semigroup(): | ||
r""" | ||
Run the ``TestSuite()`` for ``PositiveIntegerSemigroup`` | ||
(this can take quite a long time). | ||
""" | ||
from sage.misc.sage_unittest import TestSuite | ||
from sage.combinat.backtrack import PositiveIntegerSemigroup | ||
PP = PositiveIntegerSemigroup() | ||
|
||
# fewer max_runs since these are kind of slow | ||
TestSuite(PP).run(verbose=True, | ||
raise_on_failure=True, | ||
max_runs=256) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
import pytest | ||
from sage.rings.padics.factory import ZpLC, ZpLF, QpLC, QpLF | ||
|
||
# ZpLC, ZpLF, QpLC, and QpLF all raise FutureWarnings | ||
from warnings import catch_warnings, filterwarnings | ||
filterwarnings("ignore", category=FutureWarning) | ||
|
||
|
||
@pytest.fixture | ||
def R1(): | ||
return ZpLC(2) | ||
|
||
|
||
@pytest.fixture | ||
def R2(): | ||
return ZpLF(2) | ||
|
||
|
||
@pytest.fixture | ||
def R3(): | ||
return QpLC(2) | ||
|
||
|
||
@pytest.fixture | ||
def R4(): | ||
return QpLF(2) | ||
|
||
|
||
# Use strings for the fixture names here, and then later convert them | ||
# to the actual fixture objects using request.getfixturevalue(). This | ||
# is a workaround for being unable to pass fixtures directly as | ||
# parameters: | ||
# | ||
# https://github.com/pytest-dev/pytest/issues/349 | ||
# | ||
elements = ( "R1", "R2", "R3", "R4" ) | ||
|
||
|
||
@pytest.mark.parametrize("e", elements) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. the amount of boilerplate is scary. Why can't you parametrize e over [ZpLC(2), ZpLF(2), etc.] directly? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There are a couple of related problems. (I'm not happy about the boilerplate either, but it should decrease in relative volume as more tests are moved to pytest.) The big one is that we want to cache the fixtures and create them on-demand, so they aren't recreated multiple times, or created when they won't be used (if only a subset of tests is being run). For that magic to work, we need to mark the fixtures with
If we forego the caching / on-demand creation, we can create the objects ourselves, but then we lose the nice fixture naming. if I pass in
instead of
which uses the fixture name. Now obviously There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. "so that they aren't recreated multiple times"? Aren't they only used in this one test anyway? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The idea of fixtures is that they can be easily reused in other tests (either in the same file or other test files), and pytest then makes sure that only one instance is created for all those tests. This obviously only scales once more and more tests are migrated to pytest. (eg here if all the See eg symplectic forms tests for a more complicated example that would be hard to realize without fixtures. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Right, there's only one test using it because there's only one test. But there are other doctests that construct (say) If it was trivial to parameterize a test with fixtures, I probably wouldn't have bothered, since n >= 2 is usually where you'd stop and refactor. But it took me a few hours to figure out the trick, so I figure the polite thing to do is to set everything up nicely for the next person who wants to add a test using one of these lattice elements. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. doctest doesn't need to reset the namespace before every test either. But usually reusing variables from another doctest would be confusing. If you do say
then you're no worse off than the situation with pytest (and it's shorter…)
aren't the different files tested in parallel? |
||
def test_padic_lattice_element(e, request): | ||
r""" | ||
Run the ``TestSuite()`` for some examples that previously | ||
lived in the TESTS:: block of the padic_lattice_element module. | ||
""" | ||
from sage.misc.sage_unittest import TestSuite | ||
|
||
# Convert the string to a real fixture | ||
e = request.getfixturevalue(e) | ||
|
||
# Only do a few runs, _test_matrix_smith() in particular is slow. | ||
TestSuite(e).run(verbose=True, | ||
raise_on_failure=True, | ||
skip="_test_teichmuller", | ||
max_runs=8) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
import pytest | ||
|
||
|
||
@pytest.fixture | ||
def w(): | ||
from sage.rings.function_field.constructor import FunctionField | ||
from sage.rings.polynomial.polynomial_ring_constructor import PolynomialRing | ||
from sage.rings.integer import Integer | ||
from sage.rings.rational_field import QQ | ||
|
||
K = FunctionField(QQ, 'x') | ||
x = K.gen() | ||
R = PolynomialRing(K, 'y') | ||
y = R.gen() | ||
L = K.extension(y**2 - x) | ||
v = K.valuation(Integer(1)) | ||
|
||
return v.extensions(L) | ||
|
||
|
||
@pytest.mark.parametrize("idx", [0,1]) | ||
def test_finite_extension_from_limit_valuation_w(w, idx): | ||
r""" | ||
Run the ``TestSuite()`` for two examples given in the | ||
``FiniteExtensionFromLimitValuation`` documentation. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Speaking of more downsides of pytest migration, a cost of having the test far from the code is that now if the example given in the FiniteExtensionFromLimitValuation documentation is changed, this docstring will silently become false. You can change the docstring to "run the test suite for two examples that were given in the FiniteExtensionFromLimitValuation documentation", but then what's the value of the docstring? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is just laziness on my part and not inherent to pytest. I'd love to describe the objects being tested more intelligently, or explain why they're being tested... but the original doctests didn't come with an explanation, and I don't know anything about mapped valuations, so this was all I could think of. |
||
""" | ||
from sage.misc.sage_unittest import TestSuite | ||
|
||
# fewer max_runs, these are kind of slow | ||
TestSuite(w[idx]).run(verbose=True, | ||
raise_on_failure=True, | ||
max_runs=512) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the verbose = True option needed for the functionality? If not, I think, I would prefer the quiet version, or maybe couple it to pytests verbosity level via
pytestconfig.get_verbosity() > 0 (or 1?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pytest already swallows stdout unless you pass
-s
, so i was thinking the added verbosity would only matter if the tests fail. I guess we could go back and make them all depend on the pytest verbosity though. I don't feel strongly about it either way.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I noticed it mostly in CI, where it bloats the log without adding too much value (at least when everything passes). For the moment, that's okay but might get more annoying once more tests are migrated.
So maybe I do this later if I get annoyed enough - or I'll actually extract all of these testsuite _test methods (in the actual classes) to pytest.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could probably get away without
-s
in the CI. I think pytest will still dump the output that it captured in the event of a failure, while leaving everything nice and quiet if it passes.