marker expressions use the legacy mark storage #3441

Botje · 2018-05-03T13:37:43Z

I am currently chasing an issue which is very similar to the (apparently solved) #568 and the not yet solved #2947. I was happy to discover pull request #3317 and switched to the features branch to see if my issue had been fixed.
Unfortunately, it is not..

Given test_bug.py:

import pytest

class BaseTests(object):
    def test_something(self):
        pass

@pytest.mark.FOO
class TestFooClass(BaseTests):
    pass

@pytest.mark.BAR
class TestBarClass(BaseTests):
    pass

I expect pytest --collect_only -m FOO test_bug.py to return only TestFooClass::()::test_something.
However, I get back methods from both subclasses:

$ pytest --collect-only -m "FOO" test_bug.py
================================================================================= test session starts ==================================================================================
platform linux -- Python 3.6.3, pytest-3.5.2.dev80+g7834b450, py-1.5.3, pluggy-0.6.0
rootdir: /home/barco/pytest-bug, inifile:
collected 2 items
<Module 'test_bug.py'>
  <Class 'TestFooClass'>
    <Instance '()'>
      <Function 'test_something'>
  <Class 'TestBarClass'>
    <Instance '()'>
      <Function 'test_something'>

If I reverse the order of TestFooClass and TestBarClass, I get two hits for BAR instead.

My environment is Ubuntu Linux 17.10 with python 3.6.3, in a fresh virtualenv with only pytest@features installed.
Output of pip list:

Package        Version
-------------- ---------------------
atomicwrites   1.1.5
attrs          17.4.0
more-itertools 4.1.0
pip            10.0.1
pkg-resources  0.0.0
pluggy         0.6.0
py             1.5.3
pytest         3.5.2.dev80+g7834b450
setuptools     39.1.0
six            1.11.0
wheel          0.31.0

Thanks for your time and please let me know if I can provide further information.

The text was updated successfully, but these errors were encountered:

pytestbot · 2018-05-03T13:38:42Z

GitMate.io thinks possibly related issues are #725 (Markers are transferred from subclasses to base class methods), #2947 (Marking subclass affected methods of parent class.), #568 (markers stains on all related classes), #2515 (Custom marker kwargs for test case method is overwritten from class marker.), and #3359 (Allow methods to access a class fixture. ).

RonnyPfannschmidt · 2018-05-03T13:57:58Z

@Botje thanks for the note, i missed this detail when sorting out marker issues

the basic issue is that marker expressions still use the legacy marker storage, which suffers the marker smearing issues

RonnyPfannschmidt · 2018-05-03T14:03:24Z

pr upcoming

RonnyPfannschmidt · 2018-05-03T14:47:41Z

@Botje please take a look if #3442 works for you as expected

Botje · 2018-05-03T14:49:48Z

Yes, it works as expected --- both on my artificial and real test cases.
Thank you very much for the quick response!

…kstorage Fix #3441 let mark expressions be populated from the modern marker storage

nicoddemus · 2018-05-03T22:27:43Z

Fixed by #3442

This PR updates [pytest](https://pypi.org/project/pytest) from **3.5.1** to **3.6.3**. <details> <summary>Changelog</summary> ### 3.6.2 ``` ========================= Bug Fixes --------- - Fix regression in ``Node.add_marker`` by extracting the mark object of a ``MarkDecorator``. (`3555 <https://github.com/pytest-dev/pytest/issues/3555>`_) - Warnings without ``location`` were reported as ``None``. This is corrected to now report ``<undetermined location>``. (`3563 <https://github.com/pytest-dev/pytest/issues/3563>`_) - Continue to call finalizers in the stack when a finalizer in a former scope raises an exception. (`3569 <https://github.com/pytest-dev/pytest/issues/3569>`_) - Fix encoding error with `print` statements in doctests (`3583 <https://github.com/pytest-dev/pytest/issues/3583>`_) Improved Documentation ---------------------- - Add documentation for the ``--strict`` flag. (`3549 <https://github.com/pytest-dev/pytest/issues/3549>`_) Trivial/Internal Changes ------------------------ - Update old quotation style to parens in fixture.rst documentation. (`3525 <https://github.com/pytest-dev/pytest/issues/3525>`_) - Improve display of hint about ``--fulltrace`` with ``KeyboardInterrupt``. (`3545 <https://github.com/pytest-dev/pytest/issues/3545>`_) - pytest's testsuite is no longer runnable through ``python setup.py test`` -- instead invoke ``pytest`` or ``tox`` directly. (`3552 <https://github.com/pytest-dev/pytest/issues/3552>`_) - Fix typo in documentation (`3567 <https://github.com/pytest-dev/pytest/issues/3567>`_) ``` ### 3.6.1 ``` ========================= Bug Fixes --------- - Fixed a bug where stdout and stderr were logged twice by junitxml when a test was marked xfail. (`3491 <https://github.com/pytest-dev/pytest/issues/3491>`_) - Fix ``usefixtures`` mark applyed to unittest tests by correctly instantiating ``FixtureInfo``. (`3498 <https://github.com/pytest-dev/pytest/issues/3498>`_) - Fix assertion rewriter compatibility with libraries that monkey patch ``file`` objects. (`3503 <https://github.com/pytest-dev/pytest/issues/3503>`_) Improved Documentation ---------------------- - Added a section on how to use fixtures as factories to the fixture documentation. (`3461 <https://github.com/pytest-dev/pytest/issues/3461>`_) Trivial/Internal Changes ------------------------ - Enable caching for pip/pre-commit in order to reduce build time on travis/appveyor. (`3502 <https://github.com/pytest-dev/pytest/issues/3502>`_) - Switch pytest to the src/ layout as we already suggested it for good practice - now we implement it as well. (`3513 <https://github.com/pytest-dev/pytest/issues/3513>`_) - Fix if in tests to support 3.7.0b5, where a docstring handling in AST got reverted. (`3530 <https://github.com/pytest-dev/pytest/issues/3530>`_) - Remove some python2.5 compatibility code. (`3529 <https://github.com/pytest-dev/pytest/issues/3529>`_) ``` ### 3.6.0 ``` ========================= Features -------- - Revamp the internals of the ``pytest.mark`` implementation with correct per node handling which fixes a number of long standing bugs caused by the old design. This introduces new ``Node.iter_markers(name)`` and ``Node.get_closest_mark(name)`` APIs. Users are **strongly encouraged** to read the `reasons for the revamp in the docs <https://docs.pytest.org/en/latest/mark.htmlmarker-revamp-and-iteration>`_, or jump over to details about `updating existing code to use the new APIs <https://docs.pytest.org/en/latest/mark.htmlupdating-code>`_. (`3317 <https://github.com/pytest-dev/pytest/issues/3317>`_) - Now when ``pytest.fixture`` is applied more than once to the same function a ``ValueError`` is raised. This buggy behavior would cause surprising problems and if was working for a test suite it was mostly by accident. (`2334 <https://github.com/pytest-dev/pytest/issues/2334>`_) - Support for Python 3.7's builtin ``breakpoint()`` method, see `Using the builtin breakpoint function <https://docs.pytest.org/en/latest/usage.htmlbreakpoint-builtin>`_ for details. (`3180 <https://github.com/pytest-dev/pytest/issues/3180>`_) - ``monkeypatch`` now supports a ``context()`` function which acts as a context manager which undoes all patching done within the ``with`` block. (`3290 <https://github.com/pytest-dev/pytest/issues/3290>`_) - The ``--pdb`` option now causes KeyboardInterrupt to enter the debugger, instead of stopping the test session. On python 2.7, hitting CTRL+C again exits the debugger. On python 3.2 and higher, use CTRL+D. (`3299 <https://github.com/pytest-dev/pytest/issues/3299>`_) - pytest not longer changes the log level of the root logger when the ``log-level`` parameter has greater numeric value than that of the level of the root logger, which makes it play better with custom logging configuration in user code. (`3307 <https://github.com/pytest-dev/pytest/issues/3307>`_) Bug Fixes --------- - A rare race-condition which might result in corrupted ``.pyc`` files on Windows has been hopefully solved. (`3008 <https://github.com/pytest-dev/pytest/issues/3008>`_) - Also use iter_marker for discovering the marks applying for marker expressions from the cli to avoid the bad data from the legacy mark storage. (`3441 <https://github.com/pytest-dev/pytest/issues/3441>`_) - When showing diffs of failed assertions where the contents contain only whitespace, escape them using ``repr()`` first to make it easy to spot the differences. (`3443 <https://github.com/pytest-dev/pytest/issues/3443>`_) Improved Documentation ---------------------- - Change documentation copyright year to a range which auto-updates itself each time it is published. (`3303 <https://github.com/pytest-dev/pytest/issues/3303>`_) Trivial/Internal Changes ------------------------ - ``pytest`` now depends on the `python-atomicwrites <https://github.com/untitaker/python-atomicwrites>`_ library. (`3008 <https://github.com/pytest-dev/pytest/issues/3008>`_) - Update all pypi.python.org URLs to pypi.org. (`3431 <https://github.com/pytest-dev/pytest/issues/3431>`_) - Detect `pytest_` prefixed hooks using the internal plugin manager since ``pluggy`` is deprecating the ``implprefix`` argument to ``PluginManager``. (`3487 <https://github.com/pytest-dev/pytest/issues/3487>`_) - Import ``Mapping`` and ``Sequence`` from ``_pytest.compat`` instead of directly from ``collections`` in ``python_api.py::approx``. Add ``Mapping`` to ``_pytest.compat``, import it from ``collections`` on python 2, but from ``collections.abc`` on Python 3 to avoid a ``DeprecationWarning`` on Python 3.7 or newer. (`3497 <https://github.com/pytest-dev/pytest/issues/3497>`_) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pytest - Changelog: https://pyup.io/changelogs/pytest/ - Repo: https://github.com/pytest-dev/pytest/issues - Homepage: http://pytest.org </details>

pytestbot added the type: bug problem that needs to be addressed label May 3, 2018

nicoddemus added the topic: marks related to marks, either the general marks or builtin label May 3, 2018

RonnyPfannschmidt changed the title ~~Markers on subclasses still affect methods in sibling classes with respect to -m flag~~ marker expressions use the legacy mark storage May 3, 2018

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue May 3, 2018

fix pytest-dev#3441 - use iter-marks to populate markexpr namespace

63ef46d

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue May 3, 2018

add tests for pytest-dev#3441

88f84f5

RonnyPfannschmidt added a commit to RonnyPfannschmidt/pytest that referenced this issue May 3, 2018

add tests for pytest-dev#3441

e6a86e0

nicoddemus added a commit that referenced this issue May 3, 2018

Merge pull request #3442 from RonnyPfannschmidt/fix-3441-markexpr-mar…

35f53a7

…kstorage Fix #3441 let mark expressions be populated from the modern marker storage

nicoddemus closed this as completed May 3, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

marker expressions use the legacy mark storage #3441

marker expressions use the legacy mark storage #3441

Botje commented May 3, 2018

pytestbot commented May 3, 2018

RonnyPfannschmidt commented May 3, 2018

RonnyPfannschmidt commented May 3, 2018

RonnyPfannschmidt commented May 3, 2018

Botje commented May 3, 2018

nicoddemus commented May 3, 2018

marker expressions use the legacy mark storage #3441

marker expressions use the legacy mark storage #3441

Comments

Botje commented May 3, 2018

pytestbot commented May 3, 2018

RonnyPfannschmidt commented May 3, 2018

RonnyPfannschmidt commented May 3, 2018

RonnyPfannschmidt commented May 3, 2018

Botje commented May 3, 2018

nicoddemus commented May 3, 2018