skipif + parametrize not skipping tests

[arxanas]

I have this testing code:

import pytest


def params():
    dont_skip = pytest.mark.skipif(False, reason="don't skip")
    return [dont_skip("foo"), dont_skip("bar")]


@pytest.mark.skipif(True, reason="always skip")
@pytest.mark.parametrize("param", params())
@pytest.mark.skipif(True, reason="really always skip please")
def test_foo(param):
    assert False

The expected behavior is that if any of the skipif conditions returns True, the test is skipped. The actual behavior is that the skipif condition seems to depend entirely on the value of the dont_skip parameter. For example, the following test case causes the tests to be skipped (the Trues and Falses are swapped around):

import pytest


def params():
    dont_skip = pytest.mark.skipif(True, reason="don't skip")
    return [dont_skip("foo"), dont_skip("bar")]


@pytest.mark.skipif(False, reason="always skip")
@pytest.mark.parametrize("param", params())
@pytest.mark.skipif(False, reason="really always skip please")
def test_foo(param):
    assert False

Here's the full output of the first code sample:

$ py.test test_foo.py
============================= test session starts ==============================
platform darwin -- Python 2.7.10, pytest-2.8.6.dev1, py-1.4.31, pluggy-0.3.1
rootdir: /Volumes/Home/Users/Waleed/tmp/foo, inifile: 
collected 2 items

test_foo.py FF

=================================== FAILURES ===================================
________________________________ test_foo[foo] _________________________________

param = 'foo'

    @pytest.mark.skipif(True, reason="always skip")
    @pytest.mark.parametrize("param", params())
    @pytest.mark.skipif(True, reason="really always skip please")
    def test_foo(param):
>       assert False
E       assert False

test_foo.py:13: AssertionError
________________________________ test_foo[bar] _________________________________

param = 'bar'

    @pytest.mark.skipif(True, reason="always skip")
    @pytest.mark.parametrize("param", params())
    @pytest.mark.skipif(True, reason="really always skip please")
    def test_foo(param):
>       assert False
E       assert False

test_foo.py:13: AssertionError
=========================== 2 failed in 0.01 seconds ===========================

The issue manifests under:

• Python 3.5 + Pytest 2.8.5
• Python 2.7.10 + Pytest 2.8.5
• Python 2.7.10 + Pytest HEAD (0ef73ed).

I'm running OS X 10.9.5.

[nicoddemus]

Thanks for the report! 😄

[kdexd]

I have recently experienced this issue. It still persists in pytest 3.0.0.

[alessss]

Is there any solution for the issue? Still experiencing it in pytest==3.0.5

[RonnyPfannschmidt]

@alessss markers have not been fixed, this issue is not easy to resolve due to legacy

[alessss]

If i use plain True in skipif decorator it works well:
@pytest.mark.skipif(True, reason="always skip")

But if there is some function, returning True or False - tests don't start.
from distutils.version import StrictVersion
from versions import get_version
@pytest.mark.skipif(StrictVersion(get_version()) < StrictVersion('1.8.0'), reason='WRONG VERSION')

Is there any workaround for this?

[The-Compiler]

@alessss I don't see how that'd be possible - and if you don't also have skipif markers on parameters, this issue is totally unrelated.

[nicoddemus]

For reference, #1921 discusses how to start fixing markers in general.

[scopatz]

This is a non-obvious pain point I am experiencing on v3.0.6

[RonnyPfannschmidt]

@hpk42 i beleive this one breaks apart due to yet another marker transfer detail included in parametrization

[RonnyPfannschmidt]

we can fix this by landing #2522 as well

[RonnyPfannschmidt]

fixed in #3317

[RonnyPfannschmidt]

fixed in #3317