Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed tests that were failing due Pytest updates #190

Merged
merged 10 commits into from
Jan 11, 2019
Merged

Fixed tests that were failing due Pytest updates #190

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
d75a67a
Corrected Rerun failures - from https://github.com/pytest-dev/pytest-…
RibeiroAna Jan 3, 2019
fa09e9b
Removed ENV
RibeiroAna Jan 3, 2019
c320f42
Changed funcarg to pytest fixture
RibeiroAna Jan 4, 2019
0f2de35
Removed some comments
RibeiroAna Jan 4, 2019
3f28510
Style matter
RibeiroAna Jan 4, 2019
c30a995
Style
RibeiroAna Jan 4, 2019
7d9ded4
Reversed rerunfailures to 4.1
RibeiroAna Jan 5, 2019
319699d
Style
RibeiroAna Jan 5, 2019
88cd856
added comment about issue 77
RibeiroAna Jan 7, 2019
1e5eec4
Removed rerunfailures version due launch of rerunfailures 6.0
RibeiroAna Jan 8, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ dist
# IDE Specific files
### Pycharm IDE - Jetbrains
.idea/*
.vscode/*
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm using Visual Studio right now, so I added it.


### PyDev IDE - Eclipse
.metadata
Expand Down
7 changes: 3 additions & 4 deletions pytest_html/plugin.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,7 +100,7 @@ def __init__(self, logfile, config):

class TestResult:

def __init__(self, outcome, report, logfile, config):
def __init__(self, outcome, report, logfile, config, test_index=0):
self.test_id = report.nodeid
if getattr(report, 'when', 'call') != 'call':
self.test_id = '::'.join([report.nodeid, report.when])
Expand All @@ -113,8 +113,6 @@ def __init__(self, outcome, report, logfile, config):
self.config = config
self.row_table = self.row_extra = None

test_index = hasattr(report, 'rerun') and report.rerun + 1 or 0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realised from here that report doesn't have the attribute rerun anymore, so that was causing some rerun tests to fail. Instead of getting the test_index from the attribute, I'm passing it as a parameter from the property rerun of the TestResult class.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could be a regression from pytest-rerunfailures. It would seem that the latest version doesn't report the number of reruns in the report? If so, please raise an issue against the plugin. In the meantime we could pin our tests to the version of pytest-rerunfailures before this change.


for extra_index, extra in enumerate(getattr(report, 'extra', [])):
self.append_extra_html(extra, extra_index, test_index)

Expand Down Expand Up @@ -263,7 +261,8 @@ def append_log_html(self, report, additional_html):
additional_html.append(log)

def _appendrow(self, outcome, report):
result = self.TestResult(outcome, report, self.logfile, self.config)
result = self.TestResult(outcome, report, self.logfile, self.config,
self.rerun)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.rerun will only be present if the pytest-rerunfailures plugin is installed, if it's not installed I would expect this to raise an exception due to the attribute not being available

if result.row_table is not None:
index = bisect.bisect_right(self.results, result)
self.results.insert(index, result)
Expand Down
6 changes: 4 additions & 2 deletions testing/test_pytest_html.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,9 @@ def test_xpass(): pass

def test_setup_error(self, testdir):
testdir.makepyfile("""
def pytest_funcarg__arg(request):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Funcarg is deprecated since Pytest 4.

import pytest
@pytest.fixture
def arg(request):
raise ValueError()
def test_function(arg):
pass
Expand Down Expand Up @@ -442,7 +444,7 @@ def pytest_runtest_makereport(item, call):
""".format(extra_type, content))
testdir.makepyfile("""
import pytest
@pytest.mark.flaky(reruns=2)
@pytest.mark.flaky(reruns=4)
Copy link
Member Author

@RibeiroAna RibeiroAna Jan 4, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just because the loop bellow goes from 1 to 4 (so it checks for extra content 4 times) and there was only 2 reruns.

Copy link
Collaborator

@davehunt davehunt Jan 4, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see your logic, but I wonder why we had this mismatch with tests passing? This code hasn't changed in three years.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw the logic there before, actually the loop checks for content 3 times (range(1,4) = [1, 2, 3], my fault) and the first test (the one which failed) count as the file1, the first rerun is actually file2, and so on

def test_fail():
assert False""")
result, html = run(testdir)
Expand Down