TST: Use __tracebackhide__ for pd.util.testing helpers?

[jbrockmendel]

I haven't looked at the feature too carefully, but it looks like it may make test failure messages more targeted.

[jbrockmendel]

Based on half an hour of testing locally, I'm +1 on this.

[gfyoung]

@jbrockmendel : Could you illustrate with an example? I'm not sure I follow you here.

[jbrockmendel]

Could you illustrate with an example? I'm not sure I follow you here.

For illustration, I added the following test in tests/test_errors.py:

def test_hide():
    left = pd.DataFrame([1, 2])
    right = pd.DataFrame([3, 4])
    tm.assert_equal(left, right)

Under the status quo:

$ python -m pytest pandas/tests/test_errors.py -k test_hide

______________________________________________________________________________ test_hide ______________________________________________________________________________

    def test_hide():
        left = pd.DataFrame([1, 2])
        right = pd.DataFrame([3, 4])
>       tm.assert_equal(left, right)

pandas/tests/test_errors.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pandas/util/testing.py:1530: in assert_equal
    assert_frame_equal(left, right, **kwargs)
pandas/util/testing.py:1449: in assert_frame_equal
    obj='DataFrame.iloc[:, {idx}]'.format(idx=i))
pandas/util/testing.py:1285: in assert_series_equal
    obj='{obj}'.format(obj=obj))
pandas/_libs/testing.pyx:59: in pandas._libs.testing.assert_almost_equal
    cpdef assert_almost_equal(a, b,
pandas/_libs/testing.pyx:173: in pandas._libs.testing.assert_almost_equal
    raise_assert_detail(obj, msg, lobj, robj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

obj = 'DataFrame.iloc[:, 0]', message = 'DataFrame.iloc[:, 0] values are different (100.0 %)', left = '[1, 2]', right = '[3, 4]', diff = None

    def raise_assert_detail(obj, message, left, right, diff=None):
    
        if isinstance(left, np.ndarray):
            left = pprint_thing(left)
        elif is_categorical_dtype(left):
            left = repr(left)
    
        if PY2 and isinstance(left, string_types):
            # left needs to be printable in native text type in python2
            left = left.encode('utf-8')
    
        if isinstance(right, np.ndarray):
            right = pprint_thing(right)
        elif is_categorical_dtype(right):
            right = repr(right)
    
        if PY2 and isinstance(right, string_types):
            # right needs to be printable in native text type in python2
            right = right.encode('utf-8')
    
        msg = """{obj} are different
    
    {message}
    [left]:  {left}
    [right]: {right}""".format(obj=obj, message=message, left=left, right=right)
    
        if diff is not None:
            msg += "\n[diff]: {diff}".format(diff=diff)
    
>       raise AssertionError(msg)
E       AssertionError: DataFrame.iloc[:, 0] are different
E       
E       DataFrame.iloc[:, 0] values are different (100.0 %)
E       [left]:  [1, 2]
E       [right]: [3, 4]

pandas/util/testing.py:1075: AssertionError
=============================================================== 1 failed, 12 deselected in 0.43 seconds ===============================================================

If we add the line __tracebackhide__ = True at the top of tm.assert_raise_detail. we instead get:

$ python -m pytest pandas/tests/test_errors.py -k test_hide

______________________________________________________________________________ test_hide ______________________________________________________________________________

    def test_hide():
        left = pd.DataFrame([1, 2])
        right = pd.DataFrame([3, 4])
>       tm.assert_equal(left, right)

pandas/tests/test_errors.py:14: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
pandas/util/testing.py:1531: in assert_equal
    assert_frame_equal(left, right, **kwargs)
pandas/util/testing.py:1450: in assert_frame_equal
    obj='DataFrame.iloc[:, {idx}]'.format(idx=i))
pandas/util/testing.py:1286: in assert_series_equal
    obj='{obj}'.format(obj=obj))
pandas/_libs/testing.pyx:59: in pandas._libs.testing.assert_almost_equal
    cpdef assert_almost_equal(a, b,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   raise_assert_detail(obj, msg, lobj, robj)
E   AssertionError: DataFrame.iloc[:, 0] are different
E   
E   DataFrame.iloc[:, 0] values are different (100.0 %)
E   [left]:  [1, 2]
E   [right]: [3, 4]

pandas/_libs/testing.pyx:173: AssertionError
=============================================================== 1 failed, 12 deselected in 0.29 seconds ===============================================================

which I claim is more readable, particularly when scrolling through a bunch of test failures. I would also suggest putting this in tm.assert_produces_warning since that adds a lot of noise to test output that I don't find very useful.

[gfyoung]

@jbrockmendel : Gotcha. I think is reasonable, both for raise_assert_detail and assert_produces_warning. I suspect others could benefit as well.