Failures for new benefit-repeal parameters in test_compatible_data.py

[martinholmer]

The need to resolve issue #1846 has been highlighted by the merge of pull request #1719 that adds several benefit-repeal policy parameters. The lack of compatible-data test documentation means that very few developers know how to make the tests work for new policy parameters.

@MattHJensen, could you fix these test failures and add documentation as you do that?
Otherwise, you'll have to do this every time a new policy parameter is added.
Thanks very much.

Here are the messages from the four test failures:

_________________________ test_compatible_data[True-4] _________________________
        if errors:
            print(errors)
>           assert 'compatible_data' == 'invalid'
E           AssertionError: assert 'compatible_data' == 'invalid'
E             - compatible_data
E             + invalid
tests/test_compatible_data.py:257: AssertionError
----------------------------- Captured stdout call -----------------------------
You loaded data for 2009.
Tax-Calculator startup automatically extrapolated your data to 2013.
You loaded data for 2009.
Tax-Calculator startup automatically extrapolated your data to 2013.
You loaded data for 2009.
Tax-Calculator startup automatically extrapolated your data to 2013.
You loaded data for 2009.
Tax-Calculator startup automatically extrapolated your data to 2013.
You loaded data for 2009.
Tax-Calculator startup automatically extrapolated your data to 2013.
ERROR: _BEN_Unemployment_repeal not True for puf
ERROR: _BEN_OASDI_repeal not True for puf

________________________ test_compatible_data[False-4] _________________________
        if errors:
            print(errors)
>           assert 'compatible_data' == 'invalid'
E           AssertionError: assert 'compatible_data' == 'invalid'
E             - compatible_data
E             + invalid
tests/test_compatible_data.py:257: AssertionError
----------------------------- Captured stdout call -----------------------------
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
ERROR: _BEN_Other_repeal not False for cps
ERROR: _BEN_MCARE_repeal not False for cps
ERROR: _BEN_SNAP_repeal not False for cps
ERROR: _BEN_SSI_repeal not False for cps
ERROR: _BEN_MCAID_repeal not False for cps

________________________ test_compatible_data[False-5] _________________________
        if errors:
            print(errors)
>           assert 'compatible_data' == 'invalid'
E           AssertionError: assert 'compatible_data' == 'invalid'
E             - compatible_data
E             + invalid
tests/test_compatible_data.py:257: AssertionError
----------------------------- Captured stdout call -----------------------------
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
ERROR: _BEN_Vet_repeal not False for cps

________________________ test_compatible_data[False-11] ________________________
        if errors:
            print(errors)
>           assert 'compatible_data' == 'invalid'
E           AssertionError: assert 'compatible_data' == 'invalid'
E             - compatible_data
E             + invalid
tests/test_compatible_data.py:257: AssertionError
----------------------------- Captured stdout call -----------------------------
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2014.
ERROR: _ID_BenefitCap_Switch not False for cps

[MattHJensen]

I am only seeing one failure: _ID_BenefitCap_Switch not False for cps

@hdoupe, were you able to replicate the four reported by @martinholmer when you were working on #1876?

[martinholmer]

The failures related to the new benefit-repeal parameters were caused by a bug in calculating benefit_cost_total that was fixed in pull request #1886.

But the one remaining failure is for a parameter (_ID_BenefitCap_Switch) that used to pass.

That failure has been turned into a pass with the test change in pull request #1891.