Add capability to handle tmd input

[martinholmer]

Add code to handle new private tmd.csv in a manner like the private puf.csv is handled.

The private tmd.csv file contains 2021 data and has its own public tmd_weights.csv.gz file containing weights for 2021 through 2074. Also, the new public growfactors.csv file has factors extending through 2074.

These three files are created in the tax-microdata repository using the following scripts:

• create_taxcalc_input_variables.py produces the private tmd.csv file
• create_taxcalc_sampling_weights.py produces the public tmd_weights.csv.gz file
• create_taxcalc_growth_factors.py produces the public growfactors.csv file

The code in this pull request still specifies the Policy.LAST_BUDGET_YEAR to be 2034; a subsequent pull request will increase it to 2074 (which has already been tested).

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.42%. Comparing base (c339245) to head (6a869c6).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #2740   +/-   ##
=======================================
  Coverage   99.42%   99.42%           
=======================================
  Files          13       13           
  Lines        2594     2603    +9     
=======================================
+ Hits         2579     2588    +9     
  Misses         15       15           

Flag	Coverage Δ
unittests	99.42% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files	Coverage Δ
taxcalc/data.py	99.24% <ø> (ø)
taxcalc/growdiff.py	100.00% <100.00%> (ø)
taxcalc/growfactors.py	100.00% <100.00%> (ø)
taxcalc/records.py	100.00% <100.00%> (ø)
taxcalc/taxcalcio.py	100.00% <100.00%> (ø)

[jdebacker]

Is it important to provide an upper bound on the versions for certain packages, like numpy and pandas? Historically, Tax-Calculator has tried to remain compatible with the most recent versions of the dependencies.

[martinholmer]

There are several deprecation warnings in the code (mostly related to pandas) that say the current code will cause errrors when pandas 3.0 is used. I check the pandas web site and they said 3.0 would be released any day now.

So, the limits are temporary. They will be removed when the deprecated code gets fixed.

[jdebacker]

@martinholmer I've reviewed this PR and had one small question left in my review.

But there is a larger issue that this PR touches on that might be worth discussion before merging. There has been work in the past (see PR #2538) to disentangle the tax calculator logic from the data used for specific calculations. Things like the cps_constructor method and the default use of the puf.csv are a convenience, but come at a cost of making implicit assumptions about the quality of certain data files that are outside the scope of Tax-Calculator. This PR adds additional methods for convenience with a(nother) particular data file. I can see the advantage of these conveniences, but on the other hand it does push in the opposite direction that the project was hoping to go. cc @MattHJensen thoughts?

[martinholmer]

@jdebacker said in PR #2740:

But there is a larger issue that this PR touches on that might be worth discussion before merging. There has been work in the past (see PR #2538) to disentangle the tax calculator logic from the data used for specific calculations. Things like the cps_constructor method and the default use of the puf.csv are a convenience, but come at a cost of making implicit assumptions about the quality of certain data files that are outside the scope of Tax-Calculator. This PR adds additional methods for convenience with a(nother) particular data file. I can see the advantage of these conveniences, but on the other hand it does push in the opposite direction that the project was hoping to go.

There has been no work on PR #2538 in over three years. Seems like there is little or no momentum in the "direction that the project was hoping to go".

[jdebacker]

Thanks of the PR @martinholmer. Merging.