Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to customize variables included in --dump and --sqldb output #1735

Merged
merged 9 commits into from
Dec 11, 2017
Merged

Add ability to customize variables included in --dump and --sqldb output #1735

merged 9 commits into from
Dec 11, 2017

Conversation

martinholmer
Copy link
Collaborator

This pull request adds to the Tax-Calculator CLI tc an option for specifying a partial dump. After this pull request is merged, if the tcdumpvars file exists (in the directory where tc writes output files), the --dump and --sqldb output files contain only the variables listed in the tcdumpvars file (plus RECID and FLPDYR, if they aren't in the tcdumpvars file). If the tcdumpvars file does not exist, the --dump and --sqldb output files contain all input and calculated variables, just as before this pull request.

@codecov-io
Copy link

codecov-io commented Dec 9, 2017

Codecov Report

Merging #1735 into master will not change coverage.
The diff coverage is 100%.

Impacted file tree graph

@@          Coverage Diff           @@
##           master   #1735   +/-   ##
======================================
  Coverage     100%    100%           
======================================
  Files          37      37           
  Lines        2962    2984   +22     
======================================
+ Hits         2962    2984   +22
Impacted Files Coverage Δ
taxcalc/taxcalcio.py 100% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 57df1d3...1025cca. Read the comment docs.

@martinholmer
Copy link
Collaborator Author

Using this new feature, we have the following results:

iMac2:tax-calculator mrh$ date ; tc cps.csv 2018 --dump ; date
Fri Dec  8 19:22:40 EST 2017
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2018.
Fri Dec  8 19:25:24 EST 2017

iMac2:tax-calculator mrh$ ls -l cps-18*
-rw-r--r--  1 mrh  staff        205 Dec  8 19:25 cps-18-#-#-doc.text
-rw-r--r--  1 mrh  staff  448202197 Dec  8 19:25 cps-18-#-#.csv

iMac2:tax-calculator mrh$ cp tcdumpvarsX tcdumpvars

iMac2:tax-calculator mrh$ cat tcdumpvars


RECID;MARS,iitax|combined
payrolltax


c00100

iMac2:tax-calculator mrh$ date ; tc cps.csv 2018 --dump ; date
Fri Dec  8 19:29:46 EST 2017
You loaded data for 2014.
Tax-Calculator startup automatically extrapolated your data to 2018.
Fri Dec  8 19:30:27 EST 2017

iMac2:tax-calculator mrh$ ls -l cps-18*
-rw-r--r--  1 mrh  staff        205 Dec  8 19:30 cps-18-#-#-doc.text
-rw-r--r--  1 mrh  staff   21145387 Dec  8 19:30 cps-18-#-#.csv

iMac2:tax-calculator mrh$ head -2 cps-18-#-#.csv
FLPDYR,MARS,RECID,c00100,combined,iitax,payrolltax
2018,2,1,50499.58,10243.74,2518.76,7724.98

Here is a summary of those results:

                   FULL DUMP     PARTIAL DUMP
runtime(secs):       164            41
outsize(MB):         427             2

Obviously, adding more variables to the tcdumpvars file will increase the partial dump runtime and output file size.

@martinholmer martinholmer added WIP and removed ready labels Dec 10, 2017
@martinholmer martinholmer added ready and removed WIP labels Dec 11, 2017
@martinholmer martinholmer merged commit 9860c21 into PSLmodels:master Dec 11, 2017
@martinholmer martinholmer deleted the tcdumpvars branch December 12, 2017 04:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants