Add MTR Plotting

[GoFroggyRun]

This PR (mainly) adds two functions to utils.py file, namely the get_mtr_data() function and the mtr_plot function. Some tiny modifications have been applied to certain current functions. (mostly generalizations, and their tests have been updated accordingly)
Instructions, descriptions as well as test suites are available for the two aded functions. Also, the added styles.py file defines a few layout formats for the plot, which can be used for all further plotting.
Comments, concerns or remarks on these functions are more than welcomed.
@MattHJensen @martinholmer

It seems that the Travis CI have failed since we don't really have Bokeh package installed. T.J. @talumbau, I was wondering would it be sufficient if I just add Bokeh to the environment.yml file? Please advice, thanks!

[MattHJensen]

@talumbau, do you have any feedback on this PR? I'll be interested to hear from @martinholmer once he is back in town next week as well. Is there anything @GoFroggyRun could add to help you assess this?

The goal is to start adding some out-of-the-box plotting capabilities to TC. I expect we'll have several other default plots we'll want to add as well.

[MattHJensen]

@GoFroggyRun, do you know what's going on w the test failures?

[GoFroggyRun]

@MattHJensen , it seems that the test failures are coming from the fact that Bokeh is not installed on TravisCI (and I am not so sure how to install). There's no test error on my local machine.

[talumbau]

Sorry for the delay in reply. This is an interesting PR and definitely adds new capabilities to Tax-Calculator. With this PR as-is, it would mean that bokeh is now a hard dependency of the Tax-Calculator. So, if someone were to install taxcalc and not install bokeh, they would get error messages on the lack of bokeh. bokeh is a pretty hefty package, so I don't know if the right answer is to insist that bokeh be installed when Tax-Calculator is installed. I would prefer to make bokeh a "soft" dependency, in the same way that numba is a soft dependency. The idea would be that if bokeh is not installed, if the user attempts to call the functions that need bokeh, one would get an error message saying "bokeh must be installed. Please execute conda install bokeh to install bokeh."

[talumbau]

To answer @GoFroggyRun's question about how to install bokeh in the Travis CI test environment, you need to modify the .travis.yml file. In this line:

https://github.com/open-source-economics/Tax-Calculator/blob/master/.travis.yml#L16

you should change it to include bokeh as a package that is installed in to the test environment, so the line would look like this:

  - conda create -n taxcalcdev python=$TRAVIS_PYTHON_VERSION pytest setuptools pandas=0.18 toolz six mock bokeh

[martinholmer]

@talumbau suggested:

bokeh is a pretty hefty package, so I don't know if the right answer is to insist that bokeh be installed when Tax-Calculator is installed. I would prefer to make bokeh a "soft" dependency, in the same way that numba is a soft dependency. The idea would be that if bokeh is not installed, if the user attempts to call the functions that need bokeh, one would get an error message saying "bokeh must be installed. Please execute 'conda install bokeh' to install bokeh."

I like this suggestion, but if this suggestion is followed how do the unit tests work? Can test_mtr_plot in the test_utils.py file be written so that it is automatically skipped when bokeh is not installed?

@GoFroggyRun @MattHJensen

[martinholmer]

@GoFroggyRun, I don't understand your proposed new file called taxcalc/styles.py. For example, what does microseconds have to do with plotting marginal tax rates? If there is anything in the styles.py file that is used by your mtr plotting capability, please put what is needed in your new utility function. I don't see the need for a new taxcalc/styles.py file.

[GoFroggyRun]

@martinholmer:

 I don't see the need for a new taxcalc/styles.py file.

Thanks for your comments. I added the taxcalc/styles.py file that includes some other layout which might be helpful for other future plotting functions. And yes, you are right, the file is not necessary for the moment, and thus I removed it.

bokeh is a pretty hefty package, so I don't know if the right answer is to insist that bokeh be installed when Tax-Calculator is installed. I would prefer to make bokeh a "soft" dependency, in the same way that numba is a soft dependency. The idea would be that if bokeh is not installed, if the user attempts to call the functions that need bokeh, one would get an error message saying "bokeh must be installed. Please execute 'conda install bokeh' to install bokeh."

I agreed as well. But I'm having similar concerns, which is how do the unit tests work if we were not forcing bokeh dependency?

After adding bokeh to the environment (for the moment, and before the previous question is dealt with), I'm still having one error message reads

NameError: name 'TAXDATA' is not defined

I have no idea why I'm having such error message, as this TAXDATA file has been used many times in test_utils.py file.

[codecov-io]

Current coverage is 98.02% (diff: 93.39%)

Merging #948 into master will decrease coverage by 0.24%

@@             master       #948   diff @@
==========================================
  Files            35         35          
  Lines          2089       2181    +92   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
+ Hits           2053       2138    +85   
- Misses           36         43     +7   
  Partials          0          0          

Powered by Codecov. Last update 3718be0...89bbd48

[martinholmer]

@GoFroggyRun, You need to add more tests of your new code in pull request #948.

@MattHJensen

[GoFroggyRun]

@martinholmer:
I just added one more test to ensure codecov tests will pass. Please take a look.

[martinholmer]

@GoFroggyRun said:

I just added one more test to ensure codecov tests will pass.

Good.

I'll try to do some plotting with this pull request soon and give you some feedback.

But meanwhile, what is the answer to my question about what is going to happen on computers where bokeh is not installed. What is going to happen with the unit tests? It is not acceptable that they fail given all the reasons advanced by @talumbau.

@GoFroggyRun @MattHJensen

[talumbau]

Sorry for the delay in reply. If this PR is functionally complete, I would like to suggest some additional commits, which I can do as a pull request to Sean's branch. The commits that I will add will make the following changes:

• for this branch as-is, taxcalc will fail to import if bokeh is not installed. In my proposed commits, taxcalc will succeed at importing even in bokeh is not installed
• the function mtr_plot will raise an exception when called if bokeh is not installed. The exception will contain a message telling the user to install bokeh in order to access this functionality. If bokeh is installed, the mtr_plot function will execute as normal.
• In this pull request, we test the functions in the presence of bokeh. I will add some way of testing in the absence of bokeh (similar to what we do with Numba right now).

[martinholmer]

@talumbau said:

Sorry for the delay in reply. If this PR is functionally complete, I would like to suggest some additional commits, which I can do as a pull request to Sean's branch. The commits that I will add will make the following changes:

• for this branch as-is, taxcalc will fail to import if bokeh is not installed. In my proposed commits, taxcalc will succeed at importing even in bokeh is not installed
• the function mtr_plot will raise an exception when called if bokeh is not installed. The exception will contain a message telling the user to install bokeh in order to access this functionality. If bokeh is installed, the mtr_plot function will execute as normal.
• In this pull request, we test the functions in the presence of bokeh. I will add some way of testing in the absence of bokeh (similar to what we do with Numba right now).

@talumbau, These contributions would be very much appreciated, especially the last one.

@GoFroggyRun @MattHJensen

[GoFroggyRun]

@martinholmer sorry about the delay. Just addressed all your comments in the latest commit, please take a look.

[martinholmer]

@GoFroggyRun said:

Just addressed all your comments in the latest commit, please take a look.

This is much better documentation of the get_mtr_data function, but I'm still confused about the complex_weight parameter, which has a default value of False. Looking at the code, which is not something anybody using the get_mtr_data function should have to do, it seems as if "complex" simply means that the income measure percentile bins are constructed using sample weights. Is that correct? If so, why the use of "complex"? Why not something a little more informative, like "weighted_income_measure"?

If my interpretation is correct, I have two questions (one minor and one major) about the effect of changing complex_weight from False to True.

(1) Why do the percentile bins run from 0 to 99 when complex_weight=False, but run from 1 to 100 when complex_weight=True? This anomaly does nothing but create doubts in the minds of users.

(2) Why are the graphs so different? [At this stage I should be able to show you the two very different looking graphs, but I don't know how to do that and there isn't any explanation about how to do that in the documentation of the mtr_plot function.] Here is the script is used:

$ cat pr-948-test.py
from taxcalc import *
calcx = Calculator(policy=Policy(), records=Records())
calcy = Calculator(policy=Policy(), records=Records())
calcy.policy.implement_reform({2013: {'_SS_Earnings_c': [1e99]}})
df = get_mtr_data(calcx, calcy, complex_weight=False) # change False to True
fig = mtr_plot(df, plot_width=850, plot_height=500)
output_file('pr-948-fig-complexweight-F.html') # change F to T
show(fig)

I ran it twice: once the way it is above and once with the two changes described in the comments. [I will send you a private email with the two HTML plots attached.]

The plot of data generated with complex_weight=False, shows that the top 10% of (unweighted) filing units would have higher taxes. But the plot of data generated with complex_weight=True, shows that the top 45% of (weighted) filing units would have higher taxes. I simply do not believe that nearly half of the population will have higher taxes when the Social Security maximum taxable earnings is raised to infinity. Have I done something wrong?

[GoFroggyRun]

@martinholmer, regarding your concerns:

Why not something a little more informative, like "weighted_income_measure"?

I changed the option name to weight_by_income_measure. (See reasons below)

(1) Why do the percentile bins run from 0 to 99 when complex_weight=False, but run from 1 to 100 when complex_weight=True? This anomaly does nothing but create doubts in the minds of users.

Fixed. Thanks for noticing this.

(2) Why are the graphs so different? [At this stage I should be able to show you the two very different looking graphs, but I don't know how to do that and there isn't any explanation about how to do that in the documentation of the mtr_plot function.]

I just realized my wording wasn't quite right. The option actually allows s006 not only to be sorted according to certain income measure, but also to be weighted by this type of income measure. So it's not a matter of being weighted or not, but, rather, we are looking at completely different measure when weight_by_income_measure is True. Namely s006 (sorted by income_measure), and s006*income_measure (sorted by income_measure). Does this make sense to you?

[martinholmer]

@GoFroggyRun said in response to this question:

Why are the graphs so different? [At this stage I should be able to show you the two very different looking graphs, but I don't know how to do that and there isn't any explanation about how to do that in the documentation of the mtr_plot function.]

The plot of data generated with complex_weight=False, shows that the top 10% of (unweighted) filing units would have higher taxes. But the plot of data generated with complex_weight=True, shows that the top 45% of (weighted) filing units would have higher taxes. I simply do not believe that nearly half of the population will have higher taxes when the Social Security maximum taxable earnings is raised to infinity.

I just realized my wording wasn't quite right. The option actually allows s006 not only to be sorted according to certain income measure, but also to be weighted by this type of income measure. So it's not a matter of being weighted or not, but, rather, we are looking at completely different measure when weight_by_income_measure is True. Namely s006 (sorted by income_measure), and s006*income_measure (sorted by income_measure). Does this make sense to you?

No, this does not make any sense to me. All I know is that the graph with the weight_by_income_measure=True does not make any sense.

And, by the way, what's the answer to my question about how to embed these graphs in our conversation?

[GoFroggyRun]

@martinholmer said:

No, this does not make any sense to me. All I know is that the graph with the weight_by_income_measure=True does not make any sense.

So the second plot doesn't any longer, after setting weight_by_income_measure=True, mean that
45% of (weighted) filing units would have higher tax rate. It means that 45% of the aggregate income (sum of s006*e00200) is subject to higher tax rate. This option is not about weighted vs unweighted, but rather about what to weight s006 by. This option allows us to investigate different activity (wage/agi/etc..) of interests, instead of filling units.

And, by the way, what's the answer to my question about how to embed these graphs in our conversation?

Not sure what do you mean by embedding these graphs. Are you suggesting a different way of presenting the plots?

[martinholmer]

@GoFroggyRun said:

So the second plot doesn't any longer, after setting weight_by_income_measure=True, mean that 45% of (weighted) filing units would have higher taxes. It means that 45% of the aggregate income (sum of s006*e00200) is subject to higher taxes. This option is not about weighted vs unweighted, but rather about what to weight s006 by. This option allows us to investigate different activity (wage/agi/etc..) of interests, instead of filling units.

The above explanation make sense (except for the confusing use of "activity". But the explanation above is not what is in the documentation of the weight_by_income_measure parameter in the get_mtr_data docstring. You need to revise that docstring to explain what you are saying above, but avoid the confusing "activity" word and say what you mean directly.

@GoFroggyRun also said:

Not sure what do you mean by embedding these graphs.

Here is a concrete example: how do I put a generated graph into a LaTeX document? I know how to display an EPS graph as a LaTeX figure, but how do I include your graphs in my document?

Here is a second example: how do I show a generated graph in a GitHub (issue or pull request) conversation?

[martinholmer]

@GoFroggyRun, I was using the MTR plotting capability as currently structured in pending pull request #948 and got a result that was puzzling to me. Here is the script I ran:

$ cat pr-948-test.py
from taxcalc import *
calcx = Calculator(policy=Policy(), records=Records())
calcy = Calculator(policy=Policy(), records=Records())
calcy.policy.implement_reform({2013: {'_SS_Earnings_c': [1e99]}})
df = get_mtr_data(calcx, calcy, MARS='ALL', weight_by_income_measure=False)
# print df
fig = mtr_plot(df, plot_width=850, plot_height=500)
output_file('pr-948-fig-complexweight-F-MARS-ALL.html')
show(fig)

$ python pr-948-test.py
You loaded data for 2009.
Your data have been extrapolated to 2013.
You loaded data for 2009.
Your data have been extrapolated to 2013.

Here is a screenshot of what the show(fig) statement displayed in my browser:

I expected the red reform line to rise above the blue base line at high percentiles of filing unit earnings (e00200), but I did not expect to see any visible reform effect (that is, red higher than blue) at lower percentiles, certainly not between the 10th and 20th earnings percentile.

When I uncommented the print df statement and processed the printed data, I get this:

$ awk 'BEGIN{tol=1e-3}NR>1{d=$3-$2;if(d>tol||d<-tol)print $0,d}' mtr-plot-dump
%tile   base    reform     diff=reform-base
13  0.355000  0.361953 0.006953        <--------- ?
14  0.368135  0.370225 0.00209         <--------- ?
17  0.272222  0.277141 0.004919        <--------- ?
87  0.323556  0.325307 0.001751
89  0.340054  0.341650 0.001596
90  0.354713  0.356973 0.00226
91  0.370610  0.378307 0.007697
92  0.347936  0.386784 0.038848
93  0.352334  0.391306 0.038972
94  0.354834  0.388781 0.033947
95  0.355733  0.390448 0.034715
96  0.362743  0.405254 0.042511
97  0.363556  0.412745 0.049189
98  0.383581  0.448407 0.064826
99  0.406233  0.482432 0.076199

I expect the reform MTR to be higher than the base MTR at the high earnings percentiles, but I can't think of a reason why there should be a reform effect at the three percentiles below the 20th percentile.

Perhaps these results are showing me something about this tax reform that I don't understand and need to learn about. Can you think of what that might be? If not, is there some chance that your new get_mtr_data function is not working as expected?

@MattHJensen @feenberg @Amy-Xu @andersonfrailey

[martinholmer]

@GoFroggyRun, This comment follows up my prior comment on puzzling MTR-plotting results for a "pop-the-cap" reform. The plot below is generated using exactly the same script as used in the prior comment except for one change, the get_mtr_data parameter income_measure is now 'c00100' (AGI) instead of 'e00200' (wages and salaries):

Why is there no noticeable reform effect except in the 0-10th and 30-40th percentile ranges? This graph seems to be suggesting that nobody affected by the reform (which affects only those individuals with earnings above $113,700 in 2013) is in the top half of the AGI distribution. Can that really be true?

@MattHJensen @feenberg @Amy-Xu @andersonfrailey

[GoFroggyRun]

@martinholmer:
I just updated docstring that addresses your concerns with regard to weight_by_income_measure parameter and embedding. Please take a look.

Regarding your following concerns, I believe they have gone beyond the scope of this PR for the following reasons:

1. The main purpose of adding the get_mtr_data and mtr_plot functions, especially the latter one, are not for debugging purpose. It would be easier, in my opinion, to start with suspicious records themselves.
2. The get_mtr_data function simply collects some existed taxcalc utilities and puts them together to prepare for plotting data, where the calculation of mtrs and the classification of bins are based upon mtr() and add_weighted_decile_bins respectively. If there's some case where the get_mtr_data function would generate unexpected data (comparing with calling mtr() and add_weighted_decile_bins separately to generate data of similar format), I'd be more than happy to investigate get_mtr_data function. Otherwise, even if there are indeed some issues, they are more likely to come from mtr(), add_weighted_decile_bins, or both functions.
3. I'd like to note that (although this probably won't fully address your latest concern about plots against AGI) c00100 are calculated values and thus would be variant under reforms. Meaning that, when implementing different reforms, filers' AGI will be different and thus the bins for each line will be different as well.

That said, if there are records with odd MTRs and you'd like me to investigate, I'd be more than happy to take a look at them in a separate issue. Or, if there are cases where get_mtr_data function won't coincide with results generated by calling mtr() and add_weighted_decile_bins separately, I'd be more than happy to deal with them within this PR as well.

[MattHJensen]

@GoFroggyRun, could you run @martinholmer's plot and see if you get the same thing. Once you have, could you try to identify whether the problem is with get_mtr_data, _mtr(), or add_weighted_decile_bins? If there is a problem in any of those areas, then it will mean that the new features in this PR are not helpful until the problem is fixed.

[martinholmer]

@GoFroggyRun, I have learned something on my own from the plot of average combined MTR by wage-and-salary (e00200) percentile for a pop-the-cap reform, which is shown again here:

The increase in MTR for those just below the 20th wage-and-salary percentile is caused most likely by filing units with little or no wages-and-salaries, but considerable self-employment income. So that graph is no longer puzzling to me.

However, I'm still puzzled by the plot of average combined MTR by AGI (c00100) percentile for a pop-the-cap reform, which is shown again here:

My expectation is that AGI is unchanged by a pop-the-cap reform, whose reform dictionary is as follows:

{2013: {'_SS_Earnings_c': [1e99]}}

So, your argument that "AGI (c00100) is a calculated value and thus would be variant under reforms" is true in general, but not in this case, unless I'm missing something. Am I missing something about the effects of this particular reform?

[GoFroggyRun]

All right. I have some very mysterious findings.

First of all, @martinholmer is absolutely right about the plot. This is not how the plot should be looking like.

The correct plot should be the following:

I was able to generate the same plot in two, somewhat independent, ways. The first method is based on @martinholmer's script, but with modifications:

from taxcalc import *
calcx = Calculator(policy=Policy(), records=Records())
calcy = Calculator(policy=Policy(), records=Records())
calcy.policy.implement_reform({2013: {'_SS_Earnings_c': [1e99]}})
df = get_mtr_data(calcx, calcy, MARS='ALL', income_measure='e00200', weight_by_income_measure=False)
df = get_mtr_data(calcx, calcy, MARS='ALL', income_measure='c00100', weight_by_income_measure=False)
# print df
fig = mtr_plot(df, plot_width=850, plot_height=500)
output_file('pr-948-fig-complexweight-F-MARS-ALL.html')
show(fig)

I'll prepare the other method in another post.

[GoFroggyRun]

Here's the other script that yields the same plot:

from taxcalc import *
calcx = Calculator(policy=Policy(), records=Records())
calcy = Calculator(policy=Policy(), records=Records())
calcy.policy.implement_reform({2013: {'_SS_Earnings_c': [1e99]}})
a, mtr_iit_x, mtr_combined_x = calcx.mtr()
a, mtr_iit_y, mtr_combined_y = calcy.mtr()
df_x = exp_results(calcx)
df_y = exp_results(calcy)
df_x['combine'] = mtr_combined_x
df_y['combine'] = mtr_combined_y
df_y['c00100'] = df_x['c00100']
df_x.sort_values(by='c00100', inplace=True)
df_y.sort_values(by='c00100', inplace=True)
df_x = add_weighted_decile_bins(df_x, income_measure='c00100',
                             num_bins=100)
df_y = add_weighted_decile_bins(df_y, income_measure='c00100',
                             num_bins=100)
weighted_effect_x={}
weighted_effect_y={}

for i in range(1,101):
    temp = df_x[df_x['bins']==i]
    weighted_effect_x[i] = sum(temp['s006']*temp['combine'])/sum(temp['s006'])

for i in range(1,101):
    temp = df_y[df_y['bins']==i]
    weighted_effect_y[i] = sum(temp['s006']*temp['combine'])/sum(temp['s006'])

weighted_effect_x = pd.DataFrame.from_dict(weighted_effect_x, orient="index")
weighted_effect_y = pd.DataFrame.from_dict(weighted_effect_y, orient="index")

merged = pd.concat([weighted_effect_x, weighted_effect_y], axis=1, ignore_index=True)
merged.columns = ['base', 'reform']
merged.index = (merged.reset_index()).index
output_file('pr-948-fig-complexweight-F-MARS-ALL.html')
fig = mtr_plot(merged)
show(fig)

[GoFroggyRun]

I'm very confused by the situation for the first script. The commands here

df = get_mtr_data(calcx, calcy, MARS='ALL', income_measure='e00200', weight_by_income_measure=False)
df = get_mtr_data(calcx, calcy, MARS='ALL', income_measure='c00100', weight_by_income_measure=False)

seem to be duplicated. It will, however, generate different result if either line got removed. Having no idea what's going on.

cc @MattHJensen

[martinholmer]

@GoFroggyRun, Let me be very clear about what I did. This is the script I executed by entering python pr-948-test.py at the OS command prompt:

$ cat pr-948-test.py
from taxcalc import *
calcx = Calculator(policy=Policy(), records=Records())
calcy = Calculator(policy=Policy(), records=Records())
calcy.policy.implement_reform({2013: {'_SS_Earnings_c': [1e99]}})
df = get_mtr_data(calcx, calcy, income_measure='c00100')
fig = mtr_plot(df, plot_width=850, plot_height=500)
output_file('pr-948-fig.html')
show(fig)

Note that I have never called the get_mtr_data function twice in a row as you seem to incorrectly believe.

As I said before, here is the plot this script shows in my browser:

Why can't I produce what you say is the "correct" graph?

[GoFroggyRun]

@martinholmer I didn't say that you called the get_mtr_data function twice in a row.
I'm just posting what I have found, which is calling the get_mtr_data twice will yield the correct plot.

[GoFroggyRun]

Ahh, looks like we forgot to call calc_all.

The following code generates the correct plot.

from taxcalc import *
calcx = Calculator(policy=Policy(), records=Records())
calcy = Calculator(policy=Policy(), records=Records())
calcy.policy.implement_reform({2013: {'_SS_Earnings_c': [1e99]}})
calcx.calc_all()
calcy.calc_all()
df = get_mtr_data(calcx, calcy, MARS='ALL', income_measure='c00100', weight_by_income_measure=False)
# print df
fig = mtr_plot(df, plot_width=850, plot_height=500)
output_file('pr-948-fig-complexweight-F-MARS-ALL.html')
show(fig)

[feenberg]

On Fri, 21 Oct 2016, Sean.Wang wrote:

@martinholmer said:

  No, this does not make any sense to me. All I know is that the
  graph with the weight_by_income_measure=True does not make any
  sense.

So the second plot doesn't any longer, after setting
weight_by_income_measure=True, mean that
45% of (weighted) filing units would have higher taxes. It means that 45% of
the aggregate income (sum of s006*e00200) is subject to higher taxes. This
option is not about weighted vs unweighted, but rather about what to weight
s006 by. This option allows us to investigate different activity
(wage/agi/etc..) of interests, instead of filling units.

Another way to understand s006*e00200 is that it will give us the effect
on the average dollar of wage income, rather than the average taxpayer.
This is relevant if one wants to multiply the amount by an elasticity. If
the average dollar of income is taxed at a high rate, that makes the
elasticity important, even if the average taxpayer is taxed at a low rate.
Consider the propensity of some to dismiss elasticities of taxable income
with respect to income on the grouds that only high income taxpayers have
elastic taxable income. That may be true, but it isn't an argument.

BTW, is there a link to the page that this is about?

dan

  And, by the way, what's the answer to my question about how to
  embed these graphs in our conversation?

Not sure what do you mean by embedding these graphs. Are you suggesting a
different way of presenting the plots?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the
thread.[AHvQVeJsyDZYKpzKcKQbDpWRK6OuoQ6Gks5q2YGCgaJpZM4KG_GC.gif]

[martinholmer]

@GoFroggyRun, Thank your for your substantial contribution in pull request #948 that adds a marginal tax rate graphing capability. I'm merging this pull request into the master branch.