Add aggregate.py tests

[hdoupe]

This PR adds tests for the remaining functions in aggregate.py.

@jdebacker I'm having some trouble generating data with the right dimensions for the functionrevenue. What are the expected dimensions for w , e and n?

[jdebacker]

@hdoupe thanks for opening this PR.

Re dimensions:

• w is a scalar or vector of varying length (up to length T+S)
• e is S by J
• n is S by J or T by S by J (where the first dimension will actually varying in length up to T+S - same as length of w)

[hdoupe]

For the function aggregates.revenue, I'm having trouble getting the case where I.ndim == 3 and etr_params.ndim == 3. @jdebacker do you have any idea what the dimensions should be to make this case work?

Also, I'm not sure if I can get the case where eter_params.ndim == 4. It seems like for every TPI case (which I think means I.ndim == 3) in the function revenue, etr_params is sliced by j. Since etr_params is at max 4 dimensions (at least I think this is the case), the highest dimension it can be when sent to tax.tau_income is three.

Compared to the other functions in aggregates.py, the revenue function is pretty complicated. So I saved the results from the runs on the test data and then the future results will be compared to the results that I saved this morning. @jdebacker and @rickecon does this sound sensible?

[jdebacker]

@hdoupe The docstring for aggregates.revenue() doesn't see to be quite correct. etr_params is going to be TxSxJxM, where M = number of parameters in the ETR function. When slicing, we are always trying to keep that trailing dimension as the number of parameters. Note that The ETR function does not vary over J, but is tiled along the way to make array operation easier.

I is never more than three dimensions. It is TxSxJ.

Also, I'm not sure if I can get the case where eter_params.ndim == 4. It seems like for every TPI case (which I think means I.ndim == 3) in the function revenue, etr_params is sliced by j. Since etr_params is at max 4 dimensions (at least I think this is the case), the highest dimension it can be when sent to tax.tau_income is three.

I think what you are saying is right. etr_params can enter aggregates.revenue() as a 4D object, but after the slicing on lines 251-264, it is never more than 3D when passed to tax.tau_income(). But I'm not sure of the issue it raises -- other than the fact that we probably want to refactor the code to do some better array broadcasting in tax.py so that we don't need to loop over j in places like aggregates.revenue() so that thinks are cleaner and we avoid confusing slicing as much as possible.

[hdoupe]

@jdebacker Thanks for the quick reply.

I think what you are saying is right. etr_params can enter aggregates.revenue() as a 4D object, but after the slicing on lines 251-264, it is never more than 3D when passed to tax.tau_income(). But I'm not sure of the issue it raises

Ah right, I was thinking that we didn't need the if statement for etr_params.ndim == 4. But I looked again and saw that tax.tau_income is called from other places in the code where etr_params might be 4 dimensions.

[hdoupe]

I think this should be ready to merge if you guys are OK with the tests that I've added.

When I call aggregates.revenue using TPI parameters, it returns an an array of length T. I saved the output to a csv file and plan to compare that with future results. However, I could reduce the size of T to 30 or 40 and keep the arrays in the test_aggregates.py file. What do you guys think?

@jdebacker @rickecon

[jdebacker]

@hdoupe I prefer to keep things in the test_aggregates.py file. If the function works with T=30, I don't see why we'd expect any different behavior with T being larger.

[hdoupe]

@jdebacker Ok that makes sense. The most recent commit uses T = 30 and S = 20.

[jdebacker]

@hdoupe Might use capital letters for S and J (lower case in the code refers to an item in a range from 0 to S (or J), while S/J give the max number in that dimension.

[hdoupe]

Ok, sounds good to me.

[jdebacker]

@hdoupe Maybe doesn't matter for the calculations, but omega.sum() and lambdas.sum() should both be 1 in the model.

[hdoupe]

Ok. I can normalize omega and lambdas so that they sum to 1. How does that sound?

[hdoupe]

If I do normalize omega, which dimensions should sum to 1?

[jdebacker]

The sum over the S and J dimensions should be one.

…

Sent from my iPhone

On Sep 1, 2017, at 2:17 PM, Henry Doupe ***@***.***> wrote: @hdoupe commented on this pull request. In ogusa/tests/test_aggregates.py: > aggI_TPI_test = ((1 + g_n) * np.exp(g_y) * (K_p1 - res_matrix.sum(1).sum(1) / (1 + g_n)) - (1.0 - delta) * K) aggI_TPI = aggr.get_I(b_splus1, K_p1, K, (delta, g_y, omega, lambdas, imm_rates, g_n, 'TPI')) assert (np.allclose(aggI_TPI, aggI_TPI_test)) + + +def test_get_K(): + """ + Simulate data similar to observed + """ + T = 160 + s, j = 40, 2 + + b = -0.1 + (7 * np.random.rand(T * s * j).reshape(T, s, j)) + omega = 0.5 * np.random.rand(T * s).reshape(T, s, 1) + lambdas = 0.4 + (0.2 * np.random.rand(j).reshape(1, 1, j)) If I do normalize omega, which dimensions should sum to 1? ― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[hdoupe]

@jdebacker I made the suggested changes. Do you think this is ready to merge?

[jdebacker]

@hdoupe Yes - thanks.