Add benefit test that compares actual and target benefit amounts/counts #246
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request adds a benefit test that tries to do the same comparison as was done in issue #241: that is, compare actual aggregate benefit amounts and counts (filing units receiving the benefit) with target benefit amounts and counts for each year after the start year. The target amounts/counts are the start year amounts/counts blown-up by the factors in the
cps_stage4/growth_rates.csvfile. The actual amounts/counts are derived from the extrapolated-benefit information in thecps_stage4/cps_benefits.csv.gzfile. All the amounts and counts are weighted.The new test was developed off the master branch (before pull request #242 was merged), so the results of the new test are difficult to interpret. The plan is that after this pull request has been reviewed and merged, the new master branch will be merged into pull request #242. Only then will the test results provide guidance on the quality of the extrapolated benefits.
Right now, in order to make the new test pass, the relative tolerance for actual-vs-target differences has to be set to a 0.15 value. Hopefully, when the new test is merged into pending pull request #242, the new test will pass when assuming a much lower value for the relative tolerance.
@andersonfrailey @hdoupe @MattHJensen