Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Strengthen tests in 10mins notebook #1610

Closed
MattHJensen opened this issue Oct 27, 2017 · 4 comments
Closed

Strengthen tests in 10mins notebook #1610

MattHJensen opened this issue Oct 27, 2017 · 4 comments

Comments

@MattHJensen
Copy link
Contributor

In #1603 I asked:

I do wonder why our notebook tests didn't catch this though.

@martinholmer replied:

Because the notebook tests are very weak. They only detect a fatal Python error.
They have never done any validity checking on the answers the notebooks produce.

All I'm saying is that many of the results produced in the Behavioral_example.ipynb notebook are numerically incorrect after the merge of PR #1582. The notebook runs without a Python crash, but produces many incorrect results.

As @martinholmer further pointed out, this indicates a need for stronger tests in the 10min notebook.

@martinholmer
Copy link
Collaborator

This comment illustrates the weakness of the testing logic in the test_notebooks.py file.

@MaxGhenis
Copy link
Contributor

I'm not familiar with the issue here, but would be happy to contribute example notebooks as a form of documentation if it can be resolved. I have a variety in https://github.com/MaxGhenis/taxcalc-notebooks of varying degrees of complexity, ranging from notebook versions of @martinholmer's recipes to larger analyses.

@martinholmer
Copy link
Collaborator

Over the past several months, this issue has been resolved indirectly by developing a Tax-Calculator Cookbook of tested Python scripts that use the taxcalc package. These scripts are tested before each release and can be easily uploaded into Jupyter Notebooks by users who like the interactive nature of notebooks.

@martinholmer
Copy link
Collaborator

Given the development of the Cookbook, this issue is being closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants