Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Support Generalized Adjustment Criterion for Estimation plus Add Example Notebook #1297

Merged
merged 30 commits into from
Mar 10, 2025

Conversation

nparent1
Copy link
Contributor

@nparent1 nparent1 commented Jan 19, 2025

Adds support for the generalized adjustment estimands at the estimation stage. The PR primarily does two things:

  • Abstracts the get_backdoor() method of the IdentifiedEstimand class to instead be a get_adjustment_set() method. Then if the identifier_method is specified as 'general_adjustment', the get_adjustment_set() method will return the general adjustment set (rather than explicitly a backdoor set). Then throughout the repo where get_backdoor_set() was called for estimation purposes, we instead call get_adjustment_set(). Note that because we have get_adjustment_set() default to the backdoor set, the overall behavior is the same as before, unless a user explicitly sets general_adjustment as the identifier_method.
  • Adds various tests and test utils. In particular, adds support for specifying explicit graphs in the estimation tests, for more specific tests of the general_adjustment criterion.

An example notebook is also added, dowhy_generalized_covariate_adjustment_estimation_example.ipynb

This is a follow-on PR to #1292

Issue Ref: #402

Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1 nparent1 changed the title [FEA] Support Generalized Adjustment Criterion for Estimation plus Add Example Notebook [WIP][FEA] Support Generalized Adjustment Criterion for Estimation plus Add Example Notebook Jan 19, 2025
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1
Copy link
Contributor Author

This will need to be rebased once #1292 merges

Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1 nparent1 changed the title [WIP][FEA] Support Generalized Adjustment Criterion for Estimation plus Add Example Notebook [FEA] Support Generalized Adjustment Criterion for Estimation plus Add Example Notebook Jan 27, 2025
@nparent1 nparent1 marked this pull request as ready for review January 27, 2025 04:14
@nparent1 nparent1 marked this pull request as draft January 27, 2025 04:18
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1 nparent1 force-pushed the np/gen_adjustment_estimation branch from 5e10aff to 710b045 Compare January 28, 2025 01:30
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1 nparent1 marked this pull request as ready for review January 29, 2025 03:59
@nparent1
Copy link
Contributor Author

@amit-sharma - this is a follow on to #1292 to add support for general adjustment for estimation. It is ready for review

Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1
Copy link
Contributor Author

nparent1 commented Mar 7, 2025

Tagging in @bloebp as another possible reviewer!

Also - it seems like I don't have permission to actually request a reviewer in the top right

Copy link
Member

@amit-sharma amit-sharma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the reminder, @nparent1 Was travelling and could not get to this sooner.

The notebook example is very clear and well-written. Thanks for adding.

To clarify, even if we miss changing get_backdoor_variables to get_adjustment_set in some places, the code should still work if it was assuming backdoor, right?

I'm just trying to understand how wide these changes would apply and if there is a risk of breaking existing functionality.

Currently there is an instrumental variable estimator test failing---can you take a look at that in the build_docs run? Adding other comments inline

Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1 nparent1 force-pushed the np/gen_adjustment_estimation branch from 3a4f819 to ff6c08a Compare March 9, 2025 04:34
nparent1 added 2 commits March 8, 2025 23:36
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
Signed-off-by: Nicholas Parente <parentenickj@gmail.com>
@nparent1
Copy link
Contributor Author

Thanks for the review @amit-sharma, regarding your questions

To clarify, even if we miss changing get_backdoor_variables to get_adjustment_set in some places, the code should still work if it was assuming backdoor, right?

I'm just trying to understand how wide these changes would apply and if there is a risk of breaking existing functionality.

Correct - anywhere get_backdoor_variables() is being called should still work exactly as before, since we are still storing the backdoor variables exactly as before on the identified estimand. This PR allows users to set the identifier method to use the general adjustment criterion instead, but unless they do so everything should behave exactly as before

Currently there is an instrumental variable estimator test failing---can you take a look at that in the build_docs run? Adding other comments inline

Thanks for calling out - that should be working now. Fixed in commit ff6c08a

That said - there is still a build failing but it looks like potentially a fluke? It says "ModuleNotFoundError: No module named 'torch._dynamo'", an error I've seen in past PRs (like #1292)

I think this is ready for review again!

@nparent1 nparent1 requested a review from amit-sharma March 10, 2025 01:25
Copy link
Member

@amit-sharma amit-sharma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for these changes, @nparent1 Looks good to me.

@amit-sharma amit-sharma merged commit 996d227 into py-why:main Mar 10, 2025
36 of 37 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants