Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

T-Learner ATE, SE calculations #783

Open
ras44 opened this issue Jun 13, 2024 · 1 comment
Open

T-Learner ATE, SE calculations #783

ras44 opened this issue Jun 13, 2024 · 1 comment
Assignees
Labels
question Further information is requested

Comments

@ras44
Copy link
Collaborator

ras44 commented Jun 13, 2024

Describe the bug
Less of a bug than a question:

The T-learner takes the mean of the treatment effect te which is calculated over all subjects (so the mean over all rows of differences between each treatment group's model prediction and the control model prediction):

for i, group in enumerate(self.t_groups):
_ate = te[:, i].mean()

However, the standard errors of the ATE are calculated relative to a filtered subset- only the subjects that are within a particular treatment group and those in the control group are included:

se = np.sqrt(
(
(y_filt[w == 0] - yhat_c[w == 0]).var() / (1 - prob_treatment)
+ (y_filt[w == 1] - yhat_t[w == 1]).var() / prob_treatment
+ (yhat_t - yhat_c).var()
)
/ y_filt.shape[0]
)

It seems like the subjects in the ATE calculation should match the subjects in the SE calculation, with the SE potentially simply just being the SE of the te measurements for all subjects, if all subjects are meant to be included in the calculation.

If all subjects are not included in the ATE calculation and the ATE calculation is group-specific, then it seems like we should have:

_ate  = (yhat_t - yhat_c).mean()

And again the SE simply being the SE of the series:

se = np.sqrt((yhat_t - yhat_c).var())
@ras44 ras44 added the question Further information is requested label Jun 13, 2024
@jeongyoonlee
Copy link
Collaborator

@huigangchen, can you take a look at this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants