Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linear activation function interaction with ROC_AUC metric #9

Open
Aidan-Kehoe opened this issue Jan 22, 2021 · 1 comment
Open

Linear activation function interaction with ROC_AUC metric #9

Aidan-Kehoe opened this issue Jan 22, 2021 · 1 comment

Comments

@Aidan-Kehoe
Copy link

I see in the tox21 example , the evaluation metric is the roc_auc_score on the multilabel binary classification task. However, the final activation function in the model is linear. Does this output interact well with the roc_auc_score? I believe the sklearn implementation of the metric expects an array of shape (n_samples, n_classes) where the values in the array correspond to probability estimates for the labels in each class. As I understand, the current output of the model does not correspond to a probability estimate (linear activation), so does this pose an issue for getting the correct roc_auc value? Would a better implementation be with a sigmoid activation function at the output layer?
Thanks

@Aidan-Kehoe Aidan-Kehoe changed the title ROC_AUC score work with final linear activation function Linear activation function interaction with ROC_AUC metric Jan 22, 2021
@rusty1s
Copy link
Owner

rusty1s commented Jan 25, 2021

As far as I see, this is not a problem since roc_auc_score works for both probability estimates and non-thresholded decision values, see here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants