reject is a Python library for classification with rejection. Neural networks are often confidently wrong when confronted with out-of-distribution data. When the prediction's uncertainty is too high, the model abstains from predicting and the observation is passed on to a human expert who takes the final decision. It is useful for applications where making an error can be more costly than asking a human expert for help.
$ pip install rejectThe documentation is deployed to reject.readthedocs.io.
from reject.reject import ClassificationRejector
y_pred # Array of predictions. Shape (n_observations, n_classes) or (n_observations, n_samples, n_classes).
y_true # Array of true labels. Shape (n_observations,).
# initialize the rejector
rej = ClassificationRejector(y_true_all, y_pred_all)# single rejection point
rej.reject(threshold=0.5, unc_type="TU", relative=True, show=True) Non-rejected Rejected
--------- -------------- ----------
Correct 891 20
Incorrect 109 980
Non-rejected accuracy Classification quality Rejection quality
----------------------- ------------------------ -------------------
0.8910 0.9355 40.9908# rejection curve
fig = rej.plot_reject(unc_type="TU", metric="NRA")
print(fig)
User guide notebooks are provided in the reject.readthedocs.io documentation.