Implementation for multilabel keras model

[tkmamidi]

Hello,
Is there an implementation for using PermutationImportance for multi class models (using tensorflow sequential)?
Please help!
Thanks in advance :)

[lopuhin]

My understanding is that PermutationImportance assumes an sklearn model (although it requires very little from it so it may be quite easy to wrap a TF model in a suitable interface), but get_score_importances (https://eli5.readthedocs.io/en/latest/autodocs/permutation_importance.html) is model-agnostic as it only needs a score_func to be defined.

[tkmamidi]

Thanks for the suggestion.
I'm also trying to implement "explain_prediction" for my multiclass sequential model. I get the following error when using the following command:
eli5.explain_prediction_keras(model,test_x[0])
Error: model "sequential" is not supported, try passing the "image" argument if explaining an image model.

Any help is appreciated. Thanks in advance :)

[teabolt]

Hey. eli5.explain_prediction_keras is only for explaining Keras models using the Grad-CAM method. Currently it only supports image classifiers https://github.com/TeamHG-Memex/eli5/blob/master/eli5/keras/explain_prediction.py#L110.

[tkmamidi]

Is there an implementation to explain predictions for categorical multi-class sequential models?
Please help!
Thanks in advance!

[teabolt]

Multi-class models are supported, but the input of the model has to be an image. There's also a PR for text inputs #325. If your model doesn't accept images or text and you still want to use Grad-CAM, I'm not sure if there is an easy way. Grad-CAM only makes sense if your model layers have spatial information, like convolutional layers or recurrent layers do.

I'm not sure if a black box method can be used to explain a single prediction?

[tkmamidi]

SHAP has an implementation but was wondering if eli5 has something similar.
https://slundberg.github.io/shap/notebooks/Iris%20classification%20with%20scikit-learn.html