Alibi is calling for an old version of Kernel Shap which utils module doesn't exist anymore

[antoinecomp]

I am trying to test anchor algorithm on an NLP issue reproducing this notebook.

Unfortunately on Colaboratory notebook it seems that Alibi is calling for an old version of Kernel Shap which utils module doesn't exist anymore:

ModuleNotFoundError: No module named 'shap.utils'

Here is the full log error:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-4-75b49dcfe672> in <module>
      5 from sklearn.model_selection import train_test_split
      6 import spacy
----> 7 from alibi.explainers import AnchorText
      8 from alibi.datasets import fetch_movie_sentiment
      9 from alibi.utils.download import spacy_model

/opt/conda/lib/python3.7/site-packages/alibi/__init__.py in <module>
----> 1 from . import confidence, datasets, explainers, prototypes, utils
      2 from .version import __version__  # noqa F401
      3 
      4 __all__ = ['confidence', 'datasets', 'explainers', 'prototypes', 'utils']

/opt/conda/lib/python3.7/site-packages/alibi/explainers/__init__.py in <module>
     35 KernelShap, TreeShap = import_optional(
     36     'alibi.explainers.shap_wrappers',
---> 37     names=['KernelShap', 'TreeShap'])
     38 
     39 __all__ = [

/opt/conda/lib/python3.7/site-packages/alibi/utils/missing_optional_dependency.py in import_optional(module_name, names)
    103             raise TypeError()
    104         if err.name not in ERROR_TYPES:
--> 105             raise err
    106         missing_dependency = ERROR_TYPES[err.name]
    107         if names is not None:

/opt/conda/lib/python3.7/site-packages/alibi/utils/missing_optional_dependency.py in import_optional(module_name, names)
     93 
     94     try:
---> 95         module = import_module(module_name)
     96         # TODO: We should check against specific dependency versions here.
     97         if names is not None:

/opt/conda/lib/python3.7/importlib/__init__.py in import_module(name, package)
    125                 break
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 
    129 

/opt/conda/lib/python3.7/site-packages/alibi/explainers/shap_wrappers.py in <module>
      8 import pandas as pd
      9 import shap
---> 10 import shap.utils._legacy as shap_utils
     11 from scipy import sparse
     12 from scipy.special import expit

ModuleNotFoundError: No module named 'shap.utils'

On the Kaggle notebook, running this notebook, it seemed I had the same version but another issue. The library wasn't found:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-12-1f7ed375b3a9> in <module>
----> 1 print(alibi.__version__)

NameError: name 'alibi' is not defined

Even if it seemed I had downloaded it correctly:

Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting alibi
  Downloading alibi-0.7.0-py3-none-any.whl (445 kB)
     |████████████████████████████████| 445 kB 7.5 MB/s 
Requirement already satisfied: spacy[lookups]<4.0.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from alibi) (3.3.1)
Requirement already satisfied: scipy<2.0.0,>=1.1.0 in /usr/local/lib/python3.7/dist-packages (from alibi) (1.4.1)
Requirement already satisfied: tqdm<5.0.0,>=4.28.1 in /usr/local/lib/python3.7/dist-packages (from alibi) (4.64.0)
Requirement already satisfied: numpy<2.0.0,>=1.16.2 in /usr/local/lib/python3.7/dist-packages (from alibi) (1.21.6)
Requirement already satisfied: requests<3.0.0,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from alibi) (2.23.0)
Requirement already satisfied: dill<0.4.0,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from alibi) (0.3.5.1)
Collecting transformers<5.0.0,>=4.7.0
  Downloading transformers-4.20.1-py3-none-any.whl (4.4 MB)
     |████████████████████████████████| 4.4 MB 33.3 MB/s 
Requirement already satisfied: pandas<2.0.0,>=0.23.3 in /usr/local/lib/python3.7/dist-packages (from alibi) (1.3.5)
Requirement already satisfied: scikit-image!=0.17.1,<0.20,>=0.14.2 in /usr/local/lib/python3.7/dist-packages (from alibi) (0.18.3)
Requirement already satisfied: attrs<22.0.0,>=19.2.0 in /usr/local/lib/python3.7/dist-packages (from alibi) (21.4.0)
Requirement already satisfied: typing-extensions>=3.7.4.3 in /usr/local/lib/python3.7/dist-packages (from alibi) (4.1.1)
Requirement already satisfied: scikit-learn<2.0.0,>=0.22.0 in /usr/local/lib/python3.7/dist-packages (from alibi) (1.0.2)
Requirement already satisfied: matplotlib<4.0.0,>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from alibi) (3.2.2)
Requirement already satisfied: Pillow<10.0,>=5.4.1 in /usr/local/lib/python3.7/dist-packages (from alibi) (7.1.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib<4.0.0,>=3.0.0->alibi) (0.11.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib<4.0.0,>=3.0.0->alibi) (3.0.9)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib<4.0.0,>=3.0.0->alibi) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib<4.0.0,>=3.0.0->alibi) (1.4.3)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas<2.0.0,>=0.23.3->alibi) (2022.1)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib<4.0.0,>=3.0.0->alibi) (1.15.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.21.0->alibi) (2022.6.15)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.21.0->alibi) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.21.0->alibi) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3.0.0,>=2.21.0->alibi) (1.24.3)
Requirement already satisfied: imageio>=2.3.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image!=0.17.1,<0.20,>=0.14.2->alibi) (2.4.1)
Requirement already satisfied: networkx>=2.0 in /usr/local/lib/python3.7/dist-packages (from scikit-image!=0.17.1,<0.20,>=0.14.2->alibi) (2.6.3)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.7/dist-packages (from scikit-image!=0.17.1,<0.20,>=0.14.2->alibi) (1.3.0)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.7/dist-packages (from scikit-image!=0.17.1,<0.20,>=0.14.2->alibi) (2021.11.2)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn<2.0.0,>=0.22.0->alibi) (1.1.0)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn<2.0.0,>=0.22.0->alibi) (3.1.0)
Requirement already satisfied: typer<0.5.0,>=0.3.0 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (0.4.1)
Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (21.3)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (2.0.6)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (1.0.7)
Requirement already satisfied: spacy-loggers<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (1.0.2)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (57.4.0)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (3.0.6)
Requirement already satisfied: pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (1.8.2)
Requirement already satisfied: srsly<3.0.0,>=2.4.3 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (2.4.3)
Requirement already satisfied: catalogue<2.1.0,>=2.0.6 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (2.0.7)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (2.11.3)
Requirement already satisfied: blis<0.8.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (0.7.7)
Requirement already satisfied: spacy-legacy<3.1.0,>=3.0.9 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (3.0.9)
Requirement already satisfied: pathy>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (0.6.1)
Requirement already satisfied: thinc<8.1.0,>=8.0.14 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (8.0.17)
Requirement already satisfied: langcodes<4.0.0,>=3.2.0 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (3.3.0)
Requirement already satisfied: wasabi<1.1.0,>=0.9.1 in /usr/local/lib/python3.7/dist-packages (from spacy[lookups]<4.0.0,>=2.0.0->alibi) (0.9.1)
Collecting spacy-lookups-data<1.1.0,>=1.0.3
  Downloading spacy_lookups_data-1.0.3-py2.py3-none-any.whl (98.5 MB)
     |████████████████████████████████| 98.5 MB 97 kB/s 
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from catalogue<2.1.0,>=2.0.6->spacy[lookups]<4.0.0,>=2.0.0->alibi) (3.8.0)
Requirement already satisfied: smart-open<6.0.0,>=5.0.0 in /usr/local/lib/python3.7/dist-packages (from pathy>=0.3.5->spacy[lookups]<4.0.0,>=2.0.0->alibi) (5.2.1)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from transformers<5.0.0,>=4.7.0->alibi) (4.11.4)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers<5.0.0,>=4.7.0->alibi) (2022.6.2)
Collecting pyyaml>=5.1
  Downloading PyYAML-6.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (596 kB)
     |████████████████████████████████| 596 kB 46.6 MB/s 
Collecting tokenizers!=0.11.3,<0.13,>=0.11.1
  Downloading tokenizers-0.12.1-cp37-cp37m-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (6.6 MB)
     |████████████████████████████████| 6.6 MB 50.2 MB/s 
Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from transformers<5.0.0,>=4.7.0->alibi) (3.7.1)
Collecting huggingface-hub<1.0,>=0.1.0
  Downloading huggingface_hub-0.8.1-py3-none-any.whl (101 kB)
     |████████████████████████████████| 101 kB 10.5 MB/s 
Requirement already satisfied: click<9.0.0,>=7.1.1 in /usr/local/lib/python3.7/dist-packages (from typer<0.5.0,>=0.3.0->spacy[lookups]<4.0.0,>=2.0.0->alibi) (7.1.2)
Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->spacy[lookups]<4.0.0,>=2.0.0->alibi) (2.0.1)
Installing collected packages: pyyaml, tokenizers, spacy-lookups-data, huggingface-hub, transformers, alibi
  Attempting uninstall: pyyaml
    Found existing installation: PyYAML 3.13
    Uninstalling PyYAML-3.13:
      Successfully uninstalled PyYAML-3.13
Successfully installed alibi-0.7.0 huggingface-hub-0.8.1 pyyaml-6.0 spacy-lookups-data-1.0.3 tokenizers-0.12.1 transformers-4.20.1

[mauicv]

Hey @antoinecomp,
Thanks for opening the issue.

I'm struggling to recreate your problem. The following notebook installs and imports the AnchorText object without issue. Shap is actually an optional dependency of alibi so won't be installed with the default pip install alibi command. As per the notebook you've linked you shouldn't have the shap library? Is it possible you installed it separately? Can you run print(shap.__version__) and let me know the output?

w.r.t your second question I think you need to import alibi before you can call print(alibi.__version__)?

[antoinecomp]

Hi @mauicv
My pleasure, I'm testing the library on the first fully reproducible notebook I found as part of a state-of-the-art publication comparing how usable different NLP XAI libraries are.

First question, colaboratory, shap imported

1. No, I shouldn't have the shap library. I was just testing alibi.
2. I can't have installed shap unless it's part of the dependencies of the other libraries or part of Kaggle or Collaboratory environment.
3. print(shap.__version__) output:

shap---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-5-fd74b2440bc5> in <module>
----> 1 print(.__version__)

NameError: name 'shap' is not defined

So I guess it was instaleld and imported with alibi.
Yet, to be honest, I struggle to reproduce the error on Colaboratory either. I can import it safely and go to another error AlibiPredictorCallException: Predictor failed to be called on x=['Hello world'] which I am going to dive into.

Regarding the second question, colaboratory

Yes probably I forgot to import the library. However, I know have this ModuleNotFoundError: No module named 'shap.utils'shap. And the output of the command to get shap version, if installed:

shap---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-5-fd74b2440bc5> in <module>
----> 1 print(.__version__)

NameError: name 'shap' is not defined

[mauicv]

Ah, ok. I've figured out the Shap issue. The Kaggle environment comes with Shap preinstalled meanwhile Colab notebooks don't. This means if you install Alibi in Kaggle there is an incompatible version of Shap present and this causes the error! Thanks for bringing this to our attention.

In terms of a quick fix you can either uninstall the Shap library:

!pip uninstall -y shap
!pip install alibi
!pip install lime

or you can install the correct versions of shap and numba:

!pip install numba>=0.50.0 shap==0.40.0 alibi
!pip install lime

I'll look into the anchor prediction issue.

[mauicv]

Hey @antoinecomp,
I think the ALibiPredictorCallException is just due to the notebook calling the AnchorText API incorrectly. The correct usage should be:

explainer = AnchorText(predictor=predict_fn, nlp=nlp)