Optional-dependency kaggle error

[mauicv]

The optional dependency pattern doesn't protect against incompatible preinstalled versions of libraries. For instance (see #707), if you install Alibi in Kaggle, you get the following error when on import:

...
/opt/conda/lib/python3.7/site-packages/alibi/explainers/shap_wrappers.py in <module>
      8 import pandas as pd
      9 import shap
---> 10 import shap.utils._legacy as shap_utils
     11 from scipy import sparse
     12 from scipy.special import expit

ModuleNotFoundError: No module named 'shap.utils'

This is because Kaggle comes packaged with shap==0.35.0 whereas alibi requires shap>=0.40.0. Thus in the import_optional function when we import shap we get no error because a version of shap exists. In the next line we try to import shap_utils from shap.utils._legacy and this file doesn't exist in 0.35.0. Previously Alibi managed Shap as an optional dependency by trying to import it into explianers.__init__ and only adding it to the __all__ if it succeeded. The optional dependency functionality however imports it and only replaces it with the MissingDependency object if we receive an import error from the import shap statement. Because of Shaps presence in the environment, this error is not thrown however the module shap.utils._legacy doesn't exist so we get a ModuleNotFoundError instead!

---

As a quick fix you can either uninstall shap or install the correct versions:

pip install numba>=0.50.0 shap==0.40.0 alibi

---

In terms of fixing the issue within Alibi, we do actually check for ModuleNotFoundError errors in import_optional but we try to match it to the specific library and I think the file module name is throwing it off...

[mauicv]

Note: Kaggle has since updated its environment to have shap==0.41.00 as default!