PDP variance feature importance #758

RobertSamoilescu · 2022-09-14T15:57:10Z

This PR implements the method described here to compute feature importance and feature interactions.

The main idea for the computation of both measures is to analyse the variance of the partial dependence plots, thus the implementation relies on the PartialDependece explainer.

It also adds plotting functionalities for the feature importance and feature interactions. Furthermore, since it is recommended to analyse the results in tandem with the partial dependence plots, the explanation object is designed to be compatible with the pd_plot functionality.

… barplot.

codecov · 2022-09-14T16:45:03Z

Codecov Report

Merging #758 (ca3067b) into master (313c760) will decrease coverage by 0.21%.
The diff coverage is 41.89%.

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #758      +/-   ##
==========================================
- Coverage   75.72%   75.51%   -0.22%     
==========================================
  Files          72       73       +1     
  Lines        8223     8477     +254     
==========================================
+ Hits         6227     6401     +174     
- Misses       1996     2076      +80

Flag	Coverage Δ
macos-latest-3.10	`75.47% <41.50%> (?)`
macos-latest-3.10.6	`?`
ubuntu-latest-3.10	`75.47% <41.50%> (?)`
ubuntu-latest-3.10.6	`?`
ubuntu-latest-3.7	`75.20% <41.26%> (-0.33%)`	⬇️
ubuntu-latest-3.8	`75.25% <41.26%> (-0.24%)`	⬇️
ubuntu-latest-3.9	`75.34% <41.26%> (-0.24%)`	⬇️
windows-latest-3.9	`73.41% <41.26%> (-0.18%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
alibi/explainers/pd_variance.py	`40.72% <40.72%> (ø)`
alibi/api/defaults.py	`100.00% <100.00%> (ø)`
alibi/explainers/__init__.py	`100.00% <100.00%> (ø)`
alibi/utils/missing_optional_dependency.py	`94.44% <0.00%> (+0.15%)`	⬆️
alibi/explainers/partial_dependence.py	`48.16% <0.00%> (+0.21%)`	⬆️
alibi/models/tensorflow/autoencoder.py	`100.00% <0.00%> (+43.47%)`	⬆️
alibi/models/tensorflow/cfrl_models.py	`100.00% <0.00%> (+75.67%)`	⬆️

review-notebook-app · 2022-09-15T13:14:34Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

jklaise

Thanks Robert, looks great!

Haven't reviewed the plotting as you've been thinking of improvements.

jklaise · 2022-09-15T15:03:11Z

alibi/explainers/pd_variance.py

+     `HistGradientBoostingRegressor`, `DecisionTreeRegressor`, `RandomForestRegressor`."""
+
+    def __init__(self,
+                 predictor: Union[BaseEstimator, Callable[[np.ndarray], np.ndarray]],


I see why you've unified these two types of models into a single implementation since it's easy to select the right PD implementation, I'm a little unsure about the discrepancy between PD that it introduces. On the other hand, it's nice having just one public interface for this.

Can discuss offline.

I think that this is a good approach now.

alibi/explainers/pd_variance.py

jklaise · 2022-09-15T15:08:39Z

alibi/explainers/pd_variance.py

+        self.pd_explainer = PartialDependenceClass(predictor=predictor,
+                                                   feature_names=feature_names,
+                                                   categorical_names=categorical_names,
+                                                   target_names=target_names,
+                                                   verbose=verbose)


I'm just wondering if we should catch exceptions from this initialization and re-raise with more specific exceptions? Or would all exceptions raised from the underlying class be clear enough for a user of this method?

I think the exception are clear enough in the PD class.

alibi/explainers/pd_variance.py

jklaise · 2022-09-16T14:09:11Z

alibi/explainers/pd_variance.py

+
+        Returns
+        -------
+        An array of size `F x T x N1 x ... N(k-1)`, where `F` is the number of explained features, `T` is the number


Not sure I understand why it goes up to N(k-1) not N(k).

Because we compute the variance along the Nk and thus will be removed. For a 2D matrix, if we compute the variance along the second axis, then the result will be 1D.

alibi/explainers/pd_variance.py

jklaise · 2022-09-16T14:18:52Z

alibi/explainers/pd_variance.py

+            params.update({'kind': Kind.AVERAGE.value})  # type: ignore[dict-item]
+
+        # compute partial dependence for each feature
+        pd_explanation = self.pd_explainer.explain(**params)  # type: ignore[arg-type]


Interesting question on user experience - the verbose flag is passed through to the underlying explainer. I assume the calculation after we have pd_explanation is comparatively quick so it's fine to leave this as is?

Needs to be addressed somehow. One way to do it is to add a new progress bar for the computation of the variance with an appropriate description. Unfortunately, the PD progress bar does not have any description. Not sure if it is best to add a description for the PD in this PR, or add a a progress bar without description for PDV and later, another PR to add a description for both.

I don't mind where the PR's go too much. Would the proposal be to have 2 progress bars showing for this method or disabling the internal PD progress bar? Where is most of the time spent during the computation?

The most time spent during computation is in the PartialDependence. The other is relatively fast. I proposal would be to have 2 progress bars for this method.

If that's the case then perhaps easiest and simplest (for the user) to stick with the internal progress bar? Or perhaps we add the 2nd progress bar (with a different description), but disable the internal one?

jklaise · 2022-09-16T14:21:59Z

alibi/explainers/pd_variance.py

+                                                            pd_values=pd_explanation.data['pd_values']).T,
+        }
+
+    def _compute_feature_interaction(self,


Remind me if this all works fine for mixed data, i.e. (num, cat)?

Don't think I understand exactly what you mean. There shouldn't be any mixed data since the feature importance is defined for a single feature.

This is feature interaction so 2 features? How does it work for mixed features?

Sorry, my bad ... the feature interaction for mixed features works the same: fix one feature, let the other vary, and take the variance. If the fixed feature is numerical and the one that varies is categorical, then the standard deviation computation uses the range statistics. On the other hand, if the categorical if fixed and the numerical one varies, then the standard deviation computation uses the unbiased definition of the std ( divided by N-1). There shouldn't be any problem, and as yout can see, the method uses the internal _compute_pd_variance which is aware of the feature type.

…tance

…ing functionality. Solved docs errors.

…tance

This reverts commit 44dfea4.

CHANGELOG.md

jklaise · 2022-10-04T10:45:36Z

alibi/explainers/pd_variance.py

+                                                            pd_values=pd_explanation.data['pd_values']).T,
+        }
+
+    def _compute_feature_interaction(self,


This is feature interaction so 2 features? How does it work for mixed features?

jklaise

Nice work! Don't really have many comments regarding the code, also enjoyed the examples and method description, left quite a few minor comments there for improving the text and presentation.

I'm noting that the new plotting functions have an impact on the test coverage so as follow-up we should start addressing those as part of #760.

jklaise · 2022-10-04T10:48:16Z

alibi/explainers/pd_variance.py

+            params.update({'kind': Kind.AVERAGE.value})  # type: ignore[dict-item]
+
+        # compute partial dependence for each feature
+        pd_explanation = self.pd_explainer.explain(**params)  # type: ignore[arg-type]


I don't mind where the PR's go too much. Would the proposal be to have 2 progress bars showing for this method or disabling the internal PD progress bar? Where is most of the time spent during the computation?

jklaise · 2022-10-04T15:31:05Z

doc/source/overview/algorithms.md

@@ -14,6 +14,8 @@ The following table summarizes the capabilities of the current algorithms:
 |Method|Models|Exp. types|Classification|Regression|Tabular|Text|Image|Cat. data|Train|Dist.|
 |:---|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|:---|:---:|
 |[ALE](../methods/ALE.ipynb)|BB|global|✔|✔|✔| | | |✔| |
+|[PartialDependence](../methods/PartialDependence.ipynb)|BB WB|global|✔|✔|✔| | |✔|✔| |
+|[PartialDependenceVariance](../methods/PartialDependenceVariance.ipynb)|BB WB|global|✔|✔|✔| | |✔|✔| |


Unfortunately with this line the content of the page extends beyond the formatted page:

One fix would be to increase the width of the white part in the css style page. Another I would suggest to shorten the names of the explainers, e.g. PartialDependence -> Partial Dependence and PartialDepencenceVariance -> PD Variance since the names in this table are descriptive, not exact class names.

doc/source/overview/high_level.md

jklaise · 2022-10-04T15:35:25Z

alibi/explainers/pd_variance.py

+     `HistGradientBoostingRegressor`, `DecisionTreeRegressor`, `RandomForestRegressor`."""
+
+    def __init__(self,
+                 predictor: Union[BaseEstimator, Callable[[np.ndarray], np.ndarray]],


I think that this is a good approach now.

alibi/explainers/pd_variance.py

doc/source/examples/pd_variance_regression_friedman.ipynb

doc/source/methods/PartialDependenceVariance.ipynb

…nation and included it in the plotting functionality.

… importance and interaction.

…ually computed importance using PD.

jklaise · 2022-10-11T14:00:13Z

doc/source/methods/PartialDependenceVariance.ipynb

@@ -0,0 +1,650 @@
+{


the the -> the (can you search the doc as I think there's a few places this happens)

Reply via ReviewNB

jklaise · 2022-10-11T14:00:13Z

doc/source/methods/PartialDependenceVariance.ipynb

@@ -0,0 +1,650 @@
+{


Backticks for explain arguments still required.

Reply via ReviewNB

jklaise

Nice work! I left a couple of comments for the notebooks, otherwise all looks done.

RobertSamoilescu requested a review from jklaise September 14, 2022 15:57

RobertSamoilescu added 6 commits September 14, 2022 17:19

POC in progress

c24010e

POC PartialDependeceVariance implementation

b592197

Support for some tree-based models. Sorting features by importance in…

a8688c6

… barplot.

Introduced feature importance and feature interaction.

62738c2

PoC functional for feature importance and feature interaction.

bbef380

isort and minor refactoring.

a66af24

RobertSamoilescu force-pushed the pdp-feature-importance branch from b5d271d to a66af24 Compare September 14, 2022 16:21

Fixed default behaviour when no features are passed.

c242e83

RobertSamoilescu force-pushed the pdp-feature-importance branch from 9ecbbbd to c242e83 Compare September 15, 2022 14:26

jklaise suggested changes Sep 16, 2022

View reviewed changes

RobertSamoilescu added 8 commits September 20, 2022 13:04

Refactored plot functionality -- in progress.

63d52d6

Improved visualizations - first iteration

226aaa8

Merge remote-tracking branch 'upstream/master' into pdp-feature-impor…

2645fd5

…tance

Minor doc corrections, flake8 and mypy.

add0a1c

Included Friedman's regression problem notebook.

78b233f

Fixed plots labeling and title.

9b829fa

Example in progress.

f00197a

Finalized writing example.

e41deb6

RobertSamoilescu force-pushed the pdp-feature-importance branch from 57715ef to e41deb6 Compare September 23, 2022 17:03

RobertSamoilescu added 4 commits September 26, 2022 09:37

Merge remote-tracking branch 'upstream/master' into pdp-feature-impor…

4fd171e

…tance

Included links to the example and method. Exposed explainer and plott…

a46509d

…ing functionality. Solved docs errors.

Merge remote-tracking branch 'upstream/master' into pdp-feature-impor…

db1eee6

…tance

Method description -- in progress

7e11875

RobertSamoilescu force-pushed the pdp-feature-importance branch from 0a174ae to 7e11875 Compare September 26, 2022 16:58

RobertSamoilescu added 3 commits September 27, 2022 12:52

Finalized method description draft

139868b

Included tests.

003a359

Updated README.md, algorithms.md, high_level.md

fb937e4

RobertSamoilescu added 2 commits September 27, 2022 17:16

Updated CHANGELOG.md

3d717eb

Reordered global explanations to be first.

3022518

RobertSamoilescu force-pushed the pdp-feature-importance branch from 90701d3 to 3022518 Compare September 28, 2022 09:08

RobertSamoilescu added 3 commits September 28, 2022 10:13

More spacing between bullet points

44dfea4

Revert "More spacing between bullet points"

6c6df80

This reverts commit 44dfea4.

Addressed firts round of comments.

1adcba7

jklaise reviewed Oct 4, 2022

View reviewed changes

jklaise suggested changes Oct 4, 2022

View reviewed changes

jklaise reviewed Oct 4, 2022

View reviewed changes

RobertSamoilescu added 10 commits October 5, 2022 10:35

Minor correction CHANGELOG.md. Removed kind from the pdvariance expla…

e11ce4b

…nation and included it in the plotting functionality.

Minor correction when passing the variable. Included progress bar for…

3e8c6e9

… importance and interaction.

Fixed high_level.md and algorithms.md

dc7ffee

Solved plotting bug -- sorted -> sort

19072f0

Addressed comments for example notebook.

add0025

Addressed comments in the method description.

dcdf480

Included link to the method description in the example notebook.

26c9f7a

Included more tests to compare the PDV feature importance against man…

a7de5e3

…ually computed importance using PD.

Removed progress bar for importance/interaction computation.

f419c7d

Fixes on the method page.

964e9c3

RobertSamoilescu force-pushed the pdp-feature-importance branch from 284886a to 964e9c3 Compare October 5, 2022 16:50

Fix typos in example notebook.

3091662

jklaise reviewed Oct 11, 2022

View reviewed changes

jklaise approved these changes Oct 11, 2022

View reviewed changes

RobertSamoilescu force-pushed the pdp-feature-importance branch from 7730899 to f975340 Compare October 12, 2022 09:53

Added ticks around explain arguments. Fix repeated 'the'.

ca3067b

RobertSamoilescu force-pushed the pdp-feature-importance branch from f975340 to ca3067b Compare October 12, 2022 10:00

RobertSamoilescu merged commit 6a46fa7 into SeldonIO:master Oct 26, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PDP variance feature importance #758

PDP variance feature importance #758

RobertSamoilescu commented Sep 14, 2022

codecov bot commented Sep 14, 2022 •

edited

Loading

review-notebook-app bot commented Sep 15, 2022

jklaise left a comment

jklaise Sep 15, 2022

RobertSamoilescu Sep 29, 2022

jklaise Oct 4, 2022

jklaise Sep 15, 2022

RobertSamoilescu Sep 29, 2022

jklaise Sep 16, 2022

RobertSamoilescu Sep 29, 2022

jklaise Sep 16, 2022

RobertSamoilescu Sep 29, 2022

jklaise Oct 4, 2022

RobertSamoilescu Oct 5, 2022

jklaise Oct 5, 2022

jklaise Sep 16, 2022

RobertSamoilescu Sep 29, 2022

jklaise Oct 4, 2022

RobertSamoilescu Oct 5, 2022

jklaise Oct 4, 2022

jklaise left a comment

jklaise Oct 4, 2022

jklaise Oct 4, 2022

jklaise Oct 4, 2022

jklaise Oct 11, 2022

jklaise Oct 11, 2022

jklaise left a comment

PDP variance feature importance #758

PDP variance feature importance #758

Conversation

RobertSamoilescu commented Sep 14, 2022

codecov bot commented Sep 14, 2022 • edited Loading

Codecov Report

review-notebook-app bot commented Sep 15, 2022

jklaise left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jklaise left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jklaise left a comment

Choose a reason for hiding this comment

codecov bot commented Sep 14, 2022 •

edited

Loading