Invisible filters when re-indexing.

[ThomasFlanaghan]

Kibana version: 4.5c

Server OS version: -

Browser version: google chrome

Browser OS version: -

Original install method (e.g. download page, yum, from source, etc.): -

Description of the problem including expected versus actual behavior: when reindexing data using the elastic reindex api under a new index name the dashboard using that name applies previous filters without showing them in the green tabs. If a dashboard has filters saved and then you reindex the data using a different index name then change all of the visualisations to the new index name the filter is applied but you cant change it. the only way to solve this is to delete the dashboard and filters and then recreate them then the filters will no longer be applied.

Steps to reproduce:

1. have a simple index called index_test with two fields (field 1 and field 2)
2. create a visualisation and dashboard with this index.
3. apply a filter to only show field_1
   1. save the dashboard with the filter.
   2. reindex the data as a new index called index_test1
   3. delete in the index pattern and add the new one (index_test1)
   4. open the dashboard and you see errors saying it cannot find the old index pattern.
   5. edit the visualisations to use the new index pattern.
   6. the dashboard will now show the data with the filter still applied but you cant delete the filter because its invisible.

I have tried exporting the dasboard and visualisations to see if their is anything filtering but there is nothing. this makes it hard to reindex using a new name as the dashboard will have to be recreated.

[Bargs]

Instead of deleting the old index pattern, have you considered creating an alias with the same name as your old index which points to the new index? I believe this is how our migration tool works, and probably provides a smoother transition in general.

If you just want to remove the filter, you should be able to do so in the "Saved Object" section of the Kibana settings. Pop open the relevant dashboard and edit the kibanaSavedObjectMeta.searchSourceJSON field:

[Hugodby]

It happens when the filter index is different from the ones of the dashboard.

To recreate the issue:
Take any dashboard you want and create a simple filter on it. Copy the dashboard link using the share menu. Edit link->filter section->index by typing not existing index. Open the link on a new tab => filter will have an effect on the visualizations but will not be displayed. Every filter you add next aren't displayed. No error notification are displayed

eg:
For a dashboard displaying content from index foo. Add a filter checking my_string: term1. So the link will contain a part like

_a=(filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:foo,key:my_string,negate:!f,value:term1),query:(match:(my_string:(query:term1,type:phrase))))))

Now change index:foo to index:bar where bar is a index that doesn't exist.

_a=(filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:bar,key:my_string,negate:!f,value:term1),query:(match:(my_string:(query:term1,type:phrase))))))

Load the complete url. The filter will not be displayed but the visualization will be affected. If you add another filter it will not be displayed.
Firefox console display that error:

15:13:09.082 Error: Could not locate that index-pattern (id: bar)
KbnError@http://xxx.xxx:5601/bundles/commons.bundle.js?v=9910:57444:21
SavedObjectNotFound@http://xxx.xxx:5601/bundles/commons.bundle.js?v=9910:57573:6
applyESResp@http://xxx.xxx:5601/bundles/kibana.bundle.js?v=9910:78549:37
processQueue@http://xxx.xxx:5601/bundles/commons.bundle.js?v=9910:42385:29
scheduleProcessQueue/<@http://xxx.xxx:5601/bundles/commons.bundle.js?v=9910:42401:28
$RootScopeProvider/this.$get</Scope.prototype.$eval@http://xxx.xxx:5601/bundles/commons.bundle.js?v=9910:43629:17
$RootScopeProvider/this.$get</Scope.prototype.$digest@http://xxx.xxx:5601/bundles/commons.bundle.js?v=9910:43440:16
$RootScopeProvider/this.$get</Scope.prototype.$apply@http://xxx.xxx:5601/bundles/commons.bundle.js?v=9910:43737:14
done@http://xxx.xxx:5601/bundles/commons.bundle.js?v=9910:38186:37
completeRequest@http://xxx.xxx:5601/bundles/commons.bundle.js?v=9910:38384:8
requestLoaded@http://xxx.xxx:5601/bundles/commons.bundle.js?v=9910:38325:10
1 commons.bundle.js:40117:19
consoleLog/<() commons.bundle.js:40117
$ExceptionHandlerProvider/this.$get</<() commons.bundle.js:36886
processQueue() commons.bundle.js:42393
scheduleProcessQueue/<() commons.bundle.js:42401
$RootScopeProvider/this.$get</Scope.prototype.$eval() commons.bundle.js:43629
$RootScopeProvider/this.$get</Scope.prototype.$digest() commons.bundle.js:43440
$RootScopeProvider/this.$get</Scope.prototype.$apply() commons.bundle.js:43737
done() commons.bundle.js:38186
completeRequest() commons.bundle.js:38384
requestLoaded() commons.bundle.js:38325

[Bargs]

@Hugodby so would an acceptable solution be to display a warning in the UI, possibly with an option to remove the incorrect filters?

[Hugodby]

It depends if you judge a correct behavior masking the incorrect filter(s) but still applying them.
To my opinion it should:

• either delete the filter in a transparent way for the user, incorrect filter so let's remove it it will not be useful
• or warn the user and NOT apply the incorrect filter(s) on the visualization

[Bargs]

Ah right, I was forgetting that the filter still takes effect despite pointing at an incorrect index pattern.

I that case I'd lean towards option two: warn the user, give them a way to delete the filter if desired, don't apply the filter in its current state. This way the user will be aware of the issue, can take action to remedy it, but we won't be modifying their saved objects without their knowledge.