Discover - Unable to add any new filters after creating 'bool' filter

[stevenbaker]

Kibana version: 4.5.3
Elasticsearch version: 2.3.4
Server OS version: Ubuntu 14.04.4
Browser version: Chrome 51.0.2704.103 + Safari 9.1.2
Browser OS version: OS X El Capitan 10.11.6

Original install method (e.g. download page, yum, from source, etc.):

• "apt-get install" via packages.elastic.co

Description of the problem including expected versus actual behavior:

Expected Behavior:

• In the Discover tab, after manually editing a filter, if that filter is accepted by Kibana (the JSON parses fine, filter functions normally on the events), then I should be able to add another filter.

Actual Behavior:

• In the Discover tab, a filter that is accepted by Kibana and which functions normally can still be crafted in a way that disables subsequent addition of filters of ANY kind.

Details of our use case:

In the Discover tab, after adding a filter that uses a "bool" query, a user is not able to add any more filters. Clicking on the "+" (search-plus) or "-" (search-minus) icons does nothing.

In the console, JavaScript exceptions are thrown on each of those clicks. But nothing is reported to the screen to let the user know why their click appeared to be ignored. No filters, of any kind, on any fields, can be added.

We are using a "bool" + "should" filter as a way to remove common crawlers and bots from a set of nginx log events. Rather than having one filter for each type of bot (which could get numerous and cumbersome, lots of filters strewn out horizontally), we instead desired to create a single bot filter as such:

{  
  "query":{  
    "bool":{  
      "should":[  
        { "match": { "agent":"ELB-HealthChecker" } },
        { "match": { "agent":"Slackbot-LinkExpanding" } },
        { "match": { "agent":"Pingdom" } },
        { "match": { "request":"robots.txt" } }
      ]
    }
  }
}

The filter itself works successfully. Events that match any of the searches in the bool clause are properly filtered. It's just that new filters after this one cannot be added through the UI due to a bug.

Steps to reproduce:

1. On the Discover tab, create a filter using a "bool" query.
2. Try to add another filter on any field by clicking on the "+" or "-" magnifying glass icons on that field.

Errors in browser console (if relevant):

commons.bundle.js?v=9910:40117 TypeError: Cannot read property 'httphost' of undefined
    at kibana.bundle.js?v=9910:92503
    at baseFindIndex (commons.bundle.js?v=9910:5797)
    at Function.<anonymous> (commons.bundle.js?v=9910:8712)
    at kibana.bundle.js?v=9910:92495
    at arrayEach (commons.bundle.js?v=9910:6760)
    at Function.<anonymous> (commons.bundle.js?v=9910:8816)
    at Object.filterManager.add (kibana.bundle.js?v=9910:92493)
    at Scope.$scope.filterQuery [as updateFilterInQuery] (kibana.bundle.js?v=9910:22553)
    at fn (eval at compile (commons.bundle.js?v=9910:40962), <anonymous>:4:529)
    at callback (commons.bundle.js?v=9910:51189)

Tracking this down, the issue appears to be in filterManager.add(), at this spot when the function is attempting to see if an existing filter is in place:

 if (filter.query) {
    return filter.query.match[fieldName] && filter.query.match[fieldName].query === value;
}

Screenshots below.

Setting a breakpoint in the debugger, I was able to see that:

• filter is defined
• filter.query is defined
• filter.query.match is NOT defined

That would make sense, since the existing filter we had already specified was a bool filter, NOT a match filter.

The code right now doesn't appear to expect that.

And sincefilter.query.match is undefined ... filter.query.match[fieldName] has no chance.

It results in a TypeError of:

Cannot read property '[fieldName]' of undefined

At this point, the "+" and "-" buttons on each field don't do anything except throw errors in the console. The user is stuck with the filters they've got.

PROPOSAL

One of the following two would work:

1. Kibana should ONLY allow "match" filters to be created, since that appears to be the assumption causing this bug in the first place.
2. Kibana can continue to allow other filters to be created and then manually edited, but then the logic in filterManager.add() that checks for existing filters should recognize that the user may end up with something in there that isn't always a "match" query. :-)

For the short term, I think the quickest fix here would probably be to just change this:

if (filter.query) {
    return filter.query.match[fieldName] && filter.query.match[fieldName].query === value;
}

To this:

if (filter.query) && (filter.query.match) {
    return filter.query.match[fieldName] && filter.query.match[fieldName].query === value;
}

That way, when interrogating an existing filter that wasn't a "match", you would just evaluate to false. The return line wouldn't run and you'd be good to go.

Screenshots

[Bargs]

Hi @stevenbaker, thanks for the detailed bug report. We're already tracking this issue at #7246 so I'm going to close this one as a duplicate, but I'll link to it from the original ticket so we don't lose this detail.

I agree the quick fix would be to shore up the conditional, but I also think we should give the user an error if they create an incompatible filter. That said, I don't think it'll be a huge effort to fix this so non-match filters will work but I haven't had time to really dig into it.

[stevenbaker]

@Bargs - Ugh, sorry for missing that previous issue. Yep, that's the same issue we have. Thanks!