[RAM] add the must/must_not/should in the alert search strategy

[XavierM]

Summary

We realized that the filter from the search bar is NOT using must, must_not and should in the elastic search query. When we created our search strategy for alerts, we only supported the filter parameter to make sure, we won't by pass our rbac filter on top of the alerts. We think that we can open this different parameters now after few testing.

We created API test integration to validate that we won't broken our RBAC filter with these new params + I did few test in the dev tools like

PUT /my-index
{
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "properties": {
      "pet": { "type": "text" }
    }
  }
}

POST my-index/_doc/
{
  "pet": "cat"
}

POST my-index/_doc/
{
  "pet": "dog"
}

POST my-index/_doc/
{
  "pet": "bird"
}

Then, we did few tests like

### MUST query
POST test/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "pet": [
              "cat"
            ]
          }
        }
      ],
      "must": [
        {
          "terms": {
            "pet": [
              "bird"
            ]
          }
        }
      ]
    }
  }
}

### Response
{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 0,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  }
}

### MUST_NOT query
PUT /test
{
  "settings": {
    "number_of_shards": 1
  },
  "mappings": {
    "properties": {
      "pet": { "type": "text" }
    }
  }
}

POST test/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "pet": [
              "cat"
            ]
          }
        }
      ],
      "must_not": [
        {
          "terms": {
            "pet": [
              "cat"
            ]
          }
        }
      ]
    }
  }
}

### Response
{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 0,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  }
}

### SHOULD query
POST test/_search
{
  "query": {
    "bool": {
      "filter": [
        {
          "terms": {
            "pet": [
              "cat"
            ]
          }
        }
      ],
      "should": [
        {
          "terms": {
            "pet": [
              "bird"
            ]
          }
        }
      ]
    }
  }
}

### Response
{
  "took": 0,
  "timed_out": false,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  },
  "hits": {
    "total": {
      "value": 1,
      "relation": "eq"
    },
    "max_score": 0,
    "hits": [
      {
        "_index": "test",
        "_id": "1",
        "_score": 0,
        "_source": {
          "pet": "cat"
        }
      }
    ]
  }
}

Checklist

• Unit or functional tests were updated or added to match the most common scenarios

[ymao1]

LGTM

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)

[ymao1]

LGTM

[maryam-saeidi]

Thanks for implementing this! 🚀 (Added a suggestion for test)

[maryam-saeidi]

nit: Maybe we can have a test to ensure should or must query is applied correctly and the number of retrieved items is what we expect. (something other than zero.)

[XavierM]

When the must and filter will be together. ES will return no hits!

[XavierM]

here the explanation between the difference attribute ine the query -> https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-bool-query.html

and here a good example taht I found

The should clause of the bool query is for optional search criteria. Let's say you are looking for hotels and would prefer for the hotel to have a swimming pool, but you would also like to see hotels without a swimming pool in the search results, then the query for swimming pool would be in the should clause. Elasticsearch will give the documents that match more should queries a higher score, so those documents will be ranked higher in the search results.

When you only have a should clause (as in your first query), then at least one of the should clauses must match for a document to be considered a hit.

When you combine a should clause with a filter than all should clauses are optional, so even documents that only match the filter will be returned. You can change that behavior by [setting minimum_should_match 573](https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-minimum-should-match.html) on the bool query. You could set that for example to 1 to make it mandatory for at least one should clause to match

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 06904b2

Metrics [docs]

✅ unchanged

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.13

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation