[BUG] Exists query fails if object has "search_as_you_type" field

[apalchys]

Describe the bug

The "exists" query on an object field will fail when a "search_as_you_type" field is nested under that object. See step to reproduce for an example.

Related component

Search

To Reproduce

Queries:

PUT bug_exists
{
  "mappings": {
    "properties": {
      "user": {
        "properties": {
          "name": {
            "type": "keyword",
            "normalizer": "lowercase",
            "fields": {
              "lookup": {
                "type": "search_as_you_type",
                "doc_values": false,
                "max_shingle_size": 3
              }
            }
          }
        }
      }
    }
  }
}

POST bug_exists/_doc
{
  "user": {
    "name": "foo"
  }
}

POST bug_exists/_search
{
  "query": {
    "exists": {
      "field": "user"
    }
  }
}

Executing search query gives the following error:

{
  "error": {
    "root_cause": [
      {
        "type": "unsupported_operation_exception",
        "reason": "unsupported_operation_exception: null"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "bug_exists",
        "node": "vQR3k_EpTqGwm-tWhNHpJA",
        "reason": {
          "type": "query_shard_exception",
          "reason": "failed to create query: null",
          "index": "bug_exists",
          "index_uuid": "2XEpnkKbSL62szsFvLvOtw",
          "caused_by": {
            "type": "unsupported_operation_exception",
            "reason": "unsupported_operation_exception: null"
          }
        }
      }
    ],
    "caused_by": {
      "type": "unsupported_operation_exception",
      "reason": "unsupported_operation_exception: null"
    }
  },
  "status": 400
}

Expected behavior

The following query should not fail and return the document

POST bug_exists/_search
{
  "query": {
    "exists": {
      "field": "user"
    }
  }
}

Additional Details

Host/Environment (please complete the following information):

• AWS OpenSearch 2.11

Additional context
The same bug in ElasticSearch fixed in 8.0
elastic/elasticsearch#64609

[msfroh]

@apalchys -- Are you able to create a failing integration test for this issue? Sounds like it's very consistently reproducible.

[Luci-MG]

@msfroh -- I'm interested in contributing & raise PR to its resolution

[apalchys]

@Luci-MG thank you. let me know if I could help.

@msfroh is it right file to learn how to write integration test for this bug?

https://github.com/opensearch-project/OpenSearch/blob/main/server/src/internalClusterTest/java/org/opensearch/search/query/ExistsIT.java

[msfroh]

@apalchys -- That's one place where you can add an integration test. You can also add a case to one of the REST yaml tests: https://github.com/opensearch-project/OpenSearch/blob/main/rest-api-spec/src/main/resources/rest-api-spec/test/search/160_exists_query.yml.

I personally find the yaml tests easier to write, since you just need to specify a series of REST requests, but either way works.

@Luci-MG -- I've assigned the issue to you. Let me know if you need any help.

Thanks!

[Luci-MG]

@msfroh Raised PR for the FIX please checkout and let me know if this needs any changes, Thank you :)

[sandeshkr419]

Closing the issue as the fix is merged #12048 #13006

[dblock]

Would be awesome if someone checked https://github.com/opensearch-project/opensearch-api-specification for whether it describes search_as_you_type properly and added an example (we plan to use those in docs).