Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOR.find: allow to specify per-type search fields #119618

Closed

Conversation

pgayvallet
Copy link
Contributor

@pgayvallet pgayvallet commented Nov 24, 2021

Summary

Fix #130616

When we added a match_phrase_prefix clauses in #82693 to improve the results when searching with a prefix search (search term ending with *), we overlooked that match_phrase_prefix must be performed against a text field. Generating such clause against a keyword field returns an error looking like:

all shards failed: search_phase_execution_exception: [query_shard_exception] Reason: failed to create query: Can only use phrase prefix queries on text fields - not on [synthetics-monitor.name] which is of type [keyword]

The problem with that is the behavior when searching for multiple types at once, as the SOM backend it doing.

Let say, for instance, that we got two types:

{
   name: 'a',
   hidden: false,
   namespaceType: 'single',
   mappings: {
      properties: { title: { type: 'text' } },
   },
   management: {
      importableAndExportable: true,
      defaultSearchField: 'title',
   },
}

and

{
   name: 'b',
   hidden: false,
   namespaceType: 'single',
   mappings: {
      properties: { 
        title: { type: 'keyword' },
        name: { type: 'text' }
      },
   },
   management: {
      importableAndExportable: true,
      defaultSearchField: 'name',
   },
}

When filtering to both those types and searching by query from the SOM UI, the request to SOR.find created here: src/plugins/saved_objects_management/server/routes/find.ts looks like:

client.find({
  search: 'foo*',
  types: ['a', 'b'],
  searchFields: ['title', 'name'],
})

Causing match_phrase_prefix clauses to be created for all tuples:

  • a.title
  • a.name
  • b.title
  • b.name

But because b.title is indexed as keyword, the whole search fail.

This is problematic because that forces all type owners to specify fields that are registered as defaultSearchField by other types to be registered as text and not keyword. Such side effect makes no sense and should be avoided.

We already encountered at least 4 times in the past few months issues because teams added new SO types with fields indexed as keyword that were already defined by existing SO types as their defaultSearchField.

This PR fixes this by allowing to specify per-type searchFields when calling SOC.find.

The previous API call now looks like:

client.find({
  search: 'foo*',
  types: ['a', 'b'],
  searchFields: {
     a: ['title'],
     b: ['name'],
   },
})

Causing match_phrase_prefix clauses to now only be created for:

  • a.title
  • b.name

Note: Because this PR intent is to fix the bug causing the SOM requests to fail because of the described issue, the PR only uses these per-type fields for the match_phrase_prefix clauses of the ES request. The simple_query_string part is still using all fields. The reason is simply that doing so would have caused way more changes in the search_dsl logic, without much upsides.

Checklist

@pgayvallet pgayvallet added Feature:Saved Objects release_note:skip Skip the PR/issue when compiling release notes Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v8.0.0 v8.1.0 and removed v8.0.0 labels Nov 24, 2021
@kibana-ci
Copy link
Collaborator

kibana-ci commented Nov 25, 2021

💔 Build Failed

Failed CI Steps

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@pgayvallet
Copy link
Contributor Author

Old and abandoned - closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Saved Objects release_note:skip Skip the PR/issue when compiling release notes Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v8.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

saved_objects/_find API causes search_phase_execution_exception if a searchField isn't mapped as text
2 participants