Eagerly check keyword field length

[kkewwei]

If the UTF8 encoding of a keyword field value is bigger than the max length 32766, Lucene fill fail the indexing request and, to roll back the changes, will mark the (possibly partially indexed) document as deleted. This results in deletes, even in an append-only workload, which in turn leads to slower merges, as these will potentially have to fall back to MergeStrategy.DOC instead of MergeStrategy.BULK. To avoid this, we do a preflight check here before indexing the document into Lucene.

Relates #80865

[elasticmachine]

Pinging @elastic/es-search (Team:Search)

[ywelsch]

Thanks for opening this PR. I've left some comments.

[kkewwei]

TBR

[ywelsch]

@elasticmachine update branch
@elasticmachine test this please

[ywelsch]

@elasticmachine test this please

[ywelsch]

I've asked for the comment to be changed, otherwise LGTM

[ywelsch]

@elasticmachine test this please

[ywelsch]

Thank you @kkewwei!

[elasticsearchmachine]

@kkewwei please enable the option "Allow edits and access to secrets by maintainers" on your PR. For more information, see the documentation.

[itizir]

Hello!

Just noticed an issue around this: the exception thrown by KeywordFieldMapper is actually overridden by FieldMapper, so instead of getting the actual reason why indexing failed with a limited (30B) preview of the field value as before, the client gets a generic error message containing the whole field value. Not ideal in logs, and not very helpful as it obscures the root cause of the failure...