Fix highlighting apis to highlight more than 1 doc at once

[rmuir]

Highlighting apis seem to only handle one document at a time, but this totally breaks PostingsHighlighter design on elasticsearch.

The whole idea behind this highlighter is #queryterms < #summarydocs. But performance totally falls apart because today documents are highlighted one at a time.

[javanna]

+1 it might be tricky to move the postings hl to highlight multiple documents at a time while maintaining the per document usecase needed with plain and fvh, but it's worth investing in making the postings highlighter work as it should. I will look into this.

[nik9000]

Cool! That'll be useful!

The experimental_highlighter will actually need to decide which order to do
them in based on the mapping definition so I think the best thing to do
is to push that decision to the highlighters - like push the whole loop
over the documents into them.

For the postings highlighter I think the optimal ordering is segments,
fields, then docs, right? Or is it just fields then docs?

On Wed, May 20, 2015 at 4:20 AM, Luca Cavanna notifications@github.com
wrote:

+1 it might be tricky to move the postings hl to highlight multiple
documents at a time while maintaining the per document usecase needed with
plain and fvh, but it's worth investing in making the postings highlighter
work as it should. I will look into this.

—
Reply to this email directly or view it on GitHub
#11223 (comment)
.

[rmuir]

postingshighlighter in lucene takes your ordinary TopDocs and does any internal resorting itself. You should also give it all the fields to highlight at once too. It knows how to do the right thing.

[jimczi]

The postings highlighter has been removed in 6 but the unified highlighter could also benefit from this. For complex query this re-design could speed up things not only because we highlight one document at a time but also because we analyze the query for each highlighted field in each document.
cc @elastic/es-search-aggs

[javanna]

This has been open for quite a while, and we were not able to prioritize it so far. For now I'm going to close this as something we aren't planning on implementing. We can re-open it later if needed.