Composite aggregation could avoid loading global ordinals entirely #47452
Comments
jimczi
added
>enhancement
help wanted
|
Pinging @elastic/es-analytics-geo (:Analytics/Aggregations) |
rjernst
added
the
Team:Analytics
|
Is there any progress here? |
|
@nik9000 was talking to a user about this and I see you self assigned it a while ago. Is this still something on your list ? |
Sorry, not really. That was a speculative assignment and I ended up wandering off into another direction. Though @not-napoleon might be interested to read this as he's been spending more time with composite lately. |
not-napoleon
added
:Analytics/CompositeAggs
and removed
:Analytics/Aggregations
A composite aggregation on a keyword field requires global ordinals today to ensure fast comparisons between segments. It only needs to keep track of the top N composite buckets, however. Since N is typically small, we can just use the segment ordinal for comparison when collecting inside a segment and remap ordinals when we go to the next segment. Closes #47452
A composite aggregation on a keyword field requires global ordinals today to ensure fast comparisons between segments. It only needs to keep track of the top N composite buckets, however. Since N is typically small, we can just use the segment ordinal for comparison when collecting inside a segment and remap ordinals when we go to the next segment. Closes #47452
|
Reopening, as we still want to do this. We had to revert the original commit as it caused a perf degradation. This is pending benchmarks on composite, so marking as stalled at this time. |
|
closing as not planned. |
Currently a
compositeaggregation on a keyword field will use global ordinals to ensure that comparisons between segments is fast. However thecompositeaggregation keeps track of the top N composite bucket so we should be able to use the segment ordinal when collecting inside a segment and map the top N to the next segment ordinals when we go to the next segment. Since N should be small the overhead of mappings these ordinals should be low and the big advantage is that we wouldn't require global ordinals to be loaded.The text was updated successfully, but these errors were encountered: