Downsample functionality should handle flattened field as labels correctly. #116319
Assignees
Labels
>bug
:StorageEngine/Downsampling
Downsampling (replacement for rollups) - Turn fine-grained time-based data into coarser-grained data
Team:StorageEngine
Comments
martijnvg
added
:StorageEngine/Downsampling
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
kkrik-es
changed the title
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this issue
Dec 17, 2024
If flattened field is configured as non-dimension and non-metric field, then downsampling fails to execute successfully. Downsampling doesn't know how to use the flattened field or how to serialize it. This change addresses this. Closes elastic#116319
martijnvg
changed the title
martijnvg
added a commit
that referenced
this issue
Dec 18, 2024
If flattened field is configured as non-dimension and non-metric field, then downsampling fails to execute successfully. Downsampling doesn't know how to use the flattened field or how to serialize it. This change addresses this. Closes #116319
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this issue
Dec 18, 2024
If flattened field is configured as non-dimension and non-metric field, then downsampling fails to execute successfully. Downsampling doesn't know how to use the flattened field or how to serialize it. This change addresses this. Closes elastic#116319
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this issue
Dec 18, 2024
If flattened field is configured as non-dimension and non-metric field, then downsampling fails to execute successfully. Downsampling doesn't know how to use the flattened field or how to serialize it. This change addresses this. Closes elastic#116319
elasticsearchmachine
pushed a commit
that referenced
this issue
Dec 18, 2024
If flattened field is configured as non-dimension and non-metric field, then downsampling fails to execute successfully. Downsampling doesn't know how to use the flattened field or how to serialize it. This change addresses this. Closes #116319
elasticsearchmachine
pushed a commit
that referenced
this issue
Dec 18, 2024
If flattened field is configured as non-dimension and non-metric field, then downsampling fails to execute successfully. Downsampling doesn't know how to use the flattened field or how to serialize it. This change addresses this. Closes #116319
rjernst
pushed a commit
to rjernst/elasticsearch
that referenced
this issue
Dec 18, 2024
If flattened field is configured as non-dimension and non-metric field, then downsampling fails to execute successfully. Downsampling doesn't know how to use the flattened field or how to serialize it. This change addresses this. Closes elastic#116319
navarone-feekery
pushed a commit
to navarone-feekery/elasticsearch
that referenced
this issue
Dec 26, 2024
If flattened field is configured as non-dimension and non-metric field, then downsampling fails to execute successfully. Downsampling doesn't know how to use the flattened field or how to serialize it. This change addresses this. Closes elastic#116319
sarog
pushed a commit
to sarog/elasticsearch
that referenced
this issue
Jan 22, 2025
…astic#118935) If flattened field is configured as non-dimension and non-metric field, then downsampling fails to execute successfully. Downsampling doesn't know how to use the flattened field or how to serialize it. This change addresses this. Closes elastic#116319
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In the context of downsampling values from a flattened field as dimensions aren't handle correctly. At downsample time, flattened field values are treated as values. This is incorrect, because when indexing a downsampled document then indexing fails when trying to parse the flattened field. This is because an object is expected.
The downsample feature can handle flattened labels correctly it just needs to serialize it correctly and not treat as normal label. For example histogram fields are serialized correctly with using the
HistogramLastLabelFieldProducer. A special field producer is needed for flattend field too. The field names and values are stored as doc values by default, and the structure can be regenerated. (similar to how synthetic source is implemented for this field type).The text was updated successfully, but these errors were encountered: