Enforce synthetic source for time series indices #93380
Conversation
…source is always enabled for tsdb index
This field can be used as dimension and metric in tsdb and now that synthetic source is always enabled, it needs to support synthetic source too.
First validate index mode than source mode.
| @@ -148,94 +148,3 @@ clone: | |||
|
|
|||
| - match: {hits.total.value: 1} | |||
| - match: {hits.hits.0.fields._tsid: [{k8s.pod.uid: 947e4ced-1786-4e53-9e0c-5c447e959507, metricset: pod}]} | |||
|
|
|||
| --- | |||
| clone no source index: | |||
There was a problem hiding this comment.
Tsdb indices only support source mode synthetic now.
| @@ -93,13 +93,12 @@ public void validate(IndexSettings settings, boolean checkLimits) { | |||
| } | |||
| } | |||
|
|
|||
| settings.getMode().validateMapping(mappingLookup); | |||
There was a problem hiding this comment.
Validate index more before loading/validating source mode/loader.
The nested fields yaml tests validate the time series index mode doesn't support nested field type.
But without this the error returned is that synthetic source doesn't support nested field type.
The test can be adjusted, but logically it makes more sense to me that index mode reports the error than source mode, so that is why I moved settings.getMode().validateMapping(mappingLookup); before sourceMapper().newSourceLoader(mapping());.
|
|
||
| @Override | ||
| public SourceLoader.SyntheticFieldLoader syntheticFieldLoader() { | ||
| if (hasDocValues == false) { |
There was a problem hiding this comment.
The unsigned_long field type can be used a dimension / metric in tsdb indices.
Adding synthetic source support for this, since that is the only supported source mode for tsdb indices.
There was a problem hiding this comment.
👍 I imagine there are a bunch of tests that need poking for this.
| return DEFAULT; | ||
| return indexMode == IndexMode.TIME_SERIES ? TSDB_DEFAULT : DEFAULT; | ||
| } | ||
| if (indexMode == IndexMode.TIME_SERIES) { |
There was a problem hiding this comment.
I'd prefer to have a method on IndexMode that does this stuff if possible.
There was a problem hiding this comment.
Turns out that the usage of include and exclude in combination with synthetic source is already not allowed by validation in SourceFieldMapper's constructor.
I'll just validate that here that synthetic source is enabled.
|
|
||
| @Override | ||
| public SourceLoader.SyntheticFieldLoader syntheticFieldLoader() { | ||
| if (hasDocValues == false) { |
There was a problem hiding this comment.
👍 I imagine there are a bunch of tests that need poking for this.
elasticsearchmachine
added
the
Team:Analytics
|
Pinging @elastic/es-analytics-geo (Team:Analytics) |
Support for synthetic source is also added to `unsigned_long` field as part of this change. This is required because `unsigned_long` field types can be used in tsdb indices and this change would prohibit the usage of these field type otherwise. Closes elastic#92319
Synthetic source is the only allowed source mode with tsdb.
Closes #92319