-
Notifications
You must be signed in to change notification settings - Fork 305
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement ECS-Compatibility Mode #924
Comments
It has been recommended to me that we use the generated ECS templates from ECS directly: https://github.com/elastic/ecs/tree/master/generated/elasticsearch From what I can tell, the only real obstacle here is that the generated templates apply to indexes matching the pattern Since this is ECS, and the source of the data is less important than the schema, I would move in favor of doing the following when an ECS Compatibility Mode is enabled:
Alternatively, we can provide a default |
this is starting to make more and more sense (to me) - believe we definitely want to start with the generated ecs template and automate any tweaks as suggested.
confused about the rejection, does that mean we refuse to write to any hard-coded index that does not use the ecs-* prefix (in ecs mode), why not simply warn here as well? is it because ECS is 'strongly' opinionated about the index naming? |
The goal is to prevent people from accidentally getting into sticky situations that are very hard to get out of, which we can do by rejecting a configuration that we know will not work as specified (refusing to start up the plugin). Because Elasticsearch creates indexes and fields upon first encounter from a best-guess using the shape of the data and any present templates, the absence of a matching template when the user expects there to be one can mean that a field is created incorrectly. Elasticsearch will then attempt to coerce subsequent encounters of the field to the originally-guessed type, but that can result in indexing rejections (e.g., first encounter I think it's better to prevent people from getting in these situations in the first place. I'm okay with backing down to a warning (@jsvd had a similar recommendation), but the proposed idea was for the plugin to refuse to start up when configured to manage a template that it would not possibly be able to match -- a situation that can be resolved in one of three ways:
|
This is a stub issue, and needs to be fleshed out with details specific to this plugin.
As a part of the effort to make plugins able to run in an ECS-Compatible manner by default in an upcoming release of Logstash, this plugin needs to either implement an ECS-Compatibility mode or certify that it does not implicitly use fields that conflict with ECS.
geoip
data (see also geoip filter) :The text was updated successfully, but these errors were encountered: