Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add timestamp processor #12699

Merged

Conversation

andrewkroh
Copy link
Member

This processor enables users to set the @timestamp value based on fields in their events. Its configuration is different than the date processors of Logstash and Ingest Node so the name is different to help distinguish them. The date formats are called "layouts" in this processor.

The processor is included in the log processing Beats (Filebeat, Journalbeat, and Winlogbeat) only. It bundles the Go 1.12 zoneinfo data with the processor (via the 4d64.com/tz package) in order to allow the processor to work consistently in all environments (see golang/go#21881 more details).

This processor enables users to set the @timestamp value based on fields in their events. Its configuration is different than the date processors of Logstash and Ingest Node so the name is different to help distinguish them. The date formats are called "layouts" in this processor.

The processor is included in the log processing Beats (Filebeat, Journalbeat, and Winlogbeat) only. It bundles the Go 1.12 zoneinfo data with the processor (via the 4d64.com/tz package) in order to allow the processor to work consistently in all environments (see golang/go#21881 more details).
@andrewkroh andrewkroh force-pushed the feature/libbeat/timestamp-processor branch from 3037806 to 527c308 Compare July 2, 2019 03:46
@elasticmachine
Copy link
Collaborator

Pinging @elastic/secops

Copy link
Contributor

@adriansr adriansr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @andrewkroh, a much needed processor!

if (p.IgnoreMissing || p.IgnoreFailure) && errors.Cause(err) == common.ErrKeyNotFound {
return event, nil
}
return event, errors.Wrapf(err, "failed to get time field %v", p.Field)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is returning an error when ignore_failure is set but the cause is not a ErrKeyNotFound, is that what you want?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're right 👍 , I got the grouping wrong here. It's fixed now.

@andrewkroh andrewkroh merged commit f29af84 into elastic:master Jul 2, 2019
DStape pushed a commit to DStape/beats that referenced this pull request Aug 20, 2019
This processor enables users to set the @timestamp value based on fields in their events. Its configuration is different than the date processors of Logstash and Ingest Node so the name is different to help distinguish them. The date formats are called "layouts" in this processor.

The processor is included in the log processing Beats (Filebeat, Journalbeat, and Winlogbeat) only. It bundles the Go 1.12 zoneinfo data with the processor (via the 4d64.com/tz package) in order to allow the processor to work consistently in all environments (see golang/go#21881 more details).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants