[Filebeat] Fix checkpoint #21344
Merged
[Filebeat] Fix checkpoint #21344
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…out too much compiled scrips
Sync branch with master
botelastic
bot
added
the
needs_team
botelastic
bot
removed
the
needs_team
|
Pinging @elastic/siem (Team:SIEM) |
botelastic
bot
removed
the
needs_team
💔 Build Failed
Expand to view the summary
Build stats
Test stats 🧪
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
marc-gr
added a commit
to marc-gr/beats
that referenced
this pull request
Sep 29, 2020
* fix checkpoint @timestamp already exists * Fix checkpoint @timestamp already existing, test logs * testfile * Updated CHANGELOG * fix ASCIIDOC * remove timestamp renaming since it produced strange error messages about too much compiled scrips * reenable tiemestamp rename * move comment in asciidoc to bugfixes * move comment in asciidoc to bugfixes * fix asciidoc * fix asciidoc * fix tes log * remove empty lines * Fix pipeline and re generate test files * Reorder changelog entry Co-authored-by: bernhard.fluehmann <bernhard.fluehmann@realstuff.ch> (cherry picked from commit 452c6d2)
3 tasks
v1v
added a commit
to v1v/beats
that referenced
this pull request
Sep 29, 2020
* upstream/master: feat: prepare release pipelines (elastic#21238) Add IP validation to Security module (elastic#21325) Fixes for new 7.10 rsa2elk datasets (elastic#21240) o365input: Restart after fatal error (elastic#21258) Fix panic in cgroups monitoring (elastic#21355) Handle multiple upstreams in ingress-controller (elastic#21215) [CI] Fix runbld when workspace does not exist (elastic#21350) [Filebeat] Fix checkpoint (elastic#21344) [CI] Archive build reasons (elastic#21347) Add dashboard for pubsub metricset in googlecloud module (elastic#21326) [Elastic Agent] Allow embedding of certificate (elastic#21179) Adds a default for failure_cache.min_ttl (elastic#21085) [libbeat] Disk queue implementation (elastic#21176)
marc-gr
added a commit
that referenced
this pull request
Sep 30, 2020
* fix checkpoint @timestamp already exists * Fix checkpoint @timestamp already existing, test logs * testfile * Updated CHANGELOG * fix ASCIIDOC * remove timestamp renaming since it produced strange error messages about too much compiled scrips * reenable tiemestamp rename * move comment in asciidoc to bugfixes * move comment in asciidoc to bugfixes * fix asciidoc * fix asciidoc * fix tes log * remove empty lines * Fix pipeline and re generate test files * Reorder changelog entry Co-authored-by: bernhard.fluehmann <bernhard.fluehmann@realstuff.ch> (cherry picked from commit 452c6d2)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Re-opening of #20567, thanks @Bernhard-Fluehmann for the original work!
Original PR message:
What does this PR do?
Resolves conflicting set of timestamp field
Why is it important?
Some Checkpoint logs contain a time field and the pipeline contained a processor for it. As far as I can see the current implementation causes two errors. 1st the presence of a time field breaks the pipeline since the logs received from filebeat contain a timestamp already. 2nd problem is that the format of the time field is unix time and thus needs to be converted into a iso timestamp first.
The reason why the problem was not detected by the testsuite seems to be that none of the logs of the checkpoint.log file contains the time field. I have added such a log line at the end now.
Checklist
- [ ] I have made corresponding changes to the documentation- [ ] I have made corresponding change to the default configuration files- [ ] I have added tests that prove my fix is effective or that my feature worksCHANGELOG.next.asciidocorCHANGELOG-developer.next.asciidoc.Related issues