-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rate limit Elasticsearch client indexing error logs #40448
Rate limit Elasticsearch client indexing error logs #40448
Conversation
This pull request is now in conflicts. Could you fix it? 🙏
|
2084e8c
to
67949a1
Compare
This pull request is now in conflicts. Could you fix it? 🙏
|
47acb51
to
155122c
Compare
Use `periodic.Doer` to rate limit indexing error logs. This does not affects the logs sent to the event logger.
155122c
to
df532fc
Compare
Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just a suggestion to document why we did this.
observer := s.observer | ||
if observer == nil { | ||
observer = outputs.NewNilObserver() | ||
} | ||
|
||
log := logp.NewLogger("elasticsearch") | ||
|
||
pLogDeadLetter := periodic.NewDoer(10*time.Second, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suggest adding a comment explaining why we are rate limiting these so future readers don't need to guess, and can avoid adding new per-event info logs.
Proposed commit message
Checklist
[ ] I have commented my code, particularly in hard-to-understand areas[ ] I have made corresponding changes to the documentation[ ] I have made corresponding change to the default configuration filesCHANGELOG.next.asciidoc
orCHANGELOG-developer.next.asciidoc
.Disruptive User Impact
None
How to test this PR locally
Start Filebeat with the following configuration
Create the log file
/tmp/flog.log
with the following content:Cannot index event (status=STATUS): dropping event!
for every dropped event, you should seeFailed to index N events in last 10s: events were dropped!
Related issues
Logs