Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: Update Fluent Bit Config ADR #624

Merged
merged 2 commits into from
Dec 11, 2023
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# 1. Fluent Bit without File-System Buffer
# 1. Fluent Bit Configuration and File-System Buffer Usage

Date: 2023-11-23

Expand Down Expand Up @@ -26,7 +26,8 @@ Fluent Bit can be configured in different ways to read container logs and ingest
Currently, Telemetry Manager uses option 2.
Option 1 does not fulfill our requirement to apply individual filters per pipeline. Option 4 causes an unacceptable resource overhead for our typical setup of two pipelines (application logs and access logs). Option 2 and 3 allow log filter settings per pipeline.

We consider option 3 to be the best Fluent Bit configuration for our requirements because of its lower complexity and lower risk for log loss. The throughput of option 3 has shown to be better than option 2 without changing Fluent Bit's CPU and memory limits.
We consider option 3 to be the best Fluent Bit configuration for our requirements because of its lower complexity. The throughput of option 3 has shown to be better than option 2 without changing Fluent Bit's CPU and memory limits.
The persistent file-system buffer is still considered to be useful over pausing the `tail` input since the stored amount of logs per individual Pod is significantly lower with Kubernetes' build-in log storage. The directory-size-exporter also provides a better observability of log loss.
chrkl marked this conversation as resolved.
Show resolved Hide resolved

## Consequences

Expand Down