diff --git a/docs/contributor/arch/002-fluent-bit-without-file-system-buffer.md b/docs/contributor/arch/002-fluent-bit-without-file-system-buffer.md index b2fe42263..5541a6a99 100644 --- a/docs/contributor/arch/002-fluent-bit-without-file-system-buffer.md +++ b/docs/contributor/arch/002-fluent-bit-without-file-system-buffer.md @@ -1,4 +1,4 @@ -# 1. Fluent Bit without File-System Buffer +# 1. Fluent Bit Configuration and File-System Buffer Usage Date: 2023-11-23 @@ -26,7 +26,8 @@ Fluent Bit can be configured in different ways to read container logs and ingest Currently, Telemetry Manager uses option 2. Option 1 does not fulfill our requirement to apply individual filters per pipeline. Option 4 causes an unacceptable resource overhead for our typical setup of two pipelines (application logs and access logs). Option 2 and 3 allow log filter settings per pipeline. -We consider option 3 to be the best Fluent Bit configuration for our requirements because of its lower complexity and lower risk for log loss. The throughput of option 3 has shown to be better than option 2 without changing Fluent Bit's CPU and memory limits. +We consider option 3 to be the best Fluent Bit configuration for our requirements because of its lower complexity. The throughput of option 3 has shown to be better than option 2 without changing Fluent Bit's CPU and memory limits. +The persistent file-system buffer is still considered to be useful over pausing the `tail` input since stored amount of logs per individual Pod is significantly lower with Kubernetes' build-in log storage. Using the directory-size-exporter also provides a better observability of log loss. ## Consequences