Skip to content
This repository has been archived by the owner on May 16, 2023. It is now read-only.

[Filebeat] Support Kubernetes Deployment (not only Daemonset) #821

Closed
operatorequals opened this issue Sep 24, 2020 · 0 comments · Fixed by #964
Closed

[Filebeat] Support Kubernetes Deployment (not only Daemonset) #821

operatorequals opened this issue Sep 24, 2020 · 0 comments · Fixed by #964

Comments

@operatorequals
Copy link
Contributor

Describe the feature:

Filebeat can be seen as a log collector not only for underline Kubernetes Node resources, but also able to collect logs through the network, AWS, etc.

By assigning Filebeat only to Daemonsets though the available Helm Chart, one cannot use it as an independent and scalable Deployment, which has some use cases as well.

Specifically, a deploymentType: [Daemonset|Deployment] YAML value (defaulting with Daemonset for compatibility), could be used to determine the type of deployment.
Then all Deployment specific values (such as .spec.replicas) can be selected by deployment.spec.replicas YAML directive.

Describe a specific use case for the feature:

Filebeat can be used as a Deployment when it is used to collect AWS logs, with the AWS module.

The issue I came across is the need for scalability for collection of Cloudtrail logs of a very heavily used AWS account, and the Filebeat Daemonset (in my 3 worker cluster) is always lagging behind as it cannot process logs as fast as they are produced.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
2 participants