Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for JSONL.GZ Log Files in lambda-promtail Module #13129

Open
samuelebstein opened this issue Jun 4, 2024 · 0 comments
Open

Add Support for JSONL.GZ Log Files in lambda-promtail Module #13129

samuelebstein opened this issue Jun 4, 2024 · 0 comments
Labels
lambda-promtail type/feature Something new we should do

Comments

@samuelebstein
Copy link
Contributor

Is your feature request related to a problem? Please describe.
GuardDuty publishes findings as logs with a “.jsonl.gz” suffix, but the lambda-promtail module only allows for “.log.gz” suffixed log files. This limitation means I have to create resources outside of the module to invoke the Lambda function, create the S3 event notifications, and add the relevant get object policy to the Lambda myself.

Describe the solution you'd like
I would like the lambda-promtail module to support different log formats, specifically allowing for “.jsonl.gz” files. This would enable the entire setup, including S3 event notifications and IAM policies, to be handled within the module, simplifying the setup.

Describe alternatives you've considered
I have considered creating additional resources outside of the module to handle “.jsonl.gz” files, but this adds unnecessary complexity -- though it is simple enough. Another alternative is modifying the existing lambda-promtail module locally, but this is not ideal for long-term maintenance and updates.

Additional context
Adding support for multiple log formats within the lambda-promtail module would streamline the integration with services like GuardDuty, which may publish logs in different formats. This enhancement would improve usability and reduce the need for additional configuration outside the module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
lambda-promtail type/feature Something new we should do
Projects
None yet
Development

No branches or pull requests

2 participants