Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support tracking files through log rotation events instead of requiring static extensions in config. #5

Open
abraithwaite opened this issue Jun 10, 2024 · 0 comments

Comments

@abraithwaite
Copy link
Member

Right now, we require a static extension for the file source to prevent the following scenario:

  • logrotate creates a new file with different extension from the original
  • our file source sees the new file and starts consuming it
  • it's already consumed those logs.

Currently we prevent this by assuming that logrotate is appending a different extension onto the files (suchas .1 or .gz).

Ideally, we could watch an entire directory and collect all the logs without having to resort to filtering down to a specific extension.

Also, we may run into issues where logrotation inserts a file modification as .1.log or something like that which is still unhandled and would result in duplicate logs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant