Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tail -f like behavior for input file interfaces #15

Open
bakerkj opened this issue Nov 18, 2016 · 7 comments
Open

tail -f like behavior for input file interfaces #15

bakerkj opened this issue Nov 18, 2016 · 7 comments

Comments

@bakerkj
Copy link

bakerkj commented Nov 18, 2016

Add tail -f like behavior to input file interfaces.

@stripydog
Copy link
Owner

Do you want to clone the "tail" branch? Hard week but I got some coding in on the train before the gin took effect. Needs cleaning up but should work like this: You need to specify persist=yes but with a regular filename (persist previously only worked with FIFOs). persist=fromstart will keep trying to open a file which isn't there when kplex starts up. Once opened, persist=yes or persist=fromstart should have tail-like behaviour. When EOF is reached kplex will close the file and re-stat it every milliseconds (specified with retry=, default 1 but might change that to 10: opinion welcomed). If the file has changed sized it will seek to the previous location and start reading from there. Currently if the file is smaller it will bomb out. Need to think about further checks there. Good idea on this: might be simpler for people than using FIFOs. This isn't really clean yet: haven't thought hard about file size overflow etc.

@bakerkj
Copy link
Author

bakerkj commented Nov 19, 2016

I'll have a look. Thanks for the quick implementation!

@stripydog
Copy link
Owner

stripydog commented Nov 21, 2016 via email

@bakerkj
Copy link
Author

bakerkj commented Nov 21, 2016

I haven't had a chance to look at the new functionality yet - I hope to shortly.

When kplex starts up, does it first seek to the end of the file, and then start pushing the data out? Or does it start at the beginning of the file, send it all, and then wait for more data to arrive?

In the case of the file being smaller.... Maybe it should start from the beginning? As if the file it is reading from is a log file that's been truncated? I'm not sure...

@stripydog
Copy link
Owner

It starts at the beginning, sends it all, closes the file, then periodically stats the file. If it grows it opens the file again, seeks to the point it left off and reads from there. If the file shrinks it currently logs an error message and exits but as you say maybe that's not the right thing to do and it should start again from the beginning

@bakerkj
Copy link
Author

bakerkj commented Nov 21, 2016

Do you think you could add an option that would have kplex on startup, seek to the end of the file, and then periodically stat the file from there on?

@stripydog
Copy link
Owner

I'll look into it later this week

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants