-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Configuration for shard iterator AT_TIMESTAMP #19
Comments
Nope, it's just never come up so far. Right now it's hard-coded to TRIM_HORIZON: https://github.com/logstash-plugins/logstash-input-kinesis/blob/master/lib/logstash/inputs/kinesis.rb#L73 Making this configurable is certainly reasonable though. |
I second this request. We have a number of streams with 7 day retention, so as we're setting up new logstash stream processors, each one has to chew through a week worth of data that we've already processed with our existing system. It would be great to be able to tell it to just start processing from now and ignore the existing data. |
We have the same use case @tschran so would love to see this feature. |
hey @codekitchen I took a stab at adding support for LATEST here. I have never written ruby code in my life, so I'm sure it will need a little revision. |
+1 |
|
Hello, |
KCL provides an option to specify the initial position in the stream from where the application starts reading from. It can be one of LATEST, TRIM_HORIZON or AT_TIMESTAMP. I do not see a configuration option currently available in the logstash configuration to specify the same. Is there anything I am missing here?
The text was updated successfully, but these errors were encountered: