Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add mechanism to scan a time series (TS) key, N data points at a time #170

Closed
kmuthukk opened this issue Apr 16, 2018 · 1 comment
Closed
Assignees
Labels
kind/enhancement This is an enhancement of an existing feature

Comments

@kmuthukk
Copy link
Collaborator

kmuthukk commented Apr 16, 2018

YugaByte's Redis API introduces a timeseries (TS) type with operations such as: TSAdd, TSRangeByTime, and TSGet and TSRem.

In particular, TSRangeByTime can be used to retrieve a slice of the timeseries corresponding to the specified range of timestamps. In addition to TSRangeByTime, it'll be useful to add a mechanism to fetch the data points of a timeseries N elements a time.

Given that the natural order in which the data points are stored in DocDB is from highest timestamp to lowest, the proposal is to add a:

TSRevRangeByTime key low_ts high_ts [LIMIT N]

This will return the results in high to low TS order, but up to a max of N elements.

When the response returns fewer than N data points, it signals the end of the scan.

Using the above, it will be possible to scan through the data points of a timeseries key in the following manner:

Say you want tofetch data from highest timestamp to lowest timestamp 100 elements at time, all that an app would have to do would be to start with:

TSRevRangeByTime key -inf +inf LIMIT 100

and then follow it up with subsequent calls:

TSRevRangeByTime key -inf (<lowest_ts_received_from_previous_call> LIMIT 100

where the left-paren "(" signifies an exclusive bound (which is a standard Redis idoim).

(Note: For symmetry, we should eventually also enhance TSRangeByTime with a LIMIT clause; but we could defer that till we actually implement reverse scans in DocDB for our Redis data model.)

@kmuthukk
Copy link
Collaborator Author

Reassigning to you @hectorgcr (as per our offline discussion).

@rkarthik007 rkarthik007 added the kind/enhancement This is an enhancement of an existing feature label Apr 16, 2018
yugabyte-ci pushed a commit that referenced this issue Apr 18, 2018
Summary: #170

Test Plan: New unit test

Reviewers: kannan, rahuldesirazu, pritam.damania

Reviewed By: pritam.damania

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D4639
yugabyte-ci pushed a commit that referenced this issue May 3, 2018
Test Plan: New tests

Reviewers: rahuldesirazu, pritam.damania, mihnea

Reviewed By: mihnea

Subscribers: ybase

Differential Revision: https://phabricator.dev.yugabyte.com/D4663
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement This is an enhancement of an existing feature
Projects
None yet
Development

No branches or pull requests

4 participants