API: delete data

[ln-komandur]

To hide / filter plotted data, I am of the understanding that the "Threshold" feature in the UI works on values from 'dimension3' made with JSON calls. And the 'dimension1' is the name of the data element being plotted. Please correct me if I am wrong here.

In my application, I have data coming in from 3 sensors, thrice every second (i.e. 3 orientations of accelerometer, 3 orientations of magnetometer, 1 thermometer). I am trying to send them to 3 or 4 different graphs using Rest API calls to plot in near real time. I don't have the need for old data in my graphs in different ways and I want to delete them by sending a suitable JSON object in my Rest API call. Some examples are as below

1. Clean up all data in a graph every time the application starts by sending a JSON object in the Rest API like below

{'erase': "all"}

This should delete plots for all dimensions in that graph.

2. Clear data pertaining to one of the dimensions by giving the name of 'dimension1' before a certain number of data points or after a certain number of data points or both, going by the order in which they were plotted

{'erase' : "range", dimension1 : "X axis", 'before' : "number of data points before which to delete", 'after' : "number of data points after which to delete", 'range' : "inside" or "outside"}

If passed 'range' : "inside", (or say "within"), then data points between the "after" and "before" should be deleted
If passed 'range' : "outside", (or say "without", or "keep"), then data points other than the ones between the "after" and "before" should be deleted

For e.g (example A), calling
{'erase' : "range", dimension1 : "X axis", 'before' : "200", 'range' : "outside"}
should delete all data points before the last 200 data points. This will help keep the graphs to current interest (last 200 in this example) only.

3. Clear all data plotted for one of the dimensions

{'erase' : "dimension", dimension1 : "X axis",'range' : "all"}

This should delete the entire set of data points plotted for the name given in 'dimension1'

4. Clear just one data point whose values for all 3 dimensions are given

{'erase' : "point", 'dimension1': 'X axis', 'dimension2': aTime, 'dimension3': 1}

5. Keep only the last few data points at all times without making repeated calls to erase. For e.g. Calling

{'erase' : "keep", dimension1 : "X axis", 'before' : "200"}
should delete all data points before the last 200 data points without making repeated calls like in example A above. They both do the same thing however, example A does it by repeated calling, whereas "keep" is a one time call. Since this is a one time call, it should be possible to reset it also by passing
{'erase' : "reset", dimension1 : "X axis"}

Please advise the functionality is possible in the current implementation. Also please point to the documentation.

[Rello]

Hello,
thank you for the request.
Currently this is not possible yet as the API is "add only". But I am looking at it already as this request came up more often now.
I will not be able to facilitate all requested conditions at once. there will be some iterations I guess because its complex topic when making it dynamic. Other requests are e.g. to delete with a dynamic time-range (>2weeks ago) a little similar to your >200 records

What is your thought on the way of implementing this: via a separate REST endpoint just for deletion or to attach deletion filters to the same dataset when sending it? I think a separate endpoint would be the way to go...

[ln-komandur]

Thanks for your response. Any ability to erase/remove/delete to begin with should be fine. Incremental feature enhancement works for me.

On your other question, I was thinking of

attach deletion filters to the same dataset when sending it

as it would be easy to package and send them all in one. This is especially useful if the array feature I requested earlier is implemented too

However, if

separate endpoint would be the way to go

is easier for first cut implementation, I think it should serve the purpose too

[Rello]

yes.
I am working on the array-request already.
I will add this one as a new api endpoint. this will make it easier to extend it step by step at the beginning

[Rello]

Hello @ln-komandur
I implemented a first quick-win. the deletedata endpoint is using the same backend as the deletion in the UI where values or wildcards are allowed to remove any data from the tables.
Does this work for you partly already?

https://github.com/Rello/analytics/wiki/API#removedata

[ln-komandur]

Thank you. I am yet to check if this is works for me partly already. In the meantime, from the wiki, I wasn't sure if we can also delete by using the other data set names like Column 1 etc. like below.

{
    "delete":[
        {
            "dimension1": "x", 
            "dimension2": "y"
        },
        {
            "Column 1": "x", 
            "Free Text": "y"
        }
    ]
}

[Rello]

Hello @ln-komandur
yes, you can delete both ways. I updated the wiki.
thank you for the remark