Ensure no field value is null

[otoolep]

Fix issue #2420.

[otoolep]

@dgnorton -- you were last in this code, I think.

[toddboom]

+1, looks like it needs a rebase

[mitar]

Why it is not allowed to insert null values? This is a great way to signal that the value was missing from the sensor?

[gunnaraasen]

@mitar The TSM engine does not handle null values.

In the case of a missing sensor value, I think a separate boolean field, e.g. temp_null, could be used to record whether a signal was missing or not.

[mitar]

Hm, could null values support be added to TSM? ;-)

[crudbetter]

@gunnaraasen hi there - I have a very similar use case to @mitar - a sensor value of 0 is valid, but sensor values could also be missing. Has development progressed at all, or is using a separate boolean field (per field that could have missing values) still the recommended approach? Many thanks in advance

EDIT Have found this useful information https://groups.google.com/forum/#!topic/influxdb/L2_JkanwwOY

[gunnaraasen]

@crudbetter Yup, as mentioned in the linked mailing list thread, using a separate field is still the recommended way to store that a value is null or missing or there was some other error. The list of OPC codes linked in the last post also look really useful for recording sensor errors, although I've not personally used that approach.

[dandv]

When does this PR ensure that no field value is null? You can easily insert nulls via the HTTP API or by importing, when fields are omitted.

[schmurfy]

I am currently trying to evaluate how to migrate from 0.8 to 1.0.2 and I just stumbled on this issue, not having null is really annoying...
Adding a field to signal the data is missing is not the same, here is one of my use case: a sensor report temperature on a regular interval and I want to compute the derivative using kapacitor, as long as the stream is continuous everything will be fine but if two data are missing what I would expect is a cut on the graph (in grafana for example) but instead what I will get is the derivative computed from the last valid point :(

And in the same case if I graph the temperature there also won't be an visible way to tell data were missing, for example a loss of power/reboot.

is there any way to do that ?

Edit: in this use case the sensor is not directly writing to influxdb, an intermediate process is, otherwise there would be no null anyway.

[archont00]

Same for me, I migrated from SQL to influx so that I could use Grafana to chart data from home heating unit within OpenHab. When heating unit pump is off, I'd like to insert null values for heating water temperature and display no line in Grafana (because displaying non-flowing water temperature makes no sense).

[luhn]

I've indicated NULL in my data by omitting the field from the row. It seems to be working well for my purposes. Are there any gotchas with using this method?