Statistics sensor not counting 0

[tungmeister]

Home assistant: 84.6

I've got a statistics sensor setup as below:

sensor:
  - platform: statistics
    entity_id: binary_sensor.bedside_lamp_toggle
    name: toggle
    max_age:
      seconds: 2

I would expect this to show the count for the binary sensor from the last 2 seconds and then return to 0 if the sensor doesn't change in the next 2 seconds. Rather than this the count always shows the last recorded value e.g. 1 or 2 and will only change when new values are recorded rather than the absence of values.

[mickdekkers]

I think this also affects statistics for non-binary sensors. I've got a DHT temperature sensor hooked up and I'd like to display some stats for it, mainly the change in temperature over the past 5 minutes.

  - platform: dht
    sensor: AM2302
    pin: 23
    monitored_conditions:
      - temperature
  - platform: statistics
    name: Temperature Stats
    entity_id: sensor.dht_sensor_temperature
    sampling_size: 12
    max_age:
      minutes: 5

The temperature sensor itself updates every 30 seconds, but the temperature readings don't change very often. When I look at the statistics sensor, it only ever collects 1-2 samples in the 5 minute window (count in the screenshot below).

This prevents the statistics sensor from calculating variance, stdev, change, average_change, and change_rate reliably, since these all need at least 2 samples.

Perhaps some kind of sample_interval/sample_rate option for the statistics sensor could be a solution here. Something that would force the statistics sensor to sample and record the monitored sensor output regardless of whether it changed.

[alex-gh]

Is this still an issue you are experiencing? Can you please try upgrading to the latest version of Home Assistant (0.90) and report back if this is still a problem? Thanks!

[mickdekkers]

@alex-gh I can confirm this is still an issue on 0.90.1

[rikroe]

I think I found the reason for this however I am not directly sure how to solve it. I have a setup with a MQTT switch and sensor where the sensor shows power consumption and only get updated if there is a change. So if I turn off the switch, power consumption goes to 0.

In this moment, the statistics component calculates and the value lowers. But as there are no state change events afterwards (0 stays 0 after all), the component does not get triggered and therefore doesn't update.

So when using max_date, the sensor needs an additional time-based listener (or similar) that automatically refreshes at a certain interval. Not sure if this is even possible though or if there is a workaround e.g. using an automation (as sensors don't have services that could be triggered)

[tungmeister]

Any developments on this, I hate the clunky way I've got it implemented at the moment.

[tungmeister]

Just bumping this again. Is this the intended behaviour or a bug?

[lwestenberg]

also having this issue with the binary sensor. I use a binary sensor for my water meter (with line tracking sensor) and the statistics sensor keeps a count with max_age 1 min so I can see the used liter/min.

In the current situation the sensor does not reset to 0 but it will stay on the last value. So e.g. when I use 10 liter and nothing after that, then it will stay on 10 as if I was using 10 liter/min all the time.

[exxamalte]

I thought I'd give it a try to fix this by adding scheduled updates if max_age is set.

However, I've run into the following issue: If there are no state changes anymore, the scheduled update after max_age has passed would first remove all outdated states which means that very quickly there are no states left, and the calculations for mean/variance/etc. don't work anymore. The statistics sensor's state turns to unknown and so do most of its attributes. Only count makes sense as it turns to 0.

@tungmeister: I guess the above change would work for your particular use-case.

@mickdekkers: I think your use-case is a bit different. If the sensor you are monitoring does not change its state, i.e. the value does not change and the sensor does not force update its value, then I don't know how the statistics sensor can help you here. The statistics sensor listens for state changes recorded in the the HA state machine, but if the monitored sensor does not record a change, then there is not much we can do I'm afraid.

@rikroe: I'm unsure of the above scheduled update would help in your case. If you are monitoring a sensor that captures power consumption, and if at some point this sensor turns to 0, then the scheduled update would eventually turn the statistics sensor to state unknown. What attribute are you interested in that should stay 0?

@Westenberg: I understand you have a binary sensor that if on indicates you are using 10l/min water. Is the issue in your particular case that even when the binary sensor turns off, the statistics sensor does not update?

[tungmeister]

@exxamalte Will this change be in the next release and is there a commit I can try?

[exxamalte]

Will this change be in the next release and is there a commit I can try?

I've just pushed a branch with that commit: exxamalte@db1ef8f

At the moment I am a bit unsure if the risk of this change ("breaking change" potential) outweighs the benefit. Hence I was asking for feedback.

[tungmeister]

I'll test it later today

[rikroe]

I will test as well, although I'm not sure I'll get it done today.

For me your implementation (the value going to 0 or unkown) should be fine, basically I want to check on sensor_value [W] < 5. Need to check if that works well with unkown though.

[rikroe]

Tested, it works now for my use case. After the defined max_time, the sensor is updated and switched to unkown. This is something I can listen on, e.g. with a template sensor. When comparing state(sensor.stats) | int > 1, unkown is regarded to be smaller than 1.

I just wondered if it is also possible to define some refresh interval (e.g. in max_time = 10, refresh every 1 minute). Right now, the value only changes when max_time is reached.

Nevertheless, thanks for your enhancement @exxamalte 🎉 It makes this sensor much better!

[exxamalte]

Thanks for the feedback, @rikroe.
I see what you mean. Instead of performing one update after max_age, we want to update the statistics sensor the next time an included value should be purged. Let me revisit the approach.

[exxamalte]

@rikroe: I have made one more change to this: Instead of waiting the whole max_age period for the next update of the statistics sensor, I have changed the code to find the timestamp when the next purge should occur.

If you get a chance, could you please try the latest changes of the statistics sensor?
https://github.com/exxamalte/home-assistant/blob/statistics-max-age/homeassistant/components/statistics/sensor.py

[rikroe]

Looks really good now, thank you so much! Would love to see it integrated!

(Universal 2 being the actual power plug measure consumption & Stats the statistics sensor with your updated code)

[lwestenberg]

@exxamalte sorry for the late response. Thanks, your solution fixed my issue! Looking forward to the update!