System monitor Network sensors number-wrap

[Tommatheussen]

Home Assistant release (hass --version):
0.35.3

Python release (python3 --version):
3.4.3

Component/platform:
System Monitor

Description of problem:
The system monitor network interfaces do not take in account wrapping around the maximum number.
After the amount gets too high, the number just wraps around to 0.

Expected:
The number should not wrap around to 0, unless the interface has actually been set to 0 again.

Problem-relevant configuration.yaml entries and steps to reproduce:

- platform: systemmonitor
    resources:
      - type: network_in
        arg: eth0

1. Let the number accumulate high enough

Additional info:
Reading in the psutil lib docs, there's stated here: https://pythonhosted.org/psutil/#network
that applications should be prepared for this.
I got a very old system, and it already wraps around after 4096MiB.
You can see this on the following screenshots:

While in my /proc/net/dev (where psutil gets the info from) these numbers can be seen:

[fanaticDavid]

I've noticed this behaviour too, for both network_in and network_out, but silly me never thought of looking at the graph to see what the cut-off point was. I ended up removing both resources from my config as I couldn't make sense of the values (and it's pointless anyway when it resets every now and then).

I take it the values should only reset upon rebooting the host HASS is running on?

[Tommatheussen]

As far as I could find, the values are indeed reset upon restarting the host.

Knowing this, it would be more useful for me to get monthly stats, so I should check wether I can reset those manually or find another solution :p

[fanaticDavid]

I just happen to do some research and it seems this is caused by a known issue in psutil. The documentation contains the following warning about the network monitor:

on some systems such as Linux, on a very busy or long-lived system these numbers may wrap (restart from zero), see issues #802. Applications should be prepared to deal with that.

That issue was opened 10 months ago, so the system monitor may require changes to deal with the number wrapping (as suggested).

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍

[giampaolo]

(psutil author here)

It's the OS which wraps those numbers, psutil simply returns them unmodified. Unfortunately a bullet proof solution to this problem really does not exist.

psutil may do some magic in order to "watch" the numbers which wrap across function calls and do old_big_number + new_smaller_number (e.g. 233340298402 + 20) but of course when the function is called for the first time it doesn't know whether 20 is a number which has already wrapped or not.

Also, psutil is not aware of what the exact threshold at which the number is supposed to wrap is, so in the example above adding 20 is a step forward but the resulting number would still not be correct because it doesn't take the threshold into account. The right formula would be:

old_num + (threshold - old_num) + new_num

...but we don't have the threshold value.
That aside, assumed there is no bullet proof solution, rolling in the "partial" solution described above would be better than nothing and I'd be OK with it.

re: giampaolo/psutil#802

[giampaolo]

OK, I've fixed this in psutil: https://github.com/giampaolo/psutil/pull/1056/files (future 5.3.0).
Both disk_io_counters() and net_io_counters() now provide a new nowrap parameter (True by default) which adjusts the numbers which overflow and wrap (restart from 0) by adding "old value" to "new value". With this the returned numbers will always be increasing or remain the same, but never decrease.

[balloobbot]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.

Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍

[balloobbot]

This issue will be auto-closed because there hasn't been any activity for a few months. Feel free to open a new one if you still experience this problem 👍

[giampaolo]

New psutil 5.3.0 is out and it includes the fix which wraps the numbers in case they reset from zero so the fix for the home-assistant project consists in upgrading psutil.

[fanaticDavid]

That should be sorted in the next (major) release: #9253