Temperature sensors loose custom names in HA on NetModule boot

[yozik04]

Recently I had a power outage for some hours on my street. After that I noticed that all Temperature sensor names in Home Assistant were reset to default ones: <unit_name> temp <temp_sensor_id>. I had better names defined before the outage. This is a problem as I expect more frequent power outages in Europe in upcoming winter as the whole Europe decided to drown in it's energy crisis. But this is a story for another day. Let's return back to the problem.
I have flashed latest release Code Revision 20220831 2011 MQTT to all my 3 modules today and issue persists there as well.
In MQTT Explorer I see that NetModule sends empty MQTT message to temperature sensors configuration topics homeassistant/sensor/aaaaa/bbbbb/config on boot.
This is wrong and should be avoided. I see that switches do not do that, which is right behavior.

[nielsonm236]

I'm trying to remember our previous dialog on this when we developed that part of the code. Perhaps we can refresh each others memory without having to dig back through all our gitter communications and email.
First: The Network Module firmware itself does not have enough non-volatile storage space to store names for the temperature sensors. So it tracks the sensors using their serial numbers.
My recollection of our discussion at the time: Custom names could be entered in HA, and assuming HA was backed up you would then never lose the names. So we decided that was adequate coverage.
As FYI, we also don't create or store custom names for the Input/Output pins in the MQTT version of the code. The same discussion applied: You can do this in HA and back up your configs there.
Does this sound familiar?
With regard to sending empty messages on boot ... I need to look into this more. I have some recollection that we do that in case sensors are removed, added, or replaced, and the initialization messages are the only way to clear the old stuff out of HA. On this my recollection is less clear.
Mike

[yozik04]

NetModule does not need to store any names. It is not it's job.
HomeAssistant allows redefining sensor names.
When an NetworkModule sends homeassistant/.../../config topic with payload Home Assistant registers new sensor. When it recieves an empty payload to the same topic it removes sensor.
So what happens here:
On every reboot network module sends empty config to remove sensors from Home Assistant and then recreates them by sending valid payload to the same topic. Of course Home Assistant has already cleared all it's data regarding previous sensor and has created new one from scratch.

Jevgeni

[nielsonm236]

I went back and reviewed the code that handles this. There can always be an improvement (if it will fit). I wish I had made more notes on the "theory of operation" for this function. Here are the basics, and why I did what I did, even if in hindsight there is a better way:

1. The code assumption was that there wasn't room to store the Sensor serial numbers in a non-volatile way, so each time I boot I have no idea what sensors were there before, if any. So, I simply collect a list of current sensors and report them to the Browser and MQTT in serial number order.
2. There is room in RAM to store only 5 serial numbers.
3. An example operating scenario: Let's say I have 5 sensors with serial numbers 1, 2, 3, 4, 5. I will report them in order 1, 2, 3, 4, 5. Sensor serial number 3 fails and is replaced with Sensor serial number 7. On reboot I'll find 5 sensors and I'll report them in order 1, 2, 4, 5, 7 ... again because I have no knowledge of the previous serial numbers. In truth I think the order might be reversed, but they are either ascending or descending every time ...
4. I have some recollection that HA had some issue when this new reporting happened. Is HA able to handle the fact that Serial number 4 is now reported where 3 used to be and so on? And now there is a new Serial 7? I think this is where the idea of clearing the HA data and sending a new set of serial numbers came to mind.
5. Is HA somehow relating the serial numbers to the Custom Names? Or is it simply based on order?
   Let's discuss the above to see if there is a good path to a solution.
   Mike

[nielsonm236]

I should also add that this "rearrangement" can also happen during runtime if a sensor is intermittent. Let's say you have 1, 2, 3, 4, 5, but 3 is intermittent. Serial numbers 4 and 5 will shift positions during runtime if 3 disappears, then will shift back when 3 reappears. I might be able to do something about that as I haven't lost power and still know what was there, but I don't have it coded that way right now.

[yozik04]

Order of reporting is irrelevant. Serial number is key here. I do not know for what reason you store serial numbers in RAM. You can rescan sensors for every temperature read as well. No need to keep them.
If a sensor fails I think it is totally OK to go into MQTT Explorer and delete topics related to that sensor, so HomeAssistant would delete that sensor from it's database as well. It is very rare occasion and does not need a complicated solution.

I've found some time to debug this further and found multiple issues:
Here is the log when I subscribe to homeassistant/sensor/3a7c01fb9b4e/+/config topic:

# Retained mesages:
     1	{"tst":"2022-09-14T12:56:13.519937Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/06473b8b6d00/config","qos":0,"retain":1,"payloadlen":313,"payload":{"uniq_id":"3a7c01fb9b4e_temp_06473b8b6d00","name":"heat2floor temp 06473b8b6d00","~":"NetworkModule/heat2floor","avty_t":"~/availability","stat_t":"~/temp/06473b8b6d00","unit_of_meas":"°C","dev":{"ids":["NetworkModule_3a7c01fb9b4e"],"mdl":"HW-584","mf":"NetworkModule","name":"heat2floor","sw":"20220831 2011"}}}
     2	{"tst":"2022-09-14T12:56:13.520192Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/04473b4d5f00/config","qos":0,"retain":1,"payloadlen":313,"payload":{"uniq_id":"3a7c01fb9b4e_temp_04473b4d5f00","name":"heat2floor temp 04473b4d5f00","~":"NetworkModule/heat2floor","avty_t":"~/availability","stat_t":"~/temp/04473b4d5f00","unit_of_meas":"°C","dev":{"ids":["NetworkModule_3a7c01fb9b4e"],"mdl":"HW-584","mf":"NetworkModule","name":"heat2floor","sw":"20220831 2011"}}}
# Doing reboot
     3	{"tst":"2022-09-14T12:57:02.049042Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/06473b8b6d00/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
     4	{"tst":"2022-09-14T12:57:02.189550Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/04473b4d5f00/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
     5	{"tst":"2022-09-14T12:57:02.350007Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/000000000000/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
     6	{"tst":"2022-09-14T12:57:02.490319Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/000000000000/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
     7	{"tst":"2022-09-14T12:57:02.650851Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/000000000000/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
     8	{"tst":"2022-09-14T12:57:02.791230Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/06473b8b6d00/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
     9	{"tst":"2022-09-14T12:57:02.951664Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/04473b4d5f00/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    10	{"tst":"2022-09-14T12:57:03.092124Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/000000000000/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    11	{"tst":"2022-09-14T12:57:03.252424Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/000000000000/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    12	{"tst":"2022-09-14T12:57:03.392882Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/000000000000/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    13	{"tst":"2022-09-14T12:57:03.557952Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/06473b8b6d00/config","qos":0,"retain":0,"payloadlen":313,"payload":{"uniq_id":"3a7c01fb9b4e_temp_06473b8b6d00","name":"heat2floor temp 06473b8b6d00","~":"NetworkModule/heat2floor","avty_t":"~/availability","stat_t":"~/temp/06473b8b6d00","unit_of_meas":"°C","dev":{"ids":["NetworkModule_3a7c01fb9b4e"],"mdl":"HW-584","mf":"NetworkModule","name":"heat2floor","sw":"20220831 2011"}}}
    14	{"tst":"2022-09-14T12:57:03.702020Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/04473b4d5f00/config","qos":0,"retain":0,"payloadlen":313,"payload":{"uniq_id":"3a7c01fb9b4e_temp_04473b4d5f00","name":"heat2floor temp 04473b4d5f00","~":"NetworkModule/heat2floor","avty_t":"~/availability","stat_t":"~/temp/04473b4d5f00","unit_of_meas":"°C","dev":{"ids":["NetworkModule_3a7c01fb9b4e"],"mdl":"HW-584","mf":"NetworkModule","name":"heat2floor","sw":"20220831 2011"}}}

Found problems:

1. New firmware does not send retain: 1 (Messages 3-14). All configuration related topics must be retained! Seems that previous firmware actually did retain. Otherwise I would not have initial 2 retained messages in the log (Message 1,2). These messages are kept by the broker from previous firmware.
2. Unit sends empty payloads for all sensors that are in your array. You forgot to filter out serial numbers 000000000000. Messages 5-7,10-12
3. Unit resets(empty mqtt payload) config topics two times: Message blocks repeat: 3-7 and 8-12.
4. For proper operation with HomeAssistant Messages 3-12 should not be sent. This also solves 2. and 3.

I need to validate switch and binary sensor topics as well. I suspect there will be similar problems. If I will find something I will create a separate thread for that.

[yozik04]

When NetModule publishes MQTT config topics to broker without retain, then a bit later started HomeAssistant will not be able to setup these sensors. Broker will not resend these messages to HA and HA will know nothing about them.

[yozik04]

Checked binary_sensor and switch topics. The only thing is missing is message retaining. When you send empty payloads then you also need to set retain to 1.

[nielsonm236]

Thanks for digging into this. These issues must have been present for a very long time. The last time this area of code was touched was the May 9, 2021 release when Issues #55 #57 #58 and #66 were addressed ... probably creating all the problems you are describing. I'll get to work on this shortly.
Mike

[nielsonm236]

I think I've found the "fails to send retain" problem. Not sure what I was thinking when I broke that so I'm looking more closely to get it right. I can also filter out the temperature sensors with "0" as an ID. I'm still looking at why it happens twice on boot.

Just to be sure, you are saying I should never send an empty payload for a temperature sensor, and instead leave it up to the user to remove sensors in HA (or whatever they are using) once they are gone, correct?

[yozik04]

Correct

[nielsonm236]

Well, it appears I did NOT find the "fails to send retain" problem. The code I thought I may have broken always sets the Retain bit, so it is being lost further into the MQTT packing code. I will have to go through some detailed debugging to find where this is being disrupted. I'm going to release what I have now for Issues #94 and #100, then will work through this problem. This may take some time but I will get it done.

[yozik04]

Thank you @nielsonm236 for looking into it.

[nielsonm236]

@yozik04 I applied debug in my code all the way down to the point that messages are packed for sending on MQTT. Retain was set all the way down to that level. Attaching a log file from my Mosquitto server. I started it in verbose mode within a PowerShell window so that I could see all messaging. It appears to show that all messages sent from the Network Module have Retain set to 1 (as shown in the log entries by "r1"). I am not sure what to do next to determine the difference between your setup and mine. Thinking ... but could also use some suggestions.
Mike
20220918 mosquitto session.txt

[nielsonm236]

I'm also not seeing the multiple delete messages or the Temp Sensor ID 000000000000 message. I'm easily confused so it could still be me, but is it possible you inadvertently loaded a very old code load? I have to ask, although I consider that unlikely since you said you tried the latest code. If you look at the bottom of the Configuration page it should show the code revision level. I'm still looking at code.

[yozik04]

[yozik04]

Here is how I grabbed the log:
mosquitto_sub -u myuser -P mypass -F "%J" -t homeassistant/binary_sensor/3a7c01fb9b4e/+/config | nl

I did that from Raspberry Pi where mosquitto-clients is installed.

[nielsonm236]

I think I have part of the mystery solved. When running mosquitto_sub we've actually started a client that, with your command above, subscribes as a client and is displaying messages sent to it by the broker (so, not really a log). Watching the traffic under PowerShell with "mosquitto -v" I see the following with each publish of a temperature sensor from the Network Module:
1663532256: Received PUBLISH from NetworkModulec24d696b0182 (d0, q0, r1, m0, 'NetworkModule/NetworkModule-182/temp/3c01d0753da5', ... (6 bytes))
1663532256: Sending PUBLISH to mosq-9cu47kQ1q4nx3UySwz (d0, q0, r0, m0, 'NetworkModule/NetworkModule-182/temp/3c01d0753da5', ... (6 bytes))
The first line is traffic from the Network Module. The second line is the traffic going from the broker to the mosquitto_sub client. So the traffic from the NetworkModule has Retain set to 1, and the traffic sent to the mosquitto_sub client has Retain set to 0. At the mosquitto_sub client we are only seeing its incoming traffic (line 2). We were being misled by the mosquitto_sub display.
I do believe the Network Module always sends Retain set to 1. So I will set this part of the issue aside unless you see something I don't.
I will go on to figuring out why messages are sent twice, and why the ID 000000000000 appears. FYI, so far I'm not seeing that in my monitoring, but on examining code I think it could happen so I will continue.
Mike

[nielsonm236]

As an aside it looks like the very first time the broker sends a Publish to mosquitto_sub it sets Retain to 1. Thereafter it is always 0. I'm not sure why, and I don't always see that happen.

[nielsonm236]

I changed my testing to look at the homeassistant HA messages instead of the NetworkModule messages (as you said above ... but I overlooked it). Once doing that I could see the duplication and ID 000000000000 problems. I've tracked that down in firmware, applied fixes, and I'm testing.

[yozik04]

I think I have part of the mystery solved. When running mosquitto_sub we've actually started a client that, with your command above, subscribes as a client and is displaying messages sent to it by the broker (so, not really a log). Watching the traffic under PowerShell with "mosquitto -v" I see the following with each publish of a temperature sensor from the Network Module: 1663532256: Received PUBLISH from NetworkModulec24d696b0182 (d0, q0, r1, m0, 'NetworkModule/NetworkModule-182/temp/3c01d0753da5', ... (6 bytes)) 1663532256: Sending PUBLISH to mosq-9cu47kQ1q4nx3UySwz (d0, q0, r0, m0, 'NetworkModule/NetworkModule-182/temp/3c01d0753da5', ... (6 bytes)) The first line is traffic from the Network Module. The second line is the traffic going from the broker to the mosquitto_sub client. So the traffic from the NetworkModule has Retain set to 1, and the traffic sent to the mosquitto_sub client has Retain set to 0. At the mosquitto_sub client we are only seeing its incoming traffic (line 2). We were being misled by the mosquitto_sub display. I do believe the Network Module always sends Retain set to 1. So I will set this part of the issue aside unless you see something I don't. I will go on to figuring out why messages are sent twice, and why the ID 000000000000 appears. FYI, so far I'm not seeing that in my monitoring, but on examining code I think it could happen so I will continue. Mike

Ahh. You are right... I did not thought about that. Sorry for the confusion I made.

[nielsonm236]

@yozik04 I've tested the code changes for this Issue as follows:

• Verified that at boot or reboot the Temperature Sensor Config messages are sent.
• Verified that NO Temperature Sensor Config messages with empty payload (aka a "delete") is ever sent. So, no delete at boot, reboot, or when a sensor is removed (runtime or reboot). And if a sensor is removed the Network Module simply stops sending config messages for the missing component at boot/reboot. Thus if a Temperature Sensor is configured in HA it can only be removed in HA via the HA user interface.
• Verified that no "ID 000000000000" messages are ever sent.

During test I noticed that the broker sent an empty payload message for a sensor to mosquitto_sub. It was not sent by the Network Module to the broker. I only saw that one time and I was not able to reproduce it. I have no clue why/how that happened, but it occurred in the very first test, and not again over about 25 test attempts. I will continue to watch for that.

All that sounds like what you requested. But it raised an additional question: During boot I send empty payload messages for "switch" and "binary_sensor" topics followed by their Config messages because I can't be sure if someone has changed a pin to Input, Output, or Disabled from some previous setting. Is this a problem? I did a few short tests with rebooting a module (when the deletes occur) and HA didn't lose the names I had given pins within HA. Having said that, I don't know if there is some power loss scenario for the modules and HA host that might cause a problem as all my testing left equipment powered up.

Mike

[nielsonm236]

I was just re-reading your original statement, and you indicate that switch/binary_sensor doesn't issue empty payloads. I'm pretty sure it does, for every switch/binary_sensor, at every boot ... the code reads that way. I will look at this further to be sure of my statement.

[nielsonm236]

Another question: On my test module I removed two temperature sensors so they would stop sending Config messages to HA. Now hours later I'm trying to remove the created Entities in HA and it won't let me. HA suggested I restart HA and try again ... I did that and it still won't let me remove them. Since this is the method we were hoping to use this seems like a problem. Any ideas?

[nielsonm236]

Well, I "disabled" them rather than delete. They still appeared but in a "disabled" list. Then after a few minutes they magically disappeared from HA ... at least I couldn't find them anymore. Who knows.

[yozik04]

I was just re-reading your original statement, and you indicate that switch/binary_sensor doesn't issue empty payloads. I'm pretty sure it does, for every switch/binary_sensor, at every boot ... the code reads that way. I will look at this further to be sure of my statement.

My test showed that empty payloads were not sent for switches and binary_sensors. Maybe you send these only on configuration save? I will retest again if you give me new version to flash.

[yozik04]

Another question: On my test module I removed two temperature sensors so they would stop sending Config messages to HA. Now hours later I'm trying to remove the created Entities in HA and it won't let me. HA suggested I restart HA and try again ... I did that and it still won't let me remove them. Since this is the method we were hoping to use this seems like a problem. Any ideas?

Maybe it is easier to remove whole device then?

I was using MqttExplorer to remove topics under homeassistant/<device_id>

[nielsonm236]

I haven't been able to do more testing since our messages above. I'm going to do a little during the next two hours.

re: "Maybe it is easier to remove whole device then?" I found that disabling the entity in HA (rather than deleting) appears to have caused it to be deleted, although it took a few minutes. I'm still not sure about this so I need to see if it reliably deletes the missing sensors.

Also, deleting the whole device defeats our purpose here, doesn't it? Won't that cause loss of all unique names entered in HA for that device? As FYI, I have found that I often have to delete a "card", then create a new one in order to see changes I've made in HA.

Attached is the code I'm testing with. This is the version that requires programming with the SWIM interface. All of my testing is done with the programming-over-ethernet version of the code ... but that makes no difference to what we are testing now.

Mike
NetworkModule-MQTT-test.zip

[nielsonm236]

I was able to set up my test fairly quickly, and I verified that the "empty payload" messages are indeed being sent. I can see them in PowerShell running Mosquitto with "./mosquitto -v", and I see them being forwarded to mosquitto_sub.

I think you might not be seeing them depending on how you subscribed with mosquitto_sub. I used this statement:
mosquitto_sub -F "%J" -t homeassistant/+/c24d696b0182/+/config

What I'm seeing is that at boot if I find a pin defined as an Output I will first send a blank payload for that pin # defined as a "binary_sensor" (thus deleting any previous Input definition for the pin). Then I will send the "switch" Config for that pin.

Likewise if I find a pin defined as an Input I will first send a blank payload for that pin # defined as a "switch" (thus deleting any previous Output definition for the pin. Then I will send the "binary_sensor" Config for that pin.

If I find a pin defined as Disabled I will send a "binary_sensor" message with a blank payload, followed by a "switch" message with a blank payload, thus deleting ANY definition for the pin in HA.

This means that if a pin definition did not change (stayed as a switch or stayed as a binary_sensor) HA probably ignores the delete message I send.

This is a little different than what I expected (although I think it is right) so I will look at my code a little more closely to make sure I have the correct comments describing operation.

So the way I'm handling "switch" and "binary_sensor" entities is different than the way I was handling "sensors" (ie, Temperature Sensors) in that I would actually delete the sensors in the previous code. As you point out that was leading to problems.

OK, now I'm off to look at the code again.

Mike

[nielsonm236]

That didn't take long. I looked at the comments in the code and they describe exactly what I said above with my test results. As I said before, I'm easily confused.
I want to repeat my tests where I add and remove temperature sensors to verify how to properly delete old sensor definition in HA.

[nielsonm236]

HA can sure be confusing. I re-attached the two Temperature Sensors that I had previously "Disabled" in HA. They showed up again as "Disabled entities". So HA never really forgot about them, It just wasn't showing them until they physically reappeared. Then I couldn't get them re-enabled ... or I thought I couldn't. Per a pop-up in HA I had to reboot the HA host (supervisor / system / reboot host) and even then they showed as Disabled. But after about 2 minutes they changed to "Enabled".

So I think the key here when "removing" Temperature Sensor configs is to "Disable" them, not "Delete" them. Then perform a "reboot host". Then give HA a couple of minutes to update its configs. There might be some quicker way by editing yaml, and if editing is done there might also be a way to delete any knowledge HA has about disabled configs. I don't want to go there.

I'll wait to see if you have feedback. Otherwise I think we've resolved the this Issue. I will have to add some of what I've learned to the manual.

[yozik04]

I will retest this week and get back to you. Thank you!

[yozik04]

Testing switches and binary_sensors:

mosquitto_sub -u user -P pass-F "%J" -t homeassistant/+/3a7c01fb9b5b/+/config | nl
     1  {"tst":"2022-09-21T21:44:48.617781Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/01/config","qos":0,"retain":1,"payloadlen":294,"payload":{"uniq_id":"3a7c01fb9b5b_output_01","name":"heat3floor output 01","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/output/01","cmd_t":"~/output/01/set","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
     2  {"tst":"2022-09-21T21:44:48.618235Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/02/config","qos":0,"retain":1,"payloadlen":294,"payload":{"uniq_id":"3a7c01fb9b5b_output_02","name":"heat3floor output 02","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/output/02","cmd_t":"~/output/02/set","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
     3  {"tst":"2022-09-21T21:44:48.618332Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/03/config","qos":0,"retain":1,"payloadlen":294,"payload":{"uniq_id":"3a7c01fb9b5b_output_03","name":"heat3floor output 03","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/output/03","cmd_t":"~/output/03/set","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
     4  {"tst":"2022-09-21T21:44:48.618695Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/04/config","qos":0,"retain":1,"payloadlen":294,"payload":{"uniq_id":"3a7c01fb9b5b_output_04","name":"heat3floor output 04","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/output/04","cmd_t":"~/output/04/set","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
     5  {"tst":"2022-09-21T21:44:48.618859Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/05/config","qos":0,"retain":1,"payloadlen":265,"payload":{"uniq_id":"3a7c01fb9b5b_input_05","name":"heat3floor input 05","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/input/05","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
     6  {"tst":"2022-09-21T21:44:48.619007Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/06/config","qos":0,"retain":1,"payloadlen":265,"payload":{"uniq_id":"3a7c01fb9b5b_input_06","name":"heat3floor input 06","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/input/06","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
     7  {"tst":"2022-09-21T21:44:48.619153Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/07/config","qos":0,"retain":1,"payloadlen":265,"payload":{"uniq_id":"3a7c01fb9b5b_input_07","name":"heat3floor input 07","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/input/07","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
     8  {"tst":"2022-09-21T21:44:48.619288Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/08/config","qos":0,"retain":1,"payloadlen":265,"payload":{"uniq_id":"3a7c01fb9b5b_input_08","name":"heat3floor input 08","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/input/08","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
#### REBOOT ####
     9  {"tst":"2022-09-21T21:45:31.364051Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/05/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    10  {"tst":"2022-09-21T21:45:31.507177Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/05/config","qos":0,"retain":0,"payloadlen":265,"payload":{"uniq_id":"3a7c01fb9b5b_input_05","name":"heat3floor input 05","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/input/05","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
    11  {"tst":"2022-09-21T21:45:31.666361Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/06/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    12  {"tst":"2022-09-21T21:45:31.810539Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/06/config","qos":0,"retain":0,"payloadlen":265,"payload":{"uniq_id":"3a7c01fb9b5b_input_06","name":"heat3floor input 06","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/input/06","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
    13  {"tst":"2022-09-21T21:45:31.970720Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/07/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    14  {"tst":"2022-09-21T21:45:32.113974Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/07/config","qos":0,"retain":0,"payloadlen":265,"payload":{"uniq_id":"3a7c01fb9b5b_input_07","name":"heat3floor input 07","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/input/07","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
    15  {"tst":"2022-09-21T21:45:32.273941Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/08/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    16  {"tst":"2022-09-21T21:45:32.418147Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/08/config","qos":0,"retain":0,"payloadlen":265,"payload":{"uniq_id":"3a7c01fb9b5b_input_08","name":"heat3floor input 08","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/input/08","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
    17  {"tst":"2022-09-21T21:45:33.775823Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/01/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    18  {"tst":"2022-09-21T21:45:33.919239Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/01/config","qos":0,"retain":0,"payloadlen":294,"payload":{"uniq_id":"3a7c01fb9b5b_output_01","name":"heat3floor output 01","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/output/01","cmd_t":"~/output/01/set","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
    19  {"tst":"2022-09-21T21:45:34.079188Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/02/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    20  {"tst":"2022-09-21T21:45:34.222700Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/02/config","qos":0,"retain":0,"payloadlen":294,"payload":{"uniq_id":"3a7c01fb9b5b_output_02","name":"heat3floor output 02","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/output/02","cmd_t":"~/output/02/set","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
    21  {"tst":"2022-09-21T21:45:34.382576Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/03/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    22  {"tst":"2022-09-21T21:45:34.525877Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/03/config","qos":0,"retain":0,"payloadlen":294,"payload":{"uniq_id":"3a7c01fb9b5b_output_03","name":"heat3floor output 03","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/output/03","cmd_t":"~/output/03/set","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
    23  {"tst":"2022-09-21T21:45:34.685723Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/04/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    24  {"tst":"2022-09-21T21:45:34.830138Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/04/config","qos":0,"retain":0,"payloadlen":294,"payload":{"uniq_id":"3a7c01fb9b5b_output_04","name":"heat3floor output 04","~":"NetworkModule/heat3floor","avty_t":"~/availability","stat_t":"~/output/04","cmd_t":"~/output/04/set","dev":{"ids":["NetworkModule_3a7c01fb9b5b"],"mdl":"HW-584","mf":"NetworkModule","name":"heat3floor","sw":"20220911 test"}}}
    25  {"tst":"2022-09-21T21:45:37.983254Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/09/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    26  {"tst":"2022-09-21T21:45:38.122730Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/09/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    27  {"tst":"2022-09-21T21:45:38.282564Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/10/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    28  {"tst":"2022-09-21T21:45:38.423786Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/10/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    29  {"tst":"2022-09-21T21:45:38.583002Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/11/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    30  {"tst":"2022-09-21T21:45:38.722479Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/11/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    31  {"tst":"2022-09-21T21:45:38.882248Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/12/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    32  {"tst":"2022-09-21T21:45:39.022876Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/12/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    33  {"tst":"2022-09-21T21:45:39.183665Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/13/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    34  {"tst":"2022-09-21T21:45:39.324421Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/13/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    35  {"tst":"2022-09-21T21:45:39.483934Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/14/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    36  {"tst":"2022-09-21T21:45:39.624515Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/14/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    37  {"tst":"2022-09-21T21:45:39.784307Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/15/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    38  {"tst":"2022-09-21T21:45:39.923952Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/15/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    39  {"tst":"2022-09-21T21:45:40.090628Z+0300","topic":"homeassistant/binary_sensor/3a7c01fb9b5b/16/config","qos":0,"retain":0,"payloadlen":0,"payload":null}
    40  {"tst":"2022-09-21T21:45:40.230268Z+0300","topic":"homeassistant/switch/3a7c01fb9b5b/16/config","qos":0,"retain":0,"payloadlen":0,"payload":null}

Works as expected. I do not see any empty payloads that would cleanup existing entities in HA. Testing passed.

Testing temperature sensors:

mosquitto_sub -u user -P pass -F "%J" -t homeassistant/sensor/3a7c01fb9b4e/+/config | nl
     1  {"tst":"2022-09-21T21:55:20.960922Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/06473b8b6d00/config","qos":0,"retain":1,"payloadlen":313,"payload":{"uniq_id":"3a7c01fb9b4e_temp_06473b8b6d00","name":"heat2floor temp 06473b8b6d00","~":"NetworkModule/heat2floor","avty_t":"~/availability","stat_t":"~/temp/06473b8b6d00","unit_of_meas":"°C","dev":{"ids":["NetworkModule_3a7c01fb9b4e"],"mdl":"HW-584","mf":"NetworkModule","name":"heat2floor","sw":"20220911 test"}}}
     2  {"tst":"2022-09-21T21:55:20.961111Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/04473b4d5f00/config","qos":0,"retain":1,"payloadlen":313,"payload":{"uniq_id":"3a7c01fb9b4e_temp_04473b4d5f00","name":"heat2floor temp 04473b4d5f00","~":"NetworkModule/heat2floor","avty_t":"~/availability","stat_t":"~/temp/04473b4d5f00","unit_of_meas":"°C","dev":{"ids":["NetworkModule_3a7c01fb9b4e"],"mdl":"HW-584","mf":"NetworkModule","name":"heat2floor","sw":"20220911 test"}}}
### REBOOT ###
     3  {"tst":"2022-09-21T21:56:08.098865Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/06473b8b6d00/config","qos":0,"retain":0,"payloadlen":313,"payload":{"uniq_id":"3a7c01fb9b4e_temp_06473b8b6d00","name":"heat2floor temp 06473b8b6d00","~":"NetworkModule/heat2floor","avty_t":"~/availability","stat_t":"~/temp/06473b8b6d00","unit_of_meas":"°C","dev":{"ids":["NetworkModule_3a7c01fb9b4e"],"mdl":"HW-584","mf":"NetworkModule","name":"heat2floor","sw":"20220911 test"}}}
     4  {"tst":"2022-09-21T21:56:08.243597Z+0300","topic":"homeassistant/sensor/3a7c01fb9b4e/04473b4d5f00/config","qos":0,"retain":0,"payloadlen":313,"payload":{"uniq_id":"3a7c01fb9b4e_temp_04473b4d5f00","name":"heat2floor temp 04473b4d5f00","~":"NetworkModule/heat2floor","avty_t":"~/availability","stat_t":"~/temp/04473b4d5f00","unit_of_meas":"°C","dev":{"ids":["NetworkModule_3a7c01fb9b4e"],"mdl":"HW-584","mf":"NetworkModule","name":"heat2floor","sw":"20220911 test"}}}

Also works as expected. No empty payloads are sent.

I can confirm that now HomeAssistant does not loose any custom data of temperature sensors.

You have resolved all the problems. Thank you!

[nielsonm236]

Addressed in Release 20220921 0500