All devices are lost after Z2M update #23235

paavokurkela · 2024-07-02T10:45:41Z

What happened?

After add-on update on Home Assistant, all the devices are lost from Z2M and Homa Assistant (no devices listed under MQTT instance)

Restoring a backup restores devices on Home Assistant but not on Z2M. Same issue happened to me also with earlier update, unfortunately I don't have any details on that. Log is literally empty.

What did you expect to happen?

No response

How to reproduce it (minimal and precise)

Update Z2M

Zigbee2MQTT version

1.38.0-1

Adapter firmware version

20230507

Adapter

Slaesh

Setup

Add-on under HA on Intel Nuc

Debug log

No response

kligithu · 2024-07-02T16:57:19Z

The same here

What did you expect to happen?
No response

How to reproduce it (minimal and precise)
Update Z2M

Zigbee2MQTT version
1.38.0-1 to 1.39.0-1

Adapter firmware version
core v2.3.6 : Zigbee 20240315

Adapter
SMLIGHT SLZB-06

Setup
Add-on under HA on Synology VMM

Debug log
No response

raffoul · 2024-07-02T18:04:54Z

Same. Rollback done.

Webmonsteril · 2024-07-02T18:22:35Z

Have same issue rolled back also

Najihel · 2024-07-02T20:36:41Z

Hello,

Pretty same issue here, devices are not missing but all entities are duplicated. The original one is broken and Z2M create duplicate suffixed by _2

jmcollin78 · 2024-07-02T22:30:26Z

Same here ! Full rollback running. What is strange is that the duplication of entities occurs after a full restart of HA and not just after the update to 1.39.0.

This is a severe issue and 1.39.0 deployment should be aborted if you want my opinion.

Edit: the full restor have restored the entities.

Najihel · 2024-07-03T04:53:12Z

Same here ! Full rollback running. What is strange is that the duplication of entities occurs after a full restart of HA and not just after the update to 1.39.0.

This is a severe issue and 1.39.0 deployment should be aborted if you want my opinion.

Edit: the full restor have restored the entities.

I confirm the trigger is the restart of HA on my side too.

gijsh070 · 2024-07-03T07:21:34Z

Same issue here, double sensors and entities. They even remain double after restoring.....

also all devices are offline. Can only control when directly go to zigbeetomqtt

countrr · 2024-07-03T13:22:34Z

Same here, had to roll back to 1.38.0. Will not update again until this is solved.

Zver5 · 2024-07-03T15:45:52Z

Same here, had to roll back to 1.38.0. Will not update again until this is solved.

raphmur · 2024-07-04T15:31:11Z

Had same issue updating today to 1.39.0. Rolling back to 1.38.0
What kind of log would you need? How to support?
I am running HAOS on RPi 4B

kefrens · 2024-07-04T20:39:53Z

I have same issue; How do you rollback zigbee2mqtt?

raphmur · 2024-07-04T22:31:45Z

@kefrens in home assistant settings> system> backups
Then select the backup that was normally done at z2m update and backup. Then reboot and you should be good

Vidjer · 2024-07-05T16:59:19Z

Hello have same issue. rollback 1.38 ok

paavokurkela · 2024-07-05T18:24:48Z

In my case rollback to 1.38 did not restore devices to Z2M.

raphmur · 2024-07-05T19:19:41Z

@paavokurkela , did you restart HA?

jmcollin78 · 2024-07-05T20:02:42Z

In my case rollback to 1.38 did not restore devices to Z2M.

I have to do a full restore and not only a rollback on Z2M 1.38

gijsh070 · 2024-07-05T20:11:47Z

In my case rollback to 1.38 did not restore devices to Z2M.

I have to do a full restore and not only a rollback on Z2M 1.38

Same here, full restore needed

paavokurkela · 2024-07-05T20:12:43Z

@raphmur Yes I did. I also tried full restore from backup. That restored entities on HA but Z2M devices did not restore.

Koenkk · 2024-07-07T08:52:00Z

Could you:

Provide your z2m configuration.yaml and if applicable, screenshot of the Z2M addon config page
The debug log when starting z2m 1.39.0
The version of HA that you are using

See this on how to enable debug logging.

paavokurkela · 2024-07-07T10:25:40Z

@Koenkk thank you for looking on this.

As I restored my system I currently have 1.38 installed.
I was able to reproduce the problem by pairing two Ikea lights and upgrade from 1.38 to 1.39.

Please note that this very same problem existed already earlier. I believe it was from 1.37 to 1.38 and it was first time this problem happened to me.

Please find infos requested below:

Couple devices paired to 1.38

Devices gone 1.39

HA version:

configuration.yaml (with debugs enabled)

data_path: /homeassistant/zigbee2mqtt/
socat:
  enabled: false
  master: pty,raw,echo=0,link=/tmp/ttyZ2M,mode=777
  slave: tcp-listen:8485,keepalive,nodelay,reuseaddr,keepidle=1,keepintvl=1,keepcnt=5
  options: "-d -d"
  log: true
mqtt:
  server: mqtt://core-mosquitto:1883
  user: ****
  password: ****
  client_id: Z2M
serial:
  port: /dev/ttyUSB0
zigbee_herdsman_debug: false
homeassistant: true
advanced:
  channel: 14
  log_level: debug

Add-on config screen (credentials hidden)

log (1.38 with two Ikea lights paired)

[12:40:58] INFO: Preparing to start...
[12:40:58] INFO: Socat not enabled
[12:40:59] INFO: Starting Zigbee2MQTT...
[2024-07-07 12:41:02] info: 	z2m: Logging to console, file (filename: log.log)
[2024-07-07 12:41:02] info: 	z2m: Starting Zigbee2MQTT version 1.38.0 (commit #unknown)
[2024-07-07 12:41:02] info: 	z2m: Starting zigbee-herdsman (0.49.2)
[2024-07-07 12:41:02] info: 	zh:zstack:znp: Opening SerialPort with {"path":"/dev/ttyUSB0","baudRate":115200,"rtscts":false,"autoOpen":false}
[2024-07-07 12:41:02] info: 	zh:zstack:znp: Serialport opened
[2024-07-07 12:41:02] info: 	zh:zstack:znp: Writing CC2530/CC2531 skip bootloader payload
[2024-07-07 12:41:03] info: 	zh:zstack:znp: Skip bootloader for CC2652/CC1352
[2024-07-07 12:41:27] info: 	zh:controller: Wrote coordinator backup to '/homeassistant/zigbee2mqtt/coordinator_backup.json'
[2024-07-07 12:41:27] info: 	z2m: zigbee-herdsman started (reset)
[2024-07-07 12:41:27] info: 	z2m: Coordinator firmware version: '{"meta":{"maintrel":1,"majorrel":2,"minorrel":7,"product":1,"revision":20230507,"transportrev":2},"type":"zStack3x0"}'
[2024-07-07 12:41:27] info: 	z2m: Currently 0 devices are joined:
[2024-07-07 12:41:27] info: 	z2m: Zigbee: disabling joining new devices.
[2024-07-07 12:41:27] info: 	z2m:mqtt: Connecting to MQTT server at mqtt://core-mosquitto:1883
[2024-07-07 12:41:27] info: 	z2m:mqtt: Connected to MQTT server
[2024-07-07 12:41:27] info: 	z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-07 12:41:28] info: 	z2m: Started frontend on port 8099
[2024-07-07 12:41:28] info: 	z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-07 12:41:28] info: 	z2m: Zigbee2MQTT started!
[2024-07-07 12:41:33] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/1221051039810110150109113116116_0x00124b00259bb8b4/connection_state/config', payload '{"device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.38.0"},"device_class":"connectivity","entity_category":"diagnostic","name":"Connection state","object_id":"zigbee2mqtt_bridge_connection_state","origin":{"name":"Zigbee2MQTT","sw":"1.38.0","url":"https://www.zigbee2mqtt.io"},"payload_off":"offline","payload_on":"online","state_topic":"zigbee2mqtt/bridge/state","unique_id":"bridge_0x00124b00259bb8b4_connection_state_zigbee2mqtt","value_template":"{{ value_json.state }}"}'

log (startup during upgrade process to 1.39 and all devices lost)

[13:05:07] INFO: Preparing to start...
[13:05:08] INFO: Socat not enabled
[13:05:09] INFO: Starting Zigbee2MQTT...
Starting Zigbee2MQTT without watchdog.
[2024-07-07 13:05:11] info: 	z2m: Logging to console, file (filename: log.log)
[2024-07-07 13:05:11] info: 	z2m: Starting Zigbee2MQTT version 1.39.0 (commit #unknown)
[2024-07-07 13:05:11] info: 	z2m: Starting zigbee-herdsman (0.50.1)
[2024-07-07 13:05:12] info: 	zh:zstack:znp: Opening SerialPort with {"path":"/dev/ttyUSB0","baudRate":115200,"rtscts":false,"autoOpen":false}
[2024-07-07 13:05:12] info: 	zh:zstack:znp: Serialport opened
[2024-07-07 13:05:12] info: 	zh:zstack:znp: Writing CC2530/CC2531 skip bootloader payload
[2024-07-07 13:05:13] info: 	zh:zstack:znp: Skip bootloader for CC2652/CC1352
[2024-07-07 13:05:37] info: 	zh:controller: Wrote coordinator backup to '/homeassistant/zigbee2mqtt/coordinator_backup.json'
[2024-07-07 13:05:37] info: 	z2m: zigbee-herdsman started (reset)
[2024-07-07 13:05:37] info: 	z2m: Coordinator firmware version: '{"meta":{"maintrel":1,"majorrel":2,"minorrel":7,"product":1,"revision":20230507,"transportrev":2},"type":"zStack3x0"}'
[2024-07-07 13:05:37] info: 	z2m: Currently 0 devices are joined:
[2024-07-07 13:05:37] info: 	z2m: Zigbee: disabling joining new devices.
[2024-07-07 13:05:37] info: 	z2m: Connecting to MQTT server at mqtt://core-mosquitto:1883
[2024-07-07 13:05:37] info: 	z2m: Connected to MQTT server
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-07 13:05:37] info: 	z2m: Started frontend on port 8099
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-07 13:05:37] info: 	z2m: Zigbee2MQTT started!
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/0x90395efffee6d2b7/update_available/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/0x287681fffe9cb162/update_available/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/button/0x90395efffee6d2b7/identify/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/button/0x287681fffe9cb162/identify/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/effect/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/power_on_behavior/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/identify/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/effect/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/power_on_behavior/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/identify/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/power_on_behavior/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/color_options/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/linkquality/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/update_state/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/power_on_behavior/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/color_options/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/linkquality/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/update_state/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/light/0x90395efffee6d2b7/light/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/light/0x287681fffe9cb162/light/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/update/0x90395efffee6d2b7/update/config', payload 'null'
[2024-07-07 13:05:37] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/update/0x287681fffe9cb162/update/config', payload 'null'
[2024-07-07 13:05:42] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/1221051039810110150109113116116_0x00124b00259bb8b4/connection_state/config', payload '{"device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"device_class":"connectivity","entity_category":"diagnostic","name":"Connection state","object_id":"zigbee2mqtt_bridge_connection_state","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"payload_off":"offline","payload_on":"online","state_topic":"zigbee2mqtt/bridge/state","unique_id":"bridge_0x00124b00259bb8b4_connection_state_zigbee2mqtt","value_template":"{{ value_json.state }}"}'
[2024-07-07 13:05:42] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/1221051039810110150109113116116_0x00124b00259bb8b4/restart_required/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"device_class":"problem","enabled_by_default":false,"entity_category":"diagnostic","name":"Restart required","object_id":"zigbee2mqtt_bridge_restart_required","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"payload_off":false,"payload_on":true,"state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_restart_required_zigbee2mqtt","value_template":"{{ value_json.restart_required }}"}'
[2024-07-07 13:05:42] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/button/1221051039810110150109113116116_0x00124b00259bb8b4/restart/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","command_topic":"zigbee2mqtt/bridge/request/restart","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"device_class":"restart","name":"Restart","object_id":"zigbee2mqtt_bridge_restart","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"payload_press":"","unique_id":"bridge_0x00124b00259bb8b4_restart_zigbee2mqtt"}'
[2024-07-07 13:05:42] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/1221051039810110150109113116116_0x00124b00259bb8b4/log_level/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","command_template":"{\"options\": {\"advanced\": {\"log_level\": \"{{ value }}\" } } }","command_topic":"zigbee2mqtt/bridge/request/options","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"entity_category":"config","name":"Log level","object_id":"zigbee2mqtt_bridge_log_level","options":["error","warning","info","debug"],"origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_log_level_zigbee2mqtt","value_template":"{{ value_json.log_level | lower }}"}'
[2024-07-07 13:05:42] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/1221051039810110150109113116116_0x00124b00259bb8b4/version/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"entity_category":"diagnostic","icon":"mdi:zigbee","name":"Version","object_id":"zigbee2mqtt_bridge_version","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_version_zigbee2mqtt","value_template":"{{ value_json.version }}"}'
[2024-07-07 13:05:42] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/1221051039810110150109113116116_0x00124b00259bb8b4/coordinator_version/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"enabled_by_default":false,"entity_category":"diagnostic","icon":"mdi:chip","name":"Coordinator version","object_id":"zigbee2mqtt_bridge_coordinator_version","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_coordinator_version_zigbee2mqtt","value_template":"{{ value_json.coordinator.meta.revision }}"}'
[2024-07-07 13:05:42] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/1221051039810110150109113116116_0x00124b00259bb8b4/network_map/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"enabled_by_default":false,"entity_category":"diagnostic","json_attributes_template":"{{ value_json.data.value | tojson }}","json_attributes_topic":"zigbee2mqtt/bridge/response/networkmap","name":"Network map","object_id":"zigbee2mqtt_bridge_network_map","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"state_topic":"zigbee2mqtt/bridge/response/networkmap","unique_id":"bridge_0x00124b00259bb8b4_network_map_zigbee2mqtt","value_template":"{{ now().strftime('%Y-%m-%d %H:%M:%S') }}"}'
[2024-07-07 13:05:43] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/1221051039810110150109113116116_0x00124b00259bb8b4/permit_join_timeout/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"device_class":"duration","entity_category":"diagnostic","name":"Permit join timeout","object_id":"zigbee2mqtt_bridge_permit_join_timeout","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_permit_join_timeout_zigbee2mqtt","unit_of_measurement":"s","value_template":"{{ iif(value_json.permit_join_timeout is defined, value_json.permit_join_timeout, None) }}"}'
[2024-07-07 13:05:43] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/switch/1221051039810110150109113116116_0x00124b00259bb8b4/permit_join/config', payload '{"availability":[{"topic":"zigbee2mqtt/bridge/state","value_template":"{{ value_json.state }}"}],"availability_mode":"all","command_topic":"zigbee2mqtt/bridge/request/permit_join","device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.39.0"},"icon":"mdi:human-greeting-proximity","name":"Permit join","object_id":"zigbee2mqtt_bridge_permit_join","origin":{"name":"Zigbee2MQTT","sw":"1.39.0","url":"https://www.zigbee2mqtt.io"},"payload_off":"false","payload_on":"true","state_topic":"zigbee2mqtt/bridge/info","unique_id":"bridge_0x00124b00259bb8b4_permit_join_zigbee2mqtt","value_template":"{{ value_json.permit_join | lower }}"}'

dcyou · 2024-07-09T12:09:34Z

hi guys I had same issue 😞 too but I manage myself by taking time to remove the old entities (without _2) then renaming all the entities by removing the "_2" (+/- 10 devices with 4 entity each) 💪 But I struggle with how to remove the zigbee2mqtt_bridge entities, if someone how it is possible? 🆘

jmcollin78 · 2024-07-09T12:33:28Z

Could you:

Provide your z2m configuration.yaml and if applicable, screenshot of the Z2M addon config page

The debug log when starting z2m 1.39.0

The version of HA that you are using

See this on how to enable debug logging.

Hello @Koenkk ,

Is there any chance to have a fix in the 1.39.1 release (#23221) ?
It is not listed in the release note.

Koenkk · 2024-07-09T19:08:43Z

@paavokurkela in the 1.38.0 log I see [2024-07-07 12:41:27] info: z2m: Currently 0 devices are joined:. Could you provide the debug log of:

Start z2m 1.38.0
Pair 2 devices
Restart z2m 1.38.0
Upgrade to z2m 1.39.0

See this on how to enable debug logging.

paavokurkela · 2024-07-10T06:25:04Z

@Koenkk thank you for your request.

While gathering logs you requested I found out that devices are gone every time I just simply restart Z2M, no even update needed. Please see log I was able to catch for you below:

Something I want to point out is that while I tried to store log.txt under /homeassistant/zigbee2mqtt/log/%TIMESTAMP% nothing was stored there. Also as log below says:
[2024-07-10 08:41:39] info: zh:controller: Wrote coordinator backup to '/homeassistant/zigbee2mqtt/coordinator_backup.json'
the file exists but it's timestamp is for Jan 14th which I believe is date I have installed the system at first place.

Hope this leads us to right direction.

[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:writer: --> frame [254,5,33,50,1,7,0,2,0,18]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: <-- [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --> parsed 4 - 3 - 1 - 50 - [12,0,0,0] - 91
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SRSP: <-- SYS - nvLength - {"len":12}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SREQ: --> SYS - nvRead - {"sysid":1,"itemid":7,"subid":2,"offset":0,"len":12}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:writer: --> frame [254,8,33,51,1,7,0,2,0,0,0,12,18]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: <-- [254,14,97,51,0,12,0,0,0,0,46,65,18,57,193,108,55,244,122]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext [254,14,97,51,0,12,0,0,0,0,46,65,18,57,193,108,55,244,122]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --> parsed 14 - 3 - 1 - 51 - [0,12,0,0,0,0,46,65,18,57,193,108,55,244] - 122
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SRSP: <-- SYS - nvRead - {"status":0,"len":12,"value":{"type":"Buffer","data":[0,0,0,0,46,65,18,57,193,108,55,244]}}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SREQ: --> SYS - nvLength - {"sysid":1,"itemid":7,"subid":3}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:writer: --> frame [254,5,33,50,1,7,0,3,0,19]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: <-- [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --> parsed 4 - 3 - 1 - 50 - [12,0,0,0] - 91
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SRSP: <-- SYS - nvLength - {"len":12}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SREQ: --> SYS - nvRead - {"sysid":1,"itemid":7,"subid":3,"offset":0,"len":12}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:writer: --> frame [254,8,33,51,1,7,0,3,0,0,0,12,19]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: <-- [254,14,97,51,0,12,0,0,0,0,248,166,248,169,17,156,175,132,249]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext [254,14,97,51,0,12,0,0,0,0,248,166,248,169,17,156,175,132,249]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --> parsed 14 - 3 - 1 - 51 - [0,12,0,0,0,0,248,166,248,169,17,156,175,132] - 249
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SRSP: <-- SYS - nvRead - {"status":0,"len":12,"value":{"type":"Buffer","data":[0,0,0,0,248,166,248,169,17,156,175,132]}}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SREQ: --> SYS - nvLength - {"sysid":1,"itemid":7,"subid":4}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:writer: --> frame [254,5,33,50,1,7,0,4,0,20]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: <-- [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext [254,4,97,50,12,0,0,0,91]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --> parsed 4 - 3 - 1 - 50 - [12,0,0,0] - 91
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SRSP: <-- SYS - nvLength - {"len":12}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SREQ: --> SYS - nvRead - {"sysid":1,"itemid":7,"subid":4,"offset":0,"len":12}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:writer: --> frame [254,8,33,51,1,7,0,4,0,0,0,12,20]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: <-- [254,14,97,51,0,12,158,144,20,0,255,255,255,255,255,255,255,255,74]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext [254,14,97,51,0,12,158,144,20,0,255,255,255,255,255,255,255,255,74]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --> parsed 14 - 3 - 1 - 51 - [0,12,158,144,20,0,255,255,255,255,255,255,255,255] - 74
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SRSP: <-- SYS - nvRead - {"status":0,"len":12,"value":{"type":"Buffer","data":[158,144,20,0,255,255,255,255,255,255,255,255]}}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SREQ: --> SYS - nvLength - {"sysid":1,"itemid":7,"subid":5}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:writer: --> frame [254,5,33,50,1,7,0,5,0,21]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: <-- [254,4,97,50,0,0,0,0,87]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext [254,4,97,50,0,0,0,0,87]
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --> parsed 4 - 3 - 1 - 50 - [0,0,0,0] - 87
[2024-07-10 08:41:39] debug: 	zh:zstack:znp: SRSP: <-- SYS - nvLength - {"len":0}
[2024-07-10 08:41:39] debug: 	zh:zstack:unpi:parser: --- parseNext []
[2024-07-10 08:41:39] debug: 	zh:zstack:backup: fetched adapter network security material table (capacity=5, used=5)
[2024-07-10 08:41:39] debug: 	zh:zstack:backup: Following devices with link key are missing from new backup but present in old backup and database, adding them back: 
[2024-07-10 08:41:39] info: 	zh:controller: Wrote coordinator backup to '/homeassistant/zigbee2mqtt/coordinator_backup.json'
[2024-07-10 08:41:39] info: 	zh:zstack:znp: closing
[2024-07-10 08:41:39] info: 	zh:zstack:znp: Port closed
[2024-07-10 08:41:39] info: 	z2m: Stopped zigbee-herdsman
[2024-07-10 08:41:39] info: 	z2m: Stopped Zigbee2MQTT
[08:41:40] INFO: Preparing to start...
[08:41:40] INFO: Socat not enabled
[08:41:41] INFO: Starting Zigbee2MQTT...
[2024-07-10 08:41:43] info: 	z2m: Logging to console, file (filename: log.log)
[2024-07-10 08:41:43] info: 	z2m: Starting Zigbee2MQTT version 1.38.0 (commit #unknown)
[2024-07-10 08:41:43] info: 	z2m: Starting zigbee-herdsman (0.49.2)
[2024-07-10 08:41:44] info: 	zh:zstack:znp: Opening SerialPort with {"path":"/dev/ttyUSB0","baudRate":115200,"rtscts":false,"autoOpen":false}
[2024-07-10 08:41:44] info: 	zh:zstack:znp: Serialport opened
[2024-07-10 08:41:44] info: 	zh:zstack:znp: Writing CC2530/CC2531 skip bootloader payload
[2024-07-10 08:41:45] info: 	zh:zstack:znp: Skip bootloader for CC2652/CC1352
[2024-07-10 08:42:09] info: 	zh:controller: Wrote coordinator backup to '/homeassistant/zigbee2mqtt/coordinator_backup.json'
[2024-07-10 08:42:09] info: 	z2m: zigbee-herdsman started (reset)
[2024-07-10 08:42:09] info: 	z2m: Coordinator firmware version: '{"meta":{"maintrel":1,"majorrel":2,"minorrel":7,"product":1,"revision":20230507,"transportrev":2},"type":"zStack3x0"}'
[2024-07-10 08:42:09] info: 	z2m: Currently 0 devices are joined:
[2024-07-10 08:42:09] info: 	z2m: Zigbee: disabling joining new devices.
[2024-07-10 08:42:09] info: 	z2m:mqtt: Connecting to MQTT server at mqtt://core-mosquitto:1883
[2024-07-10 08:42:09] info: 	z2m:mqtt: Connected to MQTT server
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-10 08:42:09] info: 	z2m: Started frontend on port 8099
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/state', payload '{"state":"online"}'
[2024-07-10 08:42:09] info: 	z2m: Zigbee2MQTT started!
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/0x90395efffee6d2b7/update_available/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/0x287681fffe9cb162/update_available/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/button/0x90395efffee6d2b7/identify/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/button/0x287681fffe9cb162/identify/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/effect/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/power_on_behavior/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x90395efffee6d2b7/identify/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/effect/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/power_on_behavior/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/select/0x287681fffe9cb162/identify/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/power_on_behavior/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/color_options/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/linkquality/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x90395efffee6d2b7/update_state/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/power_on_behavior/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/color_options/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/linkquality/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/sensor/0x287681fffe9cb162/update_state/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/light/0x90395efffee6d2b7/light/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/light/0x287681fffe9cb162/light/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/update/0x90395efffee6d2b7/update/config', payload 'null'
[2024-07-10 08:42:09] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/update/0x287681fffe9cb162/update/config', payload 'null'
[2024-07-10 08:42:14] info: 	z2m:mqtt: MQTT publish: topic 'homeassistant/binary_sensor/1221051039810110150109113116116_0x00124b00259bb8b4/connection_state/config', payload '{"device":{"hw_version":"zStack3x0 20230507","identifiers":["zigbee2mqtt_bridge_0x00124b00259bb8b4"],"manufacturer":"Zigbee2MQTT","model":"Bridge","name":"Zigbee2MQTT Bridge","sw_version":"1.38.0"},"device_class":"connectivity","entity_category":"diagnostic","name":"Connection state","object_id":"zigbee2mqtt_bridge_connection_state","origin":{"name":"Zigbee2MQTT","sw":"1.38.0","url":"https://www.zigbee2mqtt.io"},"payload_off":"offline","payload_on":"online","state_topic":"zigbee2mqtt/bridge/state","unique_id":"bridge_0x00124b00259bb8b4_connection_state_zigbee2mqtt","value_template":"{{ value_json.state }}"}'
[2024-07-10 08:43:02] info: 	z2m: Successfully changed options
[2024-07-10 08:43:02] info: 	z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/bridge/response/options', payload '{"data":{"restart_required":false},"status":"ok","transaction":"fsygo-1"}'

lecourtb · 2024-07-11T08:09:32Z

I noticed that in Z2M MQTT parameters, the base topic which was "zigbee2mqtt" before upgrade was renamed to "zigbee2mqttHA" after upgrade.

I reproduced two times the problem (thanks to ha backups) and give up try to upgrade !

Later, I also tried a third upgrade positioning base_topic: zigbee2mqtt in configuration.yaml but without success.

Related to #23294

Hope it will help

Koenkk · 2024-07-11T19:28:14Z

@lecourtb probably you have set base_topic: zigbee2mqttHA in the HA addon configuration page, this should be set correctly (zigbee2mqtt in your case)

Aseg · 2024-07-12T19:55:25Z

I have the same issue, I cant update to 1.39 without break my devices and automations because devices duplicate with suffix '_2'

lecourtb · 2024-07-17T09:13:11Z

That was effectively the case (base_topic: zigbee2mqttHA set in the HA addon configuration page)
I will try to upgrade Z2M as soon as possible after the change.

dystechnic · 2024-07-19T10:29:40Z

Same issue here.
Restoring a backup didn't help. I ended up deleting and renaming devices. I'm afraid to reboot HA...

dermotos · 2024-08-02T07:24:29Z

Has anyone with this issue upgraded to 1.39.1 yet? If so did this issue persist? I don't see anything in the release notes that suggests this issue has been fixed yet.

thomasknuchel · 2024-08-02T09:59:45Z

Tested and issue is still persisting

Wendigogo · 2024-08-02T12:24:47Z

You should take a look at #23294 .
I'm pretty sure the given solution will work here.

Enjoy!

Najihel · 2024-08-02T19:39:40Z

You should take a look at #23294 .
I'm pretty sure the given solution will work here.

Enjoy!

Yes it is ! Thanks

raffoul · 2024-08-04T14:17:35Z

Rename base topic to zigbee2mqtt works for me. Thank you.

paavokurkela · 2024-08-04T18:20:23Z

I got my system back to working state.

I deleted z2m completely as well all files and folders.
I reinstalled z2m (1.39.1) and used my old configuration. Z2M did not create any folders or files, not even zigbee2mqtt which I created then manually. On my old config I had /homeassistant/zigbee2mqtt which I changed to /config/zigbee2mqtt which seemed to make a trick and Z2M begun to write files. This is of course very odd as /config is simply a symlink to /homeassistant folder.

This or that my system works again.

jmcollin78 · 2024-08-05T13:16:40Z

Renaming base_topic to zigbee2mqtt and installing yhe 1.19.1 resolves the issue for me.

Koenkk · 2024-08-05T20:06:42Z

Removing the base_topic from the Z2M HA addon should fix this indeed.

kefrens · 2024-09-05T08:31:18Z

For me doesn't work as well.
In the addon config I have:
base_topic: core-mosquitto

In the config of Zigbee2MQTT I have as well:
base_topic: core-mosquitto

But it still creates all entities with a '_2'

kefrens · 2024-09-06T11:46:45Z

For me doesn't work as well. In the addon config I have: base_topic: core-mosquitto

In the config of Zigbee2MQTT I have as well: base_topic: core-mosquitto

But it still creates all entities with a '_2'

Responding to myself.

I had to delete in the the addon configuration.yaml the base topic.
make sure base topic in the addon and on the zigbee2mqtt website is the same
reboot the system (stop/start wasn't enough)

gijsh070 · 2024-09-07T13:00:21Z

Thanks a lot @kefrens

This solved my issue finally. They need to match the addon base_topic and mqtt site.

gijsh070 · 2024-09-07T19:17:42Z

Anyone noticing that triggers somehow are named differently after the latest update? I get a lot of uknown triggers suddenly after updating.

paavokurkela added the problem Something isn't working label Jul 2, 2024

Spb2005 mentioned this issue Jul 6, 2024

No access to frontend since 1.39.0 upgrade #23266

Closed

Wendigogo mentioned this issue Aug 2, 2024

Last upgrade(s) rename base_topic, entities are duplicated #23294

Closed

Koenkk closed this as completed Aug 5, 2024

All devices are lost after Z2M update #23235

All devices are lost after Z2M update #23235

Comments

paavokurkela commented Jul 2, 2024

What happened?

What did you expect to happen?

How to reproduce it (minimal and precise)

Zigbee2MQTT version

Adapter firmware version

Adapter

Setup

Debug log

kligithu commented Jul 2, 2024

raffoul commented Jul 2, 2024

Webmonsteril commented Jul 2, 2024

Najihel commented Jul 2, 2024

jmcollin78 commented Jul 2, 2024 • edited Loading

Najihel commented Jul 3, 2024 • edited Loading

gijsh070 commented Jul 3, 2024 • edited Loading

countrr commented Jul 3, 2024

Zver5 commented Jul 3, 2024

raphmur commented Jul 4, 2024 • edited Loading

kefrens commented Jul 4, 2024

raphmur commented Jul 4, 2024

Vidjer commented Jul 5, 2024

paavokurkela commented Jul 5, 2024 via email

raphmur commented Jul 5, 2024

jmcollin78 commented Jul 5, 2024

gijsh070 commented Jul 5, 2024

paavokurkela commented Jul 5, 2024 • edited Loading

Koenkk commented Jul 7, 2024

paavokurkela commented Jul 7, 2024 • edited Loading

Couple devices paired to 1.38

Devices gone 1.39

HA version:

configuration.yaml (with debugs enabled)

Add-on config screen (credentials hidden)

log (1.38 with two Ikea lights paired)

log (startup during upgrade process to 1.39 and all devices lost)

dcyou commented Jul 9, 2024

jmcollin78 commented Jul 9, 2024

Koenkk commented Jul 9, 2024

paavokurkela commented Jul 10, 2024

lecourtb commented Jul 11, 2024 • edited Loading

Koenkk commented Jul 11, 2024

Aseg commented Jul 12, 2024

lecourtb commented Jul 17, 2024

dystechnic commented Jul 19, 2024

dermotos commented Aug 2, 2024

thomasknuchel commented Aug 2, 2024

Wendigogo commented Aug 2, 2024

Najihel commented Aug 2, 2024

raffoul commented Aug 4, 2024

paavokurkela commented Aug 4, 2024

jmcollin78 commented Aug 5, 2024

Koenkk commented Aug 5, 2024

kefrens commented Sep 5, 2024

kefrens commented Sep 6, 2024

gijsh070 commented Sep 7, 2024

gijsh070 commented Sep 7, 2024

jmcollin78 commented Jul 2, 2024 •

edited

Loading

Najihel commented Jul 3, 2024 •

edited

Loading

gijsh070 commented Jul 3, 2024 •

edited

Loading

raphmur commented Jul 4, 2024 •

edited

Loading

paavokurkela commented Jul 5, 2024 •

edited

Loading

paavokurkela commented Jul 7, 2024 •

edited

Loading

lecourtb commented Jul 11, 2024 •

edited

Loading