Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Problem]: After switching to CC2652 coordinator, my network works perfectly, but no device will pair with it. #10339

Closed
convicte opened this issue Dec 23, 2021 · 74 comments
Labels
problem Something isn't working stale Stale issues

Comments

@convicte
Copy link

What happened?

I transitioned from an old and weak CC2531, which worked ok, but with 40 devices connected started to misbehave, lose connection to devices, etc. Unfortunately, upon switching to CC2652 there is no way to pair any new device (IKEA, XIAOMI, TUYA, etc.). The network rebuilt perfectly fine after the coordinator change, and is updating sensors just fine, but will not execute a pairing process.

I've changed back and forth between different coordinator firmwares, to no effect.
Unplugging the coordinator, restarting the VM in which HA is running, neither restarting HA made any effect.

What did you expect to happen?

When pairing is initiated, the interview process begins and the device joins the network as per the old CC2531.

How to reproduce it (minimal and precise)

  1. Connect the next adapter to the network
  2. Permit_join enabled
  3. Put device in pairing mode
  4. Wait until the device times out because nothing will happen

Zigbee2MQTT version

1.22.1

Adapter firmware version

20211217, 20210319, 20210708

Adapter

Ebyte CC2652P

Debug log

How to record a debug of something that doesn't happen? I've been monitoring the logs and while the permit_join is enabled, no interview process is initiated. The device sits idle until it times-out.

@convicte convicte added the problem Something isn't working label Dec 23, 2021
@Koenkk
Copy link
Owner

Koenkk commented Dec 23, 2021

try with the fw from https://github.com/Koenkk/Z-Stack-firmware/tree/develop/coordinator/Z-Stack_3.x.0/bin

@convicte
Copy link
Author

I am on it now - no luck. The exact same behavior as on the two others.

The network works great, but pairing is completely dead.
image

@henkiejan1
Copy link

henkiejan1 commented Dec 23, 2021

Hmm, i have recently switched to the CC2652 also but no problems at all with repairing some devices. Did you delete devices who exist in your network and try to add again? If its not a new device maybe exist the device just in your databse.db?

@Tonio16
Copy link

Tonio16 commented Dec 24, 2021

Hello

I switched from a zigate to a sonofff itead yesterday. No pairing issue with this firmware : 20210120

Antoine

@Koenkk
Copy link
Owner

Koenkk commented Dec 24, 2021

@convicte

@convicte
Copy link
Author

convicte commented Dec 25, 2021

@Koenkk

Thanks for the update and my apologies for the late reply.

  1. If I understand correctly from the link attached, you are asking to use the old CC2531 I have, with a sniff firmware plus a windows tool to collect a sniff of the network traffic at my selected channel - is that correct?
  2. Please see the backup below (minus the keys - obviously).
    image

One more data point for the evaluation; I've been trying to pair another IKEA Dimmer and held the reset button for about 10 seconds, while in the room with a Zigbee floor lamp (Office floor lamp). At the same time, this lamp was kicked out of my network with the following log output and can't be repaired, because nothing else can be.
chrome_6QA8yAcywg

Not sure if a fluke or related, but haven't had this happen ever.
EDIT: It must be in some way connected because keeping the button on the IKEA dimmer pressed for 10 seconds again is affecting the lamp (it starts 'breathing' in brightness as if it would enter pairing mode). After about 30 seconds, it fails and goes back to solid light. It's a Lidl floor lamp which is recognized by z2m as a LED strip controller, but works just fine.
image

@Koenkk
Copy link
Owner

Koenkk commented Dec 26, 2021

If I understand correctly from the link attached, you are asking to use the old CC2531 I have, with a sniff firmware plus a windows tool to collect a sniff of the network traffic at my selected channel - is that correct?

Yes

I've been trying to pair another IKEA Dimmer and held the reset button for about 10 seconds, while in the room with a Zigbee floor lamp (Office floor lamp).

See https://www.zigbee2mqtt.io/devices/ICTC-G-1.html#pairing how to reset this device, when you hold it 10 seconds it will setup a new network with just the dimmer and the lamp

@convicte
Copy link
Author

If I understand correctly from the link attached, you are asking to use the old CC2531 I have, with a sniff firmware plus a windows tool to collect a sniff of the network traffic at my selected channel - is that correct?

Yes

My sincere apologies, but I am failing at providing a sniff. I've spent quite some time to get the CC2531 set up as the sniffer tool, but having no CC debugger and unable to find a way to flash it via USB as I did for the 2653 using the TI Flasher V2 software, I am a bit stuck.
I've downloaded the ZBOSS V.2 since V1.0 doesn't provide the sniffer.hex firmware file, but can't get further than this.
Some guidance would be greatly appreciated!

Is there any chance my backup file gave some hints?

For your kind consideration, my z2m config as well:
image
Please note, I change the permit join from the HA interface, but I wonder if I should set it permanently to true in the config and switch off from the interface (or vice versa)

I've been trying to pair another IKEA Dimmer and held the reset button for about 10 seconds, while in the room with a Zigbee floor lamp (Office floor lamp).

See https://www.zigbee2mqtt.io/devices/ICTC-G-1.html#pairing how to reset this device, when you hold it 10 seconds it will setup a new network with just the dimmer and the lamp

There may have been a misunderstanding in that I was not trying to pair it this way. I know it's to be reset by pressing the button 4 times - https://www.zigbee2mqtt.io/devices/E1743.html (it's actually this IKEA dimmer/switch).
In my desperation, I was trying different things to make the network recognize it.

All this said, I am still unable to explain why getting the IKEA button into it's direct pair mode would kick out a different brand device from the network, as seen above.

When I fix pairing issues I'll play with this, but for the moment getting something to pair with my network is a priority.

@Koenkk
Copy link
Owner

Koenkk commented Dec 27, 2021

@convicte unfortunately I need a sniff to debug this further, one thing you can try is to pair the dimmer when holding it very close to the coordinator (to ensure it tries to pair via that)

@Fabiancrg
Copy link

@Koenkk I had some issue myself recently to pair devices.
The only way to do it was to pair them close to the coordinator (<1m), when I was too far it was not working anymore.
I experienced it just now with two devices, one Aqara contact sensor and one Tuya roller shutter I was not able to pair.

I tried the new FW on my CC2652R and the pairing worked just fine form their place (not close to the coordinator).

@convicte
Copy link
Author

convicte commented Jan 1, 2022

@Koenkk
My apologies for the late reply. It was a busy week.

  1. In desperation, having no way to flash the sniffer I borrowed a Sonoff 3.0 dongle which was running an entirely different firmware from July 2021, and it exhibited the same exact behavior - perfect network performance but inability to pair devices, so it's unlikely to be my hardware
  2. I tested 5 different devices, both holding it next to the coordinator and in an entirely different part of the house, via routers. Did not make any difference. I once managed to get the device associated with the network, but it failed to recognize and when removed never associated again, like the others.
  3. I've also changed the USB extension, which is used to get the dongle outside my server rack, for better reception. Two different extensions didn't make a difference.

Finally, I started looking for ways how to flash the sniffer with available hardware I had and found an old PI and instructions how to use it to flash a CC2532, which worked!!

I now have a sniffer and managed to follow your instructions to grab a paring process via routers with an IKEA button listed above. My only concern is with posting it, hence it will contain network keys. Is there any way to share it with you securely?
Second, I can grab another pairing process right next to the coordinator, if this is of interest?

Thank you!!

@Koenkk
Copy link
Owner

Koenkk commented Jan 1, 2022

Second, I can grab another pairing process right next to the coordinator, if this is of interest?

yes

You can send the network key to me on telegram (@Koenkk)

@convicte
Copy link
Author

convicte commented Jan 1, 2022

Thank you!

I just texted you on Telegram with the key.

Pairing attempt via router.zip
Pairing attempt via coordinator.zip

Both files as requested!

@Koenkk
Copy link
Owner

Koenkk commented Jan 2, 2022

I've checked the coordinator sniff, somehow the joining device isn't doing an association request. But while checking your sniff I found another issue. It seems the coordinator is using a different epid than the rest of the network leading to pan conflicts.

Screenshot 2022-01-02 at 11 21 16

@castorw my first thought is that this is caused by the cc2531 returning the wrong epid (0xdddd..), during the migration we write the wrong epid into the cc2652, could that be the case here?

@convicte
Copy link
Author

convicte commented Jan 2, 2022

Thank you! I've seen this conflict in the sniff but failed to recognise the significance.
I've never touched the panID during the migration but I am wondering how it could get changed if it's listed in the z2m configuration, which should have been burned in to the CC2652 upon first boot. Can I change it now on the CC2652 or attempt to correct it when the backup is loaded onto a fresh CC2652 (after flash wipe)?

Just in case you are wondering I could run another sniff with a different device entirely, to see if it's also not sending out a pairing request.
This said, I am at a loss why changing the coordinator would case the endpoints not to request pairing.

My apologies if these are trivial questions.

@Koenkk
Copy link
Owner

Koenkk commented Jan 2, 2022

@convicte can you check if the extended_pan_id in the coordinator_backup.json is also dddddddd...? If yes, change it to the one starting with 95e7 in the sniff, reflash your coordinator and start z2m might solve this problem.

@convicte
Copy link
Author

convicte commented Jan 2, 2022

Yes, indeed it is!
image

I'll change it in a moment. By reflash you mean the latest 20211217 firmware, to clear the 'old' backup on the coordinator and make it pull the new one with the correct extended_pan_id from the .json file?

Is that correct?

@Koenkk
Copy link
Owner

Koenkk commented Jan 2, 2022

  • stop z2m
  • update extended_pan_id in the coordinator_backup.json
  • reflash with 20211217
  • start z2m

@convicte
Copy link
Author

convicte commented Jan 2, 2022

Perfect, I did half of that already, so just need to run into the attic to get the stick down, to reflash it.

I'll update everyone ASAP.

@convicte
Copy link
Author

convicte commented Jan 2, 2022

Quick update:

  1. Following the changes, there is definitely progress, but it's not completely resolved yet. I am predicting you will request another sniff of the situation as it stands.
  2. When standing right next to the coordinator, the same IKEA remote successfully announced itself, and started the joining process, but eventually left the network. This happened twice, with the third try resulting in the remote remaining in the network but as an unsupported device. Below see the excerpt from the z2m logs of one such event:

debug 2022-01-02 21:52:09: Received MQTT message on 'zigbee2mqtt/bridge/request/permit_join' with data '{"device":null,"time":254,"transaction":"7fshf-1","value":true}' info 2022-01-02 21:52:09: Zigbee: allowing new devices to join. info 2022-01-02 21:52:09: MQTT publish: topic 'zigbee2mqtt/bridge/response/permit_join', payload '{"data":{"time":254,"value":true},"status":"ok","transaction":"7fshf-1"}' info 2022-01-02 21:52:25: Device '0xccccccfffe020fd9' joined info 2022-01-02 21:52:25: MQTT publish: topic 'zigbee2mqtt/bridge/event', payload '{"data":{"friendly_name":"0xccccccfffe020fd9","ieee_address":"0xccccccfffe020fd9"},"type":"device_joined"}' info 2022-01-02 21:52:25: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":{"friendly_name":"0xccccccfffe020fd9"},"type":"device_connected"}' info 2022-01-02 21:52:25: MQTT publish: topic 'zigbee2mqtt/0xccccccfffe020fd9/availability', payload 'online' info 2022-01-02 21:52:25: MQTT publish: topic 'zigbee2mqtt/0xccccccfffe020fd9', payload '{"last_seen":"2022-01-02T21:52:25+01:00"}' info 2022-01-02 21:52:25: Starting interview of '0xccccccfffe020fd9' info 2022-01-02 21:52:25: MQTT publish: topic 'zigbee2mqtt/bridge/event', payload '{"data":{"friendly_name":"0xccccccfffe020fd9","ieee_address":"0xccccccfffe020fd9","status":"started"},"type":"device_interview"}' info 2022-01-02 21:52:25: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"interview_started","meta":{"friendly_name":"0xccccccfffe020fd9"},"type":"pairing"}' info 2022-01-02 21:52:27: MQTT publish: topic 'zigbee2mqtt/0xccccccfffe020fd9', payload '{"last_seen":"2022-01-02T21:52:27+01:00"}' info 2022-01-02 21:52:27: MQTT publish: topic 'zigbee2mqtt/0xccccccfffe020fd9', payload '{"last_seen":"2022-01-02T21:52:27+01:00"}' debug 2022-01-02 21:52:27: Device '0xccccccfffe020fd9' announced itself info 2022-01-02 21:52:27: MQTT publish: topic 'zigbee2mqtt/bridge/event', payload '{"data":{"friendly_name":"0xccccccfffe020fd9","ieee_address":"0xccccccfffe020fd9"},"type":"device_announce"}' info 2022-01-02 21:52:27: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"announce","meta":{"friendly_name":"0xccccccfffe020fd9"},"type":"device_announced"}' warn 2022-01-02 21:52:36: Device '0xccccccfffe020fd9' left the network

  1. Trying to achieve the same away from the coordinator amounts to nothing - no messages or response from the interface when the device is in pairing mode. Unable to pair any other device, either, when away from the coordinator.
  2. The whole network restarted and once again is working perfectly as is.

Let me guess, a sniff of pairing next to the coordinator and one via routers? ;)

Cheers!!

@Koenkk
Copy link
Owner

Koenkk commented Jan 2, 2022

Can you provide a sniff when pairing a non battery powered device (something like a switch or light) via a router?

@convicte
Copy link
Author

convicte commented Jan 2, 2022

Absolutely!

Please see pairing attempts of the same lamp as listed above (Lidl Livarno).
It seems to be stuck in a pairing loop where association is successful, but it's actually not showing up in the interface at all.
No messages or iconography is depicted in the z2m interface at all. It's as if the process never happened, as far as the HA interface is concerned.

Pairing attempt via router for a floor lamp.zip

Same passwords as yesterday.

@fuglphoenix
Copy link

@convicte, This sounds a lot like my problem.

I have a Slaesh's CC2652RB Coordinator and just updated to 20211217.
I haven't migrated, but did a fresh setup when i recieved the stick.
z2m version is 1.22.2

i'm trying to pair my devices (ikea, Tuya and others) via a ikea router, since they are a bit far away from the coordinator.
it seems to work since the devices are stopping the light blinking sequence (join mode) but when i look in z2m no device are showing up.

if i disable join in z2m and turn the device off and on. It will revert back to the light blinking join mode.
if i move them, i can pair them directly to the coordinator

unfortunately i haven't got any spare zigbee stick to sniff what is happening.

@vadimag1
Copy link

vadimag1 commented Jan 5, 2022

I have exactly the same problem. Devices are not connected to the coordinator.

@cracrama
Copy link

cracrama commented Jan 6, 2022

Hmm similiar situation with my network - after flashing 20211217 on Ebyte CC2652P I have strange behaviour from couple of sonoff devices - temp humidity sensors, which were disconected from the network after flashing firmware.. I can re-pair them but after pairing they not reporting anything and finally disconectiong from network. However one of these devices - same model still works and reporting correctly - that was device which stayed connected after flashing. Other devices pair / report correctly. I don't know what happened and that behaviour is really strange.

@Coax88
Copy link

Coax88 commented Jan 7, 2022

Hi
Im experience the same issue after migrate from cc2531 to cc2652p, before I could pair without any problems, after upgrading i have issue parring and need alot of times take device close to the sonoff cc2652p before it pairs.

@xrust83
Copy link

xrust83 commented Jan 9, 2022

The problem is more global and does not seem to be related to the adapter firmware. I'm still using сс2531 I flashed on the latest firmware Z-Stack_Home_1.2 20211115/20211116 got the same result it seems to work but I can not connect a new or remote old device. If the problem was only in the firmware, then rollback and the old version of Z-Stack_Home_1.2 20201127/20201128 would return everything back, but it does not happen, so the reason is in the version itself zigbee2mqtt

@convicte
Copy link
Author

@Fabiancrg, if you don't have a conflict, I would assume everything is fine, as long as all your devices are on the same EPID.
Outside that I must defer to @Koenkk since it's beyond my expertise.

@LCerebo I am glad the troubleshooting steps worked for you!
In my case, all endpoint or router issues were resolved after forcibly removing them from the network and repairing.
If these are not able to repair, are you also having the other problem where the router you try to pair to is not recognized in the backup file?

@Fabiancrg
Copy link

Fabiancrg commented Jan 25, 2022

@convicte All my issues were fixed when I applied your procedure and removed/paired all routers that were not in the coordinator_backup.json file.

So I would like to help others and create a script that will scan three files:

  1. configuration.yaml
  2. database.db
  3. coordinator_backup.json

The output will list all missing routers with the friendly name, those ones will need to be removed/paired again
Missing end devices with friendly names, same actions to perform but as a second steps as some might come back after pairing the missing routers.
Extra devices... I currently don't know what to do with these, I can't force-remove them.

@Koenkk is there a way to force the generation of the coordinator_backup.json without restarting Z2M ? I think it will be better if the script can run without causing any downtime of Z2M.

Any idea if the conflict EPID could be detected without snifing the Zigbee network ?

@LCerebo
Copy link

LCerebo commented Jan 25, 2022

@LCerebo I am glad the troubleshooting steps worked for you! In my case, all endpoint or router issues were resolved after forcibly removing them from the network and repairing. If these are not able to repair, are you also having the other problem where the router you try to pair to is not recognized in the backup file?

Yes, all the routers that use the empty pan_id "dddddddddddddddd" aren't listed in the coordinator_backup.json.
I tried to repair them but they are listed as "unsupported devices". I don't know what to do to fix the pan_id conflict.

@convicte
Copy link
Author

Sorry to sound like a broken record, but you have to follow the steps I outlined above.
If the routers are not listed, they are not considered in the pairing process, and thus your network will be handicapped until this is resolved.

Review the routers needing to be repaired, starting from the one closest to the coordinator, and take it from there. In the meantime, please reflash the coordinator as indicated above.

If you have any specific questions or concerns, please let us know. Hopefully someone will be able to help if I fail to do so.

@fuglphoenix
Copy link

I can confirm that i had the same problem with the EPID.
in my configuration it wasn't set to anything and it then defaulted to dddddd.... which I confirmed in Settings->RAW
but for some reason it was changed in my coordinator_backup.json to the same value as my coordinator_ieee ?!
I proceeded to stop Z2M, changed the EPID in my coordinator_backup.json reflashed with CC2652RB_coordinator_20211217 and started Z2M again.
now it works and I can pair my devices through a router which was my initial problem

@MartB
Copy link

MartB commented Feb 3, 2022

Edit: I did it again and now it sticks correctly.

Can someone please tell me in simple terms what my EPID is supposed to be and how i get it to stick?

Config:

"ext_pan_id": [
                221,
                221,
                221,
                221,
                221,
                221,
                221,
                221
            ],

Coordinator backup:

"extended_pan_id": "00124RESTOFIEEE",

I did what @fuglphoenix did, but reflashing does nothing, extended_pan_id keeps being my coordinator ieee even if i change the backup and re-flash.

@arjanvanleent
Copy link

Hello my "extended_pan_id": "dddddddddd", and not all end device are in the backup.
How do I find the correct "extended_pan_id" ?
Don't have another stick to snif
Is it possible to use another installation?

  • clean the stick by reflashing it
  • Put it in another installation pair 1 end device
  • Check the new backup for the correct "extended_pan_id":
    Can this work also?

@drogfild
Copy link

drogfild commented Feb 9, 2022

My story here. I could not pair new devices reliably, especially Ikea E1743 were failing. Some router devices behaved strange during joining; they removed them selves from the network after interview.

My coordinator is Smartlight CC2652P, don't have original fw version info anymore but it was stock from manufacturer.

I did EPID change to dddddddddddddddd with this guide #10339 (comment)
Also did those two reflash with 20211217
That dddd... is my guess as I don't have siffing device. But previously I was CC2531 so I took a wild guess.

That didn't seem to resolve any of my pairing issues. I then flashed dev version CC1352P2_CC2652P_other_coordinator_20220103

At least for now that seems to resolve my issues. I was able to pair those Ikea E1743 and all other devices seem to behave now correctly. This as an information to all others.

@convicte
Copy link
Author

convicte commented Feb 9, 2022

@drogfild
Not sure if I understand you correctly, but changing from something TO dddd... is the exact opposite of my instructions.
If I was not clear, my apologies.
Considering it seems to work, you may have done something that helped, but it doesn't seem to be similar to my situation.
If I were you, I would look into the backup file, to see if the routers are there, but that's just me.
@arjanvanleent
I am sorry, but I cannot help in this regard.
The sniff to look for the EPID was the only route I was informed of.

@drogfild
Copy link

drogfild commented Feb 9, 2022

Thanks @convicte . In my case I had routers in the backup file before all this and they are there also afterwards. Haven't checked if all of my routers are there. Maybe I should check that.

It might be that all jumps and hoops with EPID was not needed for me. And new FW alone would have done the needed corrections for me. But as I don't have a sniffer I'm not able to tell. It was wild guess and maybe it was correct or maybe it wasn't, but I'm happy now that everything seems to be working.

There's pretty similar case about EPID with dddd.. at #9117 (comment)

It would be really cool to have some other way to see EPID conflicts than sniffing them. But that's most probably a limitation of zigbee network structure.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 2, 2022

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days

@github-actions github-actions bot added the stale Stale issues label Apr 2, 2022
@github-actions github-actions bot closed this as completed Apr 9, 2022
@mihaimdinca
Copy link

I have the same issue with the CC2652P. Everything pairs with it but not with any other router in my ZigBee network. I found the discussion here pretty confusing and I was hoping that someone who actually managed to fix the problem could wrap things up in a sort of step by step tutorial for us noobs. Thank you in advance.

@Fabiancrg
Copy link

Fabiancrg commented Jul 10, 2022

@mihaimdinca, if you have the same problem as I had, t's due to some router devices that are not know correctly by the coordinator.
They might be working properly, e.g.: a bulb might be working but not forwarding the messages to the coordinator.
You can try to remove and pair every router device one by one or look for the one(s) not defined correctly.
For this, I did the python script bellow, you can execute it and it will list the routers that you will need to remove and pair again.

Copy the following code in Z2McheckDevices.py and execute it using python3 Z2McheckDevices.py

import json
import yaml


devices = {}     # Dictionnary with all devices
missingRouters = {}
missingEndDevices = {}
additionalDevices = {}

backupfile = open('/opt/zigbee2mqtt/data/coordinator_backup.json',"r")

DBfile = open('/opt/zigbee2mqtt/data/database.db', 'r')
Lines = DBfile.readlines()


# Parse database.db to retrive all devices IEEE address and type define in Z2M

print('Parsing database.db file')

for line in Lines:
    DBdata = json.loads(line)
    if "ieeeAddr" in DBdata and "type" in DBdata:
        if DBdata ["type"] != 'Coordinator':
            devices[DBdata["ieeeAddr"]] = [DBdata ["type"], "", False]
            #print(DBdata["ieeeAddr"] + " is a " + DBdata ["type"])

DBfile.close()
print('Parsing database.db file... Done')

# Parse coordinator_backup.json to retreive all IEEE present in coordinator

data = json.loads(backupfile.read())


for device in data['devices']:
     dev = devices.get("0x" + device['ieee_address'])
     if dev:
         devices["0x" + device['ieee_address']][2] = True
     else:
         devices["0x" + device['ieee_address']]=["Unknown","", True]

backupfile.close()
print('Parsing backup file... Done')

# Retreive friendly name from configuration.yaml

configfile = open('/opt/zigbee2mqtt/data/configuration.yaml',"r")

configData = yaml.safe_load(configfile)
configDevices = configData["devices"]


for key,value in devices.items():
    dev = configDevices.get(key)
    if dev:
        devices[key][1]=configDevices[key]["friendly_name"]
    else:
        devices[key][1]="Unknown"
    #print (key , '-->', value)

configfile.close()

# Print summary
for key,value in devices.items():
    if devices[key][0] == "Router" and not devices[key][2]:
        missingRouters[key] = devices[key][1]
    elif devices[key][0] == "EndDevice" and not devices[key][2]:
        missingEndDevices[key] = devices[key][1]
    if devices[key][0] == "Unknown":
        additionalDevices[key] = devices[key][1]

if len(missingRouters) > 0:
    print ("The following routers are not correctly set up, please remove them from Z2M config and execute the pairing operation again")
    for key, value in missingRouters.items():
        print ("\t" + key + " - " + value)
    if len(missingEndDevices) > 0:
        print ("Some end devices are missing too but you must first fix the routers")
elif len(missingEndDevices) > 0:
    print ("The following devices are not correctly set up, please remove them from Z2M config and execute the pairing operation again")
    for key, value in missingEndDevices.items():
        print ("\t" + key + " - " + value)
else :
   print ("Routers and end devices are correctly setup on the controller")

if len(additionalDevices) > 0:
   print ("Some devices are knwown by the controller but not present on your Zigbee network")
   for key, value in additionalDevices.items():
       print ("\t" + key)

PS: you will have to install python with the json and yaml modules.
I did this script for Z2M running on Linux, if you are using Windows, you will have to update the paths
PPS: make sure you have a recent coordinator_backup.json file, if you have the last Z2M version, this file is created every few hours, if you are running an older version, please restart Z2M before starting the script to trigger a backup (or upgrade to the latest Z2M version)

@mihaimdinca
Copy link

mihaimdinca commented Jul 10, 2022 via email

@RuinedOne
Copy link

@Fabiancrg Thanks for your script. I was going nuts trying to figure out why nothing would route/pair w/anything but the coordinator (Sonoff CC2652) and this solved my issue.

@odelma
Copy link

odelma commented Jan 4, 2023

Thanks for the script 😃
How should I fix: "Some devices are known by the controller but not present on your Zigbee network" - I guess they are only in the database, but I am not sure how I should remove them....?

@Fabiancrg
Copy link

Hi
I put this for information but I don't now how to clean these devices.
I am not sure it is causing any issue so I did not investigate further.

@rhvs
Copy link

rhvs commented Aug 8, 2023

@Fabiancrg

I tried to edit your code to use it directly in HA OS.... but no luck (I have to admit it was a long shot :) ). Would be great if you could have a look!

import json
import yaml


devices = {}     # Dictionnary with all devices
missingRouters = {}
missingEndDevices = {}
additionalDevices = {}

backupfile = open('/config/zigbee2mqtt/coordinator_backup.json',"r")

DBfile = open('/config/zigbee2mqtt/database.db', 'r')
Lines = DBfile.readlines()


# Parse database.db to retrive all devices IEEE address and type define in Z2M

print('Parsing database.db file')

for line in Lines:
    DBdata = json.loads(line)
    if "ieeeAddr" in DBdata and "type" in DBdata:
        if DBdata ["type"] != 'Coordinator':
            devices[DBdata["ieeeAddr"]] = [DBdata ["type"], "", False]
            #print(DBdata["ieeeAddr"] + " is a " + DBdata ["type"])

DBfile.close()
print('Parsing database.db file... Done')

# Parse coordinator_backup.json to retreive all IEEE present in coordinator

data = json.loads(backupfile.read())


for device in data['devices']:
     dev = devices.get("0x" + device['ieee_address'])
     if dev:
         devices["0x" + device['ieee_address']][2] = True
     else:
         devices["0x" + device['ieee_address']]=["Unknown","", True]

backupfile.close()
print('Parsing backup file... Done')

# Retreive friendly name from configuration.yaml

configfile = open('/config/zigbee2mqtt/configuration.yaml',"r")

configData = yaml.safe_load(configfile)
configDevices = configData["devices"]

for key,value in devices.items():
    dev = configDevices.get(key)
    if dev:
        devices[key][1]=configDevices[key]["friendly_name"]
    else:
        devices[key][1]="Unknown"
    #print (key , '-->', value)

configfile.close()

# Print summary
for key,value in devices.items():
    if devices[key][0] == "Router" and not devices[key][2]:
        missingRouters[key] = devices[key][1]
    elif devices[key][0] == "EndDevice" and not devices[key][2]:
        missingEndDevices[key] = devices[key][1]
    if devices[key][0] == "Unknown":
        additionalDevices[key] = devices[key][1]

if len(missingRouters) > 0:
    print ("The following routers are not correctly set up, please remove them from Z2M config and execute the pairing op>
    for key, value in missingRouters.items():
        print ("\t" + key + " - " + value)
    if len(missingEndDevices) > 0:
        print ("Some end devices are missing too but you must first fix the routers")
elif len(missingEndDevices) > 0:
    print ("The following devices are not correctly set up, please remove them from Z2M config and execute the pairing op>
    for key, value in missingEndDevices.items():
        print ("\t" + key + " - " + value)
else :
   print ("Routers and end devices are correctly setup on the controller")

if len(additionalDevices) > 0:
   print ("Some devices are knwown by the controller but not present on your Zigbee network")
   for key, value in additionalDevices.items():
       print ("\t" + key)

@MnM001
Copy link

MnM001 commented Aug 11, 2023

Hi,

@Fabiancrg I have tried this script with my Sonoff ZB Dongle-E.

It gave me a list of routers that needed to be "fixed". That list looked suspiciously like all the routers I have in my environment.
So I took one router out (held the button until it left my z2m environment). Requested a new backup and then I run the script again. That router was gone from the list of routers that needed to be "fixed".

I waited a few minutes and then added the router back.

After I requested a new backup I run the script again and the router that I removed and re-added again is in the list of routers that needs to be "fixed".

How can that be? I just removed it (by button so the network key was deleted) and then add it back. It should had been fixed I assume.

Is the script not working as it should with the Sonoff ZB Dongle-E?
Or I am doing something wrong?

@Koenkk
Copy link
Owner

Koenkk commented Aug 14, 2023

The check is now supported directly from Zigbee2MQTT: #18599

@Koenkk Koenkk unpinned this issue Feb 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
problem Something isn't working stale Stale issues
Projects
None yet
Development

No branches or pull requests