[Signify SML004] Illuminance not working correctly

[peterforeman]

What happened?

After updating to z2m latest-dev from the stable version my illuminance sensors have changed.

z2m:mqtt: MQTT publish: topic 'zigbee2mqtt/motion_tuin', payload '{"battery":100,"device":{"applicationVersion":2,"dateCode":"20210830","friendlyName":"motion_tuin","hardwareVersion":1,"ieeeAddr":"0x001788010cdcb3a6","manufacturerID":4107,"manufacturerName":"Signify Netherlands B.V.","model":"9290030674","networkAddress":7082,"powerSource":"Battery","softwareBuildID":"2.53.6","stackVersion":1,"type":"EndDevice","zclVersion":2},"illuminance":3,"illuminance_lux":168,"last_seen":"2024-12-30T15:44:32.570Z","led_indication":false,"linkquality":87,"motion_sensitivity":"high","occupancy":false,"occupancy_timeout":0,"temperature":7.38}'

What did you expect to happen?

Before z2m v2 the "illuminance" gave the raw sensor value. This was nice, because it had a good resolution. Now we have:

• A "illuminance" sensor with lx
• A "illuminance_lux" sensor, which reports 168 and never changes

I would like to use the raw sensor value as before and hoped it would be reported, but it does not seem to work anymore.

How to reproduce it (minimal and precise)

From 1.x update to latest-dev.

Zigbee2MQTT version

latest-dev

Adapter firmware version

20240710

Adapter

UZG-01

Setup

Docker on x86

Debug log

No response

[Koenkk]

illuminance is now always the lux value, pushed a fix which removes illuminance_lux (restart z2m once after updating)

Changes will be available in the dev branch in a few hours from now. Note that the dev branch currently contains breaking changes, see #24198

[peterforeman]

That is a partial fix for me ;-) I would still like to access the raw value. In this case the resolution when it's getting dark is bad and the difference between 1 and 4 lux is huge. My do-things-when-its-getting-dark automations do not work nicely anymore because of the bad resolution. The original (raw) sensor had values like 45000 when it's sunny and 0-5000 when it's getting dark.

Updated to the pushed fix and the illuminance_lux value is indeed gone now. So your change as you described seems to work - but does not fix my problem.

[Koenkk]

I see your point, let's see after the 2.0.0 how many people would like to have illuminance_raw. For those that want illuminance_raw, please 👍 the OP.

[mr2c12]

Please bring back the raw value. OP already thumb'd up!

I just have a bunch of illuminance sensors around the house, but the pain is to re-set the automations to the new values! Old values worked like a charm.

[Typ1er]

Did both values come directly from the sensor or was one value calculated from the other?

[Logonz]

Why would we ever not want the raw value? If you want a more stable value just use a Helper to avg or something.
Thumbs up, this change was so unnecessary.

[Viald]

Same issue here. When the day breaks my external sensor goes from 4769 lux to 3011 lux, and from 3011 lux to 1 lux in 10 minutes. Before, each step was around 80 lux.
Before dusk was between 120 lux and 20 lux. And night was around 20 lux and lower. It's now impossible to detect this steps.
I don't really understand what conversion is applied by z2m 2.0

[Claude2666]

All my Xiaomi sensors GZCGQ01LM sometimes report wrong illuminance_lux value.
On a bright sunny day, no clouds, sometimes the illuminance_lux drops to a very low value.
In the logs, I see for the same illuminance value, different illuminance lux_values.
Now it is winter and no sunlight, but when it is bright and sunny, drop in wrong illuminance_lux is significant, and this affects my sunshade automations.

As A workaround, I calculate the illuminance_lux from the illuminance value.
But with the zigbee2mqtt 2.0 this will not be possible anymore.
So it would be very helpful to have the raw data back.

I am still running 1.41.0-dev commit: 32e2637

[2025-01-08 11:51:53] info: z2m:mqtt: MQTT publish: topic 'zigbee2mqtt2/LuxSensor1', payload '{"battery":47,"illuminance":36711,"illuminance_lux":4688,"linkquality":127,"voltage":2921}'
[2025-01-08 11:54:32] info: z2m:mqtt: MQTT publish: topic 'zigbee2mqtt2/LuxSensor1', payload '{"battery":47,"illuminance":36711,"illuminance_lux":4894,"linkquality":131,"voltage":2920}'

[2025-01-07 12:08:27] info: z2m:mqtt: MQTT publish: topic 'zigbee2mqtt2/LuxSensor1', payload '{"battery":73,"illuminance":33668,"illuminance_lux":2326,"linkquality":131,"voltage":2959}'
[2025-01-07 12:08:32] info: z2m:mqtt: MQTT publish: topic 'zigbee2mqtt2/LuxSensor1', payload '{"battery":73,"illuminance":33668,"illuminance_lux":2311,"linkquality":134,"voltage":2959}'

With debug on, it seems that when cluster 'manuSpecificLumi', data '{"247": ... is received, wrong illuminance_lux is published.

[2025-01-08 13:27:18] info: z2m:mqtt: MQTT publish: topic 'zigbee2mqtt2/LuxSensor1', payload '{"battery":45,"illuminance":37136,"illuminance_lux":5170,"linkquality":123,"voltage":2918}'
[2025-01-08 13:44:14] debug: z2m: Received Zigbee message from 'LuxSensor1', type 'attributeReport', cluster 'manuSpecificLumi', data '{"247":{"data":[1,33,105,11,3,40,25,4,33,168,67,5,33,114,3,6,36,15,0,0,0,0,8,33,21,1,10,33,151,83,12,32,1,100,33,126,18],"type":"Buffer"}}' from endpoint 1 with groupID 0
[2025-01-08 13:44:15] info: z2m:mqtt: MQTT publish: topic 'zigbee2mqtt2/LuxSensor1', payload '{"battery":47,"illuminance":37136,"illuminance_lux":4734,"linkquality":123,"voltage":2921}'

correct case:
[2025-01-08 13:27:17] debug: zh:zstack:unpi:parser: <-- [254,7,69,196,254,176,2,151,83,206,25,217]
[2025-01-08 13:27:17] debug: zh:zstack:unpi:parser: --- parseNext [254,7,69,196,254,176,2,151,83,206,25,217]
[2025-01-08 13:27:17] debug: zh:zstack:unpi:parser: --> parsed 7 - 2 - 5 - 196 - [254,176,2,151,83,206,25] - 217
[2025-01-08 13:27:17] debug: zh:zstack:znp: <-- AREQ: ZDO - srcRtgInd - {"dstaddr":45310,"relaycount":2,"relaylist":[21399,6606]}
[2025-01-08 13:27:17] debug: zh:zstack:unpi:parser: --- parseNext []
[2025-01-08 13:27:17] debug: zh:zstack:unpi:parser: <-- [254,28,68,129,0,0,0,4,254,176,1,1,0,123,0,212,221,174,0,0,8,24,103,10,0,0,33,16,145,206,25,27,94]
[2025-01-08 13:27:17] debug: zh:zstack:unpi:parser: --- parseNext [254,28,68,129,0,0,0,4,254,176,1,1,0,123,0,212,221,174,0,0,8,24,103,10,0,0,33,16,145,206,25,27,94]
[2025-01-08 13:27:17] debug: zh:zstack:unpi:parser: --> parsed 28 - 2 - 4 - 129 - [0,0,0,4,254,176,1,1,0,123,0,212,221,174,0,0,8,24,103,10,0,0,33,16,145,206,25,27] - 94
[2025-01-08 13:27:17] debug: zh:zstack:znp: <-- AREQ: AF - incomingMsg - {"groupid":0,"clusterid":1024,"srcaddr":45310,"srcendpoint":1,"dstendpoint":1,"wasbroadcast":0,"linkquality":123,"securityuse":0,"timestamp":11460052,"transseqnumber":0,"len":8,"data":{"type":"Buffer","data":[24,103,10,0,0,33,16,145]}}
[2025-01-08 13:27:17] debug: zh:controller: Received payload: clusterID=1024, address=45310, groupID=0, endpoint=1, destinationEndpoint=1, wasBroadcast=false, linkQuality=123, frame={"header":{"frameControl":{"frameType":0,"manufacturerSpecific":false,"direction":1,"disableDefaultResponse":true,"reservedBits":0},"transactionSequenceNumber":103,"commandIdentifier":10},"payload":[{"attrId":0,"dataType":33,"attrData":37136}],"command":{"ID":10,"name":"report","parameters":[{"name":"attrId","type":33},{"name":"dataType","type":32},{"name":"attrData","type":1000}]}}
[2025-01-08 13:27:17] debug: zh:zstack:unpi:parser: --- parseNext []
[2025-01-08 13:27:17] debug: z2m: Received Zigbee message from 'LuxSensor1', type 'attributeReport', cluster 'msIlluminanceMeasurement', data '{"measuredValue":37136}' from endpoint 1 with groupID 0
[2025-01-08 13:27:18] info: z2m:mqtt: MQTT publish: topic 'zigbee2mqtt2/LuxSensor1', payload '{"battery":45,"illuminance":37136,"illuminance_lux":5170,"linkquality":123,"voltage":2918}'

wrong case:
[2025-01-08 13:44:14] debug: zh:zstack:unpi:parser: <-- [254,7,69,196,254,176,2,151,83,206,25,217]
[2025-01-08 13:44:14] debug: zh:zstack:unpi:parser: --- parseNext [254,7,69,196,254,176,2,151,83,206,25,217]
[2025-01-08 13:44:14] debug: zh:zstack:unpi:parser: --> parsed 7 - 2 - 5 - 196 - [254,176,2,151,83,206,25] - 217
[2025-01-08 13:44:14] debug: zh:zstack:znp: <-- AREQ: ZDO - srcRtgInd - {"dstaddr":45310,"relaycount":2,"relaylist":[21399,6606]}
[2025-01-08 13:44:14] debug: zh:zstack:unpi:parser: --- parseNext []
[2025-01-08 13:44:14] debug: zh:zstack:unpi:parser: <-- [254,66,68,129,0,0,192,252,254,176,1,1,0,123,0,118,144,184,0,0,46,28,95,17,90,10,247,0,65,37,1,33,105,11,3,40,25,4,33,168,67,5,33,114,3,6,36,15,0,0,0,0,8,33,21,1,10,33,151,83,12,32,1,100,33,126,18,206,25,27,179]
[2025-01-08 13:44:14] debug: zh:zstack:unpi:parser: --- parseNext [254,66,68,129,0,0,192,252,254,176,1,1,0,123,0,118,144,184,0,0,46,28,95,17,90,10,247,0,65,37,1,33,105,11,3,40,25,4,33,168,67,5,33,114,3,6,36,15,0,0,0,0,8,33,21,1,10,33,151,83,12,32,1,100,33,126,18,206,25,27,179]
[2025-01-08 13:44:14] debug: zh:zstack:unpi:parser: --> parsed 66 - 2 - 4 - 129 - [0,0,192,252,254,176,1,1,0,123,0,118,144,184,0,0,46,28,95,17,90,10,247,0,65,37,1,33,105,11,3,40,25,4,33,168,67,5,33,114,3,6,36,15,0,0,0,0,8,33,21,1,10,33,151,83,12,32,1,100,33,126,18,206,25,27] - 179
[2025-01-08 13:44:14] debug: zh:zstack:znp: <-- AREQ: AF - incomingMsg - {"groupid":0,"clusterid":64704,"srcaddr":45310,"srcendpoint":1,"dstendpoint":1,"wasbroadcast":0,"linkquality":123,"securityuse":0,"timestamp":12095606,"transseqnumber":0,"len":46,"data":{"type":"Buffer","data":[28,95,17,90,10,247,0,65,37,1,33,105,11,3,40,25,4,33,168,67,5,33,114,3,6,36,15,0,0,0,0,8,33,21,1,10,33,151,83,12,32,1,100,33,126,18]}}
[2025-01-08 13:44:14] debug: zh:controller: Received payload: clusterID=64704, address=45310, groupID=0, endpoint=1, destinationEndpoint=1, wasBroadcast=false, linkQuality=123, frame={"header":{"frameControl":{"frameType":0,"manufacturerSpecific":true,"direction":1,"disableDefaultResponse":true,"reservedBits":0},"manufacturerCode":4447,"transactionSequenceNumber":90,"commandIdentifier":10},"payload":[{"attrId":247,"dataType":65,"attrData":{"type":"Buffer","data":[1,33,105,11,3,40,25,4,33,168,67,5,33,114,3,6,36,15,0,0,0,0,8,33,21,1,10,33,151,83,12,32,1,100,33,126,18]}}],"command":{"ID":10,"name":"report","parameters":[{"name":"attrId","type":33},{"name":"dataType","type":32},{"name":"attrData","type":1000}]}}
[2025-01-08 13:44:14] debug: zh:zstack:unpi:parser: --- parseNext []
[2025-01-08 13:44:14] debug: z2m: Received Zigbee message from 'LuxSensor1', type 'attributeReport', cluster 'manuSpecificLumi', data '{"247":{"data":[1,33,105,11,3,40,25,4,33,168,67,5,33,114,3,6,36,15,0,0,0,0,8,33,21,1,10,33,151,83,12,32,1,100,33,126,18],"type":"Buffer"}}' from endpoint 1 with groupID 0
[2025-01-08 13:44:14] debug: zhc:lumi: GZCGQ01LM: Processed buffer into data {"1":2921,"3":25,"4":17320,"5":882,"6":15,"8":277,"10":21399,"12":1,"100":4734}
[2025-01-08 13:44:14] debug: zhc:lumi: GZCGQ01LM: Processed data into payload {"voltage":2921,"battery":47,"power_outage_count":881,"illuminance_lux":4734}
[2025-01-08 13:44:14] debug: zhc:lumi: GZCGQ01LM: Processed data into payload {"voltage":2921,"battery":47,"power_outage_count":881,"illuminance_lux":4734}
[2025-01-08 13:44:15] info: z2m:mqtt: MQTT publish: topic 'zigbee2mqtt2/LuxSensor1', payload '{"battery":47,"illuminance":37136,"illuminance_lux":4734,"linkquality":123,"voltage":2921}'

[Koenkk]

The illuminance_raw option has now been added, it can be enabled via the frontend -> device -> settings (specific) -> illuminance raw option.

Changes will be available in the dev branch in a few hours from now.

[peterforeman]

The illuminance_raw option has now been added, it can be enabled via the frontend -> device -> settings (specific) -> illuminance raw option.

Changes will be available in the dev branch in a few hours from now.

Tested and it works, thanks. This creates a new _raw entity.

[shrx]

Thanks for restoring the raw values! Would be nice if this workaround was mentioned in the Breaking changes notes #24198 as well, I had to dig around to find it.

[Koenkk]

@shrx done!