Issue with "Optimistic" and MQTT Lock. The state in HA is not updated even if optimistic=True

[plapointe6]

Home Assistant release with the issue:
0.91.1

Last working Home Assistant release :

Upgraded form a version a little older than the Lovelace-UI release ( < 0.86 )

Operating environment :

Official Docker image running on a Raspberry PI 3.

docker run --name="home-assistant" -d \
--init \
--restart unless-stopped \
-p 8123:8123 \
-v /home/pi/homeassistant:/config \
-v /etc/localtime:/etc/localtime:ro \
homeassistant/raspberrypi3-homeassistant

Component/platform:

https://www.home-assistant.io/components/mqtt/
Could be linked to Lovelace-UI, not sure ...

Description of problem:

I am using MQTT to communicate with a door lock. This lock register to the Home Assistant MQTT Discovery like this :

{"~":"ha/lock/mylock","name":"MyLock","stat_t":"~","cmd_t":"~/c","ret":"true","opt":"true"}

The lock have a simple function : when it wake up, the lock connect to the MQTT broker and check if the value of the state topic is LOCK or UNLOCK. If the state is UNLOCK, the door will be allowed to open and it will go to sleep again.
So, within Home Assistant, I can LOCK or UNLOCK the lock entity in the front-end, even when my lock is offline because I use the "optimistic" flag.

This worked fine until I updated to 0.91.1 from an older release (0.8?.?, before 0.86.0).
Now, with the same config, the state of my lock in home assistant stay to "UNLOCKED" even if I try do LOCK it from the front-end.
The MQTT command message is sent, but the state is not updated. Maybe it's an issue with the "optimistic" functionnality ?

Problem-relevant configuration.yaml entries :

mqtt:
  broker: 192.168.1.101
  username: my_user
  password: !secret mqtt_password
  discovery: true
  discovery_prefix: ha
  birth_message:
    topic: 'ha/status'
    payload: 'online'
    retain: true
  will_message:
    topic: 'ha/status'
    payload: 'offline'
    retain: true

Traceback :

Here is the discovery related message and shortly after I am trying to LOCK the lock two times.
We see that the message is sent and no state topic message is replied (but it's normal, my lock is offline).

2019-04-07 16:17:27 DEBUG (MainThread) [homeassistant.components.mqtt] Received message on ha/lock/Barrure_chambre/config: b'{"~":"ha/lock/Barrure_chambre","name":"Barrure chambre","stat_t":"~","cmd_t":"~/c","ret":"true","opt":"true"}'
2019-04-07 16:17:27 INFO (MainThread) [homeassistant.components.mqtt.discovery] Found new component: lock Barrure_chambre
2019-04-07 16:17:27 DEBUG (MainThread) [homeassistant.components.mqtt] Subscribing to ha/lock/Barrure_chambre

...

2019-04-07 19:57:41 DEBUG (MainThread) [homeassistant.components.mqtt] Transmitting message on ha/lock/Barrure_chambre/c: LOCK
2019-04-07 19:57:42 DEBUG (MainThread) [homeassistant.components.mqtt] Received message on ha/lock/Barrure_chambre/c: b'LOCK'
2019-04-07 19:57:52 DEBUG (MainThread) [homeassistant.components.mqtt] Transmitting message on ha/lock/Barrure_chambre/c: LOCK
2019-04-07 19:57:52 DEBUG (MainThread) [homeassistant.components.mqtt] Received message on ha/lock/Barrure_chambre/c: b'LOCK'

After these log, if I go to my_home_assistant_url/dev_state, I see that my lock is "UNLOCKED"

Entity               State
lock.barrure_chambre unlocked

[github-actions]

Hey there @home-assistant/core, mind taking a look at this issue as its been labeled with a integration (mqtt) you are listed as a codeowner for? Thanks!

[robbiet480]

@home-assistant/core

[emontnemery]

@plapointe6 thanks for reporting. As you suspect it's an issue with optimistic mode (introduced by #19468), I'll push a fix soon. Just a question, why does your lock include a state_topic in the discovery message if it doesn't report state?

[plapointe6]

Thanks for getting on this.

@emontnemery, you are right, the state topic is useless. I will fix this in the lock.

[emontnemery]

@plapointe6 If you remove the state_topic, optimistic mode should work. I've pushed a fix in #22899 which should fix optimistic mode when state_topic is set, maybe you can give it a try.

[raomin]

Hi, I have the same problem, on 0.91.2
I don't have state_topic set.
To reproduce:
mosquitto_pub -t 'homeassistant/fan/underground/config' -m '{"name":"fanUnderground","cmd_t":"homeassistant/fanUnderground/set"}'

I add the fan in the UI (lovelace).
Then click on the switch to set it on.
2 sec later it turns back to off.

Same behaviour when I set a state_topic and/or "optimistic":"true".

[plapointe6]

I will test the #22899 fix soon and I will update with the results when it will be done.

[raomin]

I'm confused as #22899 touches only mqtt/lock.py and I have the similar behaviour with mqtt/fan.py.
Same bug in 2 sub-components of mqtt. Maybe the answer is in mqtt?
But it might also be different bugs...

[emontnemery]

@raomin it's not the same bug. Can you please open a separate issue for MQTT fan optimistic mode not working?

[raomin]

just did: #22967 MQTT fan does not work in optimistic mode

[plapointe6]

I tested the changes referenced in fix #22899 (lock.py) and I confirm that it solved the issue with MQTT Lock and optimistic flag. Thanks !