vwsfriend stuck with error in the log output

[stheine]

I realized that the container stopped working properly (no more updates of the charging state), and the log shows as error:

2024-11-01T09:50:58+0000:ERROR:vwsfriend_base:MQTT Connection failed
  File "/opt/venv/bin/vwsfriend", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/vwsfriend/vwsfriend_base.py", line 516, in main
    mqttCLient.updateWeConnect(reraise=True)
  File "/opt/venv/lib/python3.12/site-packages/weconnect_mqtt/weconnect_mqtt_base.py", line 462, in updateWeConnect
    self.weConnect.update(updateCapabilities=self.updateCapabilities, updatePictures=self.updatePictures, selective=self.selective, force=True)
  File "/opt/venv/lib/python3.12/site-packages/weconnect/weconnect.py", line 197, in update
    self.updateComplete()
  File "/opt/venv/lib/python3.12/site-packages/weconnect/addressable.py", line 533, in updateComplete
    child.updateComplete()
  File "/opt/venv/lib/python3.12/site-packages/weconnect/addressable.py", line 533, in updateComplete
    child.updateComplete()
  File "/opt/venv/lib/python3.12/site-packages/weconnect/addressable.py", line 533, in updateComplete
    child.updateComplete()
  [Previous line repeated 3 more times]
  File "/opt/venv/lib/python3.12/site-packages/weconnect/addressable.py", line 96, in updateComplete
    observer(element=self, flags=self.onCompleteNotifyFlags)
  File "/opt/venv/lib/python3.12/site-packages/vwsfriend/agents/charge_agent.py", line 196, in __onChargingStateChange
    self.updatePosition()
  File "/opt/venv/lib/python3.12/site-packages/vwsfriend/agents/charge_agent.py", line 385, in updatePosition
    self.chargingSession.charger = chargerFromLatLonWithGeofence(weConnect=self.vehicle.weConnectVehicle.weConnect, session=self.session,
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/sqlalchemy/orm/attributes.py", line 537, in __set__
    self.impl.set(
  File "/opt/venv/lib/python3.12/site-packages/sqlalchemy/orm/attributes.py", line 1467, in set
    value = self.fire_replace_event(state, dict_, value, old, initiator)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/sqlalchemy/orm/attributes.py", line 1510, in fire_replace_event
    state._modified_event(dict_, self, previous)
  File "/opt/venv/lib/python3.12/site-packages/sqlalchemy/orm/state.py", line 927, in _modified_event
    session._autobegin_t()
  File "/opt/venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 1877, in _autobegin_t
    trans = SessionTransaction(
            ^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 931, in __init__
    TransactionalContext._trans_ctx_check(session)
  File "/opt/venv/lib/python3.12/site-packages/sqlalchemy/engine/util.py", line 111, in _trans_ctx_check
    raise exc.InvalidRequestError(
sqlalchemy.exc.InvalidRequestError: Can't operate on closed transaction inside context manager.  Please complete the context manager before emitting further commands.

The container is still reported as 'healthy'.

I now manually restarted the container, and immediately, updates are coming in again.

[stheine]

I had the same symptom again today, vwsfriend hanging (no more updates to MQTT) with a different error:

Exception in thread Thread-1 (mqttWorker):
Traceback (most recent call last):
  File "/usr/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
    self.run()     
  File "/usr/lib/python3.12/threading.py", line 1012, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/venv/lib/python3.12/site-packages/vwsfriend/vwsfriend_base.py", line 470, in mqttWorker
    mqttCLient.loop_forever(retry_first_connection=True)
  File "/opt/venv/lib/python3.12/site-packages/paho/mqtt/client.py", line 2297, in loop_forever
    rc = self._loop(timeout)
         ^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/paho/mqtt/client.py", line 1686, in _loop 
    rc = self.loop_read()
         ^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/paho/mqtt/client.py", line 2100, in loop_read
    rc = self._packet_read()
         ^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/paho/mqtt/client.py", line 3142, in _packet_read
    rc = self._packet_handle()
         ^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/paho/mqtt/client.py", line 3804, in _packet_handle 
    return self._handle_pubackcomp("PUBACK")
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/paho/mqtt/client.py", line 4462, in _handle_pubackcomp
    rc = self._do_on_publish(mid, reasonCode, properties)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/paho/mqtt/client.py", line 4432, in _do_on_publish 
    rc = self._update_inflight()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/paho/mqtt/client.py", line 4231, in _update_inflight
    for m in self._out_messages.values():
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: OrderedDict mutated during iteration