error in self._client.connection_status())

[ederenn]

encountered on

golem source version, branch b0.16.0

b'GOLEM Version: 0.15.1+dev309.g398e18d'
b'Protocol Version: 27'
b'golem_messages Version: 2.9.4

2018-05-30 14:05:52 ERROR    golem                               Service Error: Traceback (most recent call last):
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 916, in _bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python3/3.6.3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/threading.py", line 864, in run
    self._target(*self._args, **self._kwargs)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/twisted/_threads/_threadworker.py", line 46, in work
    task()
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/twisted/_threads/_team.py", line 190, in doWork
    task()
--- <exception caught here> ---
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/twisted/python/threadpool.py", line 250, in inContext
    result = inContext.theWork()
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/twisted/python/threadpool.py", line 266, in <lambda>
    inContext.theWork = lambda: context.call(ctx, func, *args, **kw)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/twisted/python/context.py", line 122, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/twisted/python/context.py", line 85, in callWithContext
    return func(*args,**kw)
  File "/Users/ederenn/projects/golem/golem/client.py", line 1435, in _run
    self._client.connection_status())
  File "/Users/ederenn/projects/golem/golem/client.py", line 1234, in _publish
    self.rpc_publisher.publish(event_name, *args, **kwargs)
  File "/Users/ederenn/projects/golem/golem/rpc/session.py", line 241, in publish
    self.session.publish(str(event_alias), *args, **kwargs)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/autobahn/wamp/protocol.py", line 1301, in publish
    raise e
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/autobahn/wamp/protocol.py", line 1297, in publish
    self._transport.send(msg)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/autobahn/wamp/websocket.py", line 125, in send
    self.sendMessage(payload, isBinary)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/autobahn/websocket/protocol.py", line 2215, in sendMessage
    self.sendFrame(opcode=opcode, payload=payload, sync=sync, rsv=4 if sendCompressed else 0)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/autobahn/websocket/protocol.py", line 1841, in sendFrame
    self.sendData(raw, sync, chopsize)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/autobahn/websocket/protocol.py", line 1298, in sendData
    self.transport.write(data)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/twisted/protocols/tls.py", line 464, in write
    self._write(bytes)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/twisted/protocols/tls.py", line 544, in _write
    self._flushSendBIO()
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/twisted/protocols/tls.py", line 252, in _flushSendBIO
    bytes = self._tlsConnection.bio_read(2 ** 15)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1672, in bio_read
    self._handle_bio_errors(self._from_ssl, result)
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1653, in _handle_bio_errors
    _raise_current_error()
  File "/Users/ederenn/golem/venv/lib/python3.6/site-packages/OpenSSL/_util.py", line 54, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: []
NoneType: None

self._client.connection_status()) not in a main thread
possible solution:
reactor.callFromThread(self._client._publish, Network.evt_connection, self._client.connection_status())

[tworec]

cannot replicate. Please add description how did you encountered it.

[etam]

It might be one of many cases, where our event loop is overloaded with network activity. It sometimes manifests with random errors happening in rpc connections.

It doesn't change the fact, that we should somehow handle such errors :)

[etam]

I think I've found possible source of the problem. I've reported it here: pyca/pyopenssl#766

[mfranciszkiewicz]

This error often appears when you use the WS connection from another thread. NetworkConnectionPublisherService should call client.publish in the main thread (reactor.callFromThread).