[Question] Asynchronous Comm.on_msg callback

[duburcqa]

I'm registering a callback function that forward every messages associated to given registered target to a ZMQ socket (which is a remote server). It works great when the main thread is idle, but if the main thread is busy for some reason, the callback is not call until the main thread finishes its processing. It is a big issue in by case because in practice I'm busy waiting for the server to acknowledge it has received the data...

Here is how I'm registering the callback:

class ZMQIPythonBridge(threading.Thread):
    def __init__(self, comm_url):
        threading.Thread.__init__(self)
        self.zmq_url = zmq_url

    def comm_register(self, comm, msg):
        @comm.on_msg
        def _on_msg(msg):
            data = msg['content']['data']
            self.comm_zmq.send(f"data:{comm.comm_id}:{data}".encode())

        self.comm_zmq.send(f"open:{comm.comm_id}".encode())

    def run(self):
        loop = asyncio.new_event_loop()
        asyncio.set_event_loop(loop)
        ioloop = tornado.ioloop.IOLoop()
        ioloop.make_current()
        context = zmq.Context()
        self.zmq_socket = context.socket(zmq.XREQ)
        self.zmq_socket.connect(self.zmq_url)
        get_ipython().kernel.comm_manager.register_target(
            'my_channel', self.comm_register)
        ioloop.start()

comm_bridge = ZMQIPythonBridge(comm_url)
comm_bridge.start()

Is there a way to still be enable to trigger _on_msg while the main thread is busy wait for the ZMQ socket to acknowledge ? I tried to use a different thread, but as expected in it not changing anything (since the callback mechanism is still handle by the main thread...)