Simplify Concurrency Design

[callumforrester]

Background

At this point, blueapi uses both subprocess and threads manage the concurrent operations of:

• REST API exposure
• Management/introspection of RunEngine, plans, devices, etc.
• Actually running plans

I wonder if this can be simplified.

Proposal

We could ditch the worker thread and make the running of plans block the subprocess. To maintain state querying throughout the running of a plan, we would have to cache state somewhere, possibly in the main process.

It's important to add that I don't know if this is actually possible! I don't know if asynchronous updating is supported by subprocess, but it's worth considering.

Would be interested in your thoughts @joeshannon

[joeshannon]

There are queues and pipes in the multiprocessing module so I imagine the worker would be able to update the cached state in the API process by putting something onto a queue.

In terms of blocking the subprocess, this would simplify things but do we need any provision for cancelling running plans?

[stan-dot]

what if we use something like redis