A very simple file format: int64 length field
followed by bytes of that length per message.

- Add `lz4` dependency
- Set `trigger_mode` explicitly, and ensure `nimages` and/or `ntrigger`
  is set according to the current `nav_shape`
- Make the `trigger` function optional, as it is not needed unless you
  want to either set additional detector parameters, or automate
  acquisition with a microscope API

Use own fork for now, see
kiyo-masui/bitshuffle#120

This simulator can replay an acquisition that was recorded with
prototypes/zmq_record.py.

Usage: `libertem-live-dectris-sim <file>`

This is not a complete simulator, but implements just enough
of the API to allow performance
testing and run unit tests of the whole acquisition code.

It reads the metadata from the recording and ensures that
the user settings match the file. For exte and exts triggers
it will assume to be triggered. For inte and ints
it will wait for the 'trigger' command.

ZMQ sockets balance load, meaning several co-existing listeners
will only receive part of the data.

For that reason we only connect while an acquisition is running and
destroy the context directly after. That closes all sockets.

Drive-by: Deactivate debug print.

Co-authored-by: Alexander Clausen <a.clausen@fz-juelich.de>

Still needs to be extracted into an executor and probably an adjusted
`UDFRunner`.

- properly drain and close queues
- use a single response queue - this simplifies result handling on the
  receiving side (might become bottleneck in many-core situations)
- perform decoding in the worker processes
- `ZeroMQReceiver`: zero-copy recv, at least for the payload
- some hacks to inject the receiver into the partition

The duplicated code for `StoppableThreadMixin` was modified to take
a `stop_event`, which is now moved to `libertem_live.detectors.common`.

This should now continue to work, even if we change further details of
the communication between acquisition and partition

Also fixes tracing span names

* Restructure to make all relevant components of the simulator
  importable and runnable from other code
* Run Flask app for API in separate process to make it killable
* Allow running both API and data on random free port
* Fixture and simple test case, FIXME integrate meaningful tests
* FIXME the other Dectris test cases seem to be broken, not sure what to do
  there

Disregard copyright notice, will be changed before merging (!)

FIXME documentation and tests

hdbscan does not exist as an extra of LiberTEM-live

Caused a freeze in the pipelined executor before

* Ignore extra data at the beginning of the file
* Add missing acquisition footer

Debug connection lag on Windows

FYI it was DNS for localhost -- examples should use IP address instead

+ Clean up some now-unused code
+ Move mocking/offline receiver into
  `libertem_live.detectors.dectris.mock`
+ dectris sim: use Python sim when `data_filter` is used
  (not needed for performance tests)

Don't install libertem-dectris on Python 3.6

Throw a helpful error message when dependencies are missing

Instead, pull up a new one which we can close at the end

To at least make it importable - this doesn't add full 3.6 support

Ignore and warn. Also remove some unused commented code.

On older versions, it does still hang, so we can't run that test in CI.

* Switch to RTD theme
* Run the dectris sim in docs-check to check the example code
* Separate sections for API reference and user documentation
* Add verbosity switch in dectris sim to make sphinx doctest happy (it's
  not yet possible to ignore output of testsetup / testcleanup
  directives)
* dectris sim: catch StopException

for more information, see https://pre-commit.ci

for more information, see https://pre-commit.ci

Thx @sk1p!