Rerun Visualization on dora-rs

[haixuanTao]

If you never heard about rerun.io, it is an extremely impressive visualization library made by some awesome people, and we believe that it would be a great for the dora-rs community to be able to visualize data with something much more powerful than standard tooling such as matplotlib.

There's two reasons, I think that beyond convenience, we can delivers a very performant software:

• fully cross-compatible, statically linked, multi-threaded 100% rust based software.

This discussion is a tracking issue. Feel free to share ideas, opinions, or just support this idea.

Demos

• Real-time kHz time series in a multimodal visualizer: https://www.rerun.io/blog/fast-plots

Tracking issues

• What is the ideal interface between dora-rs and rerun in both data structure and APIs
• How to deal with coordinate frame?
  • It seems that we will need to pass additional frame field within dora message to anchor message data.
• How should we deal with timestamping?
• Can we make visualization ergonomics with mechanism for starting, stopping?
• Quick performance demo to validate hypothesis and baseline for iteration?
• Should we modify dora-record to support other message format to be directly readable within rerun such as: mcap, ...

Special thanks

The @rerun-io team for trusting us. Hoping that beyond code, we can foster robotics as a community, make innovation open source and free (both as in beer and as in freedom)

[haixuanTao]

And the first PR is merged! #479 :)

What is the ideal interface between dora-rs and rerun in both data structure and APIs

This first PR provides, IMO, a good first draft on interfacing dora messages with rerun objects. Essentially using rerun Tensordata to consume our arrow buffer.

Can we make visualization ergonomics with mechanism for starting, stopping?

Ergonomics is pretty neat as each new start and stop is spawned in the same rerun window.

Quick performance demo to validate hypothesis and baseline for iteration?

Current demo example show no performance cost when coming from matplotlib. :)

[haixuanTao]

So it seems that there is an additional 20ms latency when using rerun.io for camera video stream when compared to a barebone opencv stream, that we will have to solve.

This can be a bit annoying when doing realtime teleoperation.

IMG_2953.mov

[emilk]

You can reduce this a bit by setting RERUN_FLUSH_TICK_SECS=0.001. See https://www.rerun.io/docs/reference/sdk-micro-batching