Offboard total thrust and torque control mode

[varadVaidya]

For my project, I aim to control the Crazyflie using total thrust and torques as inputs. These signals are generated by a deep reinforcement learning (RL) model that is too large to be implemented onboard, similar to the approach used in TinyMPC.

To achieve this, I attempted to create a custom thrust-torque setpoint in both cflib and crazyflie-firmware. However, I encountered a significant bottleneck: the Crazyradio seems unable to transmit these commands at the rate required for the stabilizer to function effectively.

I would like to know if there are any existing examples or implementations that support this type of control mode off-board. Alternatively, is there a way to "emulate" the Crazyflie dynamics off-board to generate a higher-level setpoint (e.g., position or velocity) that closely follows the original control signals' intention?

Any insights, suggestions, or examples would be greatly appreciated.

By "emulation," I mean using an approximate dynamics model of the Crazyflie to perform an RK4 update at its current state, using the total thrust and torques generated by the neural network. The resulting position, velocity, attitude, etc., would then be sent to the Crazyflie as setpoints via cmd_full_state_changed (from Crazyswarm2) or send_full_state_setpoint (from cflib), but at a lower frequency, such as 50Hz.

What do you think of this approach? While it feels somewhat like a hack, I’m open to suggestions or alternatives for achieving off-board control effectively.

PS: I have raised the same discussion on the bitcraze forum. I am raising it here as well to see if the changes in the way the cpp backend communicate can help solve the radio bottlenecks (RF communication is black magic to me right now).

[whoenig]

What frequency did you achieve on the radio? With a single CF, I think it should be possible to up to 400 Hz (but if you also send mocap updates, the radio is shared for these packets as well).

The standard approach is to model the UAV as a double integrator and then use full_state_setpoint to send the resulting position, velocity, and acceleration commands. For "regular" flight this is more than sufficient; for aggressive maneuvers an on-board controller is required, IMO. Crazyswarm2 also comes with a simulator that has the basic Newton-Euler equations implemented, so in principle you could do what you propose (not sure if that would create a significant benefit over a double integrator model, though).

It is possible to run inference for a RL model directly on-board, see https://sites.google.com/view/sim-to-multi-quad for paper and code. The key idea is to convert the model to C-code.

[varadVaidya]

What frequency did you achieve on the radio?

I don't have exact numbers, but i think it was around 100Hz, with sending mocap updates, and gyro logging back to offboard computer for angular velocities. If there is any standard procedure to get the frequency at which updates are sent, I would love to compare and check.

For "regular" flight this is more than sufficient; for aggressive maneuvers an on-board controller is required, IMO.

I am planning to track trajectories with say velocity of 1.25 - 1.5 m/s, something like a figure 8 or some form of lissajous curve, does that classify as aggressive?

It is possible to run inference for a RL model directly on-board, see https://sites.google.com/view/sim-to-multi-quad for paper and code. The key idea is to convert the model to C-code.

Yes. The paper and the codebase is one of the inspiration for my research work. Unfortunately, my model currently is too big to fit in STM32's flash, requiring around 6MB for the weights alone. The possible options for onboard control are either quantizing the model or using AI Deck for control. Not to mention writing and testing of the complex architecture of the Actor Network. Both of those, at least from afar seem to be some challenging tasks. Hence the requirement for offboard control.

The standard approach is to model the UAV as a double integrator and then use full_state_setpoint to send the resulting position, velocity, and acceleration commands.

Thanks. I was just worried that the idea might seem to be a hack of sort. Thank you for providing clarity on it.

I think in future, I might develop some sort of tool to convert NNs into C code that can be deployed on AI deck. This should also provide some headroom for larger model and enable vision based controllers too. I am thinking on using the existing tools of TFLite/ONXX, but these are still concept ideas, and i will need to explore the embedded ML area in more detail.

Thanks for the help and apologies for any incoherent language. Its been quite a long few weeks.

[varadVaidya]

@whoenig I recently came across this toolbox which seems to allow for deployment of NNs onto any STM32. From reading the quick start guide, it does seem a promising direction, assuming all the weights fit in the memory.

I am curious to learn more about this. If there are any more resources that enables automatic NN code generation, please do let me know.

[whoenig]

We recently tried that toolbox here: https://github.com/IMRCLab/crazyflie-firmware/tree/stm32cubeai. It's very fast, but uses a closed-source library which makes software-in-the-loop testing very difficult. Another option is https://github.com/ucb-bar/Baremetal-NN/tree/main (not tested by us yet on the Crazyflie).

[varadVaidya]

Thanks alot for the resources and all the help! Will look into these options sometime in the near future.