unable to get imu measurements above 185Hz

[SinnedHog]

In the stereo_inertial_publisher.cpp (line 104 and 105) of the depthai-ros-examples, the accel and gyro were set at 500Hz and 400Hz respectively. However, when I run stereo_inertial_node.launch (with stereo images (400p) at 20Hz, depth_aligned=false and enable_spatial_detection=false), the rostopic (stereo_inertial_publisher/imu) for the IMU messages only comes in at max 185Hz. When I modified the accel and gyro setting to 200Hz, the rate for the IMU messages is still cap at max 185Hz. It only matches the setting if I set the output rate for accel and gyro to 100Hz or below. Why is there a cap in the imu measurement? Is the cap in the output rate a software or hardware limitation?

[saching13]

@SinnedHog when was this device bought ? and which device is this ?

[SinnedHog]

OAK-D Pro, brought it around 3 months ago.

[saching13]

Can you check this and see if it gives better Freq ?

[SinnedHog]

Yes. Got around 250Hz. What is the reason for the difference?

[heavenbo]

Hi, I had the same problem. My gyro and acc rate are set to 400 and 500 respectively, and I got an imu rate around 207. Then I tried to set the stereo publishing rate to 20Hz and I got an imu rate at 223. But I want a fixed imu rate at 200Hz so I set gyro and acc raw rate at both 200. Strangely, I got the imu rate around 180 by doing that. I wonder if there is a way to get a stable imu rate?

[saching13]

https://docs.luxonis.com/projects/api/en/latest/samples/IMU/imu_rotation_vector/#imu-rotation-vector
Can you guys check if this example publishes rotation vector ?

[heavenbo]

The device I'm using is OAK-D-PRO-W
Here is the result.

  File "imu_rotation_vector.py", line 31, in <module>
    with dai.Device(pipeline) as device:
RuntimeError: IMU(0) - IMU invalid settings!: ROTATION_VECTOR output is unsupported. BMI270 supports only ACCELEROMETER_RAW and/or GYROSCOPE_RAW outputs. ```  

[saching13]

Yup. you have a new one with BMI270 whose max hz for accel and gyro is 200Hz

[kpsgf7]

@saching13 We pre-ordered the newest OAK-D Pro POE W and received it maybe a month ago. We get the same message about the BMI270 when running the script mentioned above. Your documentation lead us to believe this camera would have a BNO08X 9 axis imu.

If I start here https://shop.luxonis.com/products/oak-d-pro-w-poe and follow through to the documentation here https://docs.luxonis.com/projects/hardware/en/latest/pages/NG9097prow.html#ng9097prow it says the only difference between the OAK-D Pro POE W and the OAK-D S2 POE is the IR Dot projector and the wide angle cameras. Following the link in the documentation leads to here https://docs.luxonis.com/projects/hardware/en/latest/pages/NG9097s2.html#oak-d-s2-poe. Scrolling to the bottom of that page finally gives us a link to the datasheet here https://github.com/luxonis/depthai-hardware/blob/master/NG9097_OAK-D-S2-PoE/Datasheet/OAK-D-S2-PoE_Datasheet.pdf which specifies a BNO086 9 axis IMU on page 5.

I can't find anything that mentions a BMI270 until I run the script from above. Is this script reporting an incorrect IMU or is the documentation incorrect?

[saching13]

Hello @kpsgf7 ,
Sorry for the confusion.
We had issues with procurement. And some of the models ended up with BMI217 in 1 or 2 batches of production. We will be adding some API to detect that.
if 9 DOF is a must for your use case. please shoot us a mail on this. We would be happy to help you.

[kpsgf7]

Hi @saching13,

I can certainly understand procurement difficulties. The IMU in and of itself is not the end of the world.

I am concerned that you guys are selling one thing and shipping a different thing. Are there any other parts on this camera that aren't going to match the spec sheet?

I appreciate your help.

[saching13]

Hello @kpsgf7
Thanks for understanding.

Are there any other parts on this camera that aren't going to match the spec sheet?

No. The main sensors will always be the same. There will be small changes on the board when certain part is not available. but it shouldn't change how it works. IMU is the only main part which we had hard issue with sourcing and since most of the customers were not using it much. We made one batch with BMI so that we don't fully stock out.
We have already sourced the BNO back. So next and future batches will include the BNO IMU.

And I also requested the team to update the docs and Datasheet to reflect the changes. Sorry it was not updated.

[borongyuan]

But why we only get about 180Hz output from IMU even though we set it to 200Hz (synced). It is written in your document that up to 400Hz is stable. I tried different BatchReportThreshold, MaxBatchReports, and queue size, but it didn't help. So is this an SDK issue or a firmware issue? For tightly coupled VIO/VSLAM algorithms, this issue is fatal.

[saching13]

But why we only get about 180Hz output from IMU even though we set it to 200Hz (synced). It is written in your document that up to 400Hz is stable. I tried different BatchReportThreshold, MaxBatchReports, and queue size, but it didn't help. So is this an SDK issue or a firmware issue? For tightly coupled VIO/VSLAM algorithms, this issue is fatal.

Please read the conversation above on when the cap is 200Hz and check which device and IMU the device contains.

As of 185 when set to 200hz. That much effect is expected AFAIK.

[saching13]

And I would also like to know which SLAM or VIO are you testing with. So that we can see if we can apply some patches in the future for better Integration.

[borongyuan]

I made a series of updates to DepthAI driver in RTAB-Map a while ago. Using OAK cameras directly in RTAB-Map now works well. But you can take a look at our commits history to see how many issues we encountered and fixed. Currently the RTAB-Map launch file in the DepthAI ROS package does not work well. I'm solving the same issues as before. I didn't pay attention to the frame rate of the IMU before, because the VO of RTAB-Map does not depend on the IMU, but only adds loose coupling constraints. But now I want to try other VIO algorithms like VINS and MSCKF. So I double check first to make sure the inputs are all processed correctly.
I currently mainly use OAK-D Pro W, with BMI270 on board.

[borongyuan]

I tested some implementations again without using depthhai-ros. I still can't get higher IMU frame rate (synced only, no interpolation). Even if I set the IMU rate to 400 Hz, what I actually get is only about 200 Hz. Only set to 100Hz is stable.

[saching13]

Our devices comes with BNO086 or BMI270. BMI270 has a limitation of 200hz max.
We will update the docs.

However we cannot guarantee the absolute 200hz consistently because due to bandwidth, backlogged packets there can be loss of packets sometimes IIRC.

[borongyuan]

Is this releated to luxonis/depthai-core#866 and luxonis/depthai-core#684? Latency issue is even more fatal. I'm going to try older versions.

[saching13]

cc: @JanLipovsek if you have any suggestions.

[JanLipovsek]

Hi @borongyuan , we have fixed the latency issue on the BMI270 IMU on the latest develop branch.

[borongyuan]

Thanks, I'll give it a try. I also discussed the frequency issue of BMI270 with your partners in China yesterday. We can try to do a comparison test between BNO086 and BMI270 later.

[Serafadam]

Closing due to inactivity, please reopen if there are still questions.