Camera frame loss

[loupdmer]

Hello everybody,

I am using a D415 camera on NVIDIA Jetson Xavier NX with the 4.9.201-tegra kernel version.
My issue is that I cannot see the pointcloud /camera/depth/color/points on rviz.

I am not able to install librealsense2-dkms, so my uvcvideo version is only 1.1.1 with no any realsense string as you can see the following.

$ modinfo uvcvideo | grep "version:" version: 1.1.1 srcversion: 7FB2396C50DCC682C47C8FC

I read that would be linked with kernel incompatibilities, however, I can run realsense-viewer as follow:

I can run the pointcloud example of the librealsense SDK:

However, the realsense_ros demo_pointcloud.launch doesn't work.

My USB is 3.0 according to bcdUSB :

I have all these definitions available:

This with 30Hz rate for the depth and rgb camera.

I also noticed that it could come from the video frame:

The color/image_raw topics stop when I start demo_pointcloud.launch

but it also works when I try rs_rgbd.launch with a bad rate:

Does someone have any idea to help ? Why the librealsense pointcloud works and not ros ? Why is it only this node ? is it pcl ?

Thank you

[MartyG-RealSense]

Hi @loupdmer As a starting point in investigating this case, could you try the following steps:

1. In the PointCloud2 options of RViz, change 'Color Transformer' from "Intensity" to "RGB8"

#975 (comment)

2. Set the style to points.

#1327 (comment)

[loupdmer]

Actually, it doesn't work without rviz.
I have forgotten to mention that I guess it is linked with image definitions:

• When I set my launch file to use sensors at 30Hz in 640x480:

When I run rs_camera.launch, I can see rostopic hz at around 30Hz for both /camera/color/image_raw and /camera/depth/image_rect_raw

When I run rs_pointcloud.launch, the rates of /camera/color/image_raw, /camera/depth/image_rect_raw and /camera/depth/color/points falls to around 8Hz (I can see the pointcloud with rviz without issue with flat square or points)

• When I set my launch file to use sensors at 30Hz or 15Hz in 1280x720:

When I run rs_camera.launch, I can see rostopic hz at around 30Hz for both /camera/color/image_raw and /camera/depth/image_rect_raw but I sometimes have Incomplete frame received: Incomplete video frame detected!

When I run rs_pointcloud.launch, I got this:

On my computer with ubuntu 18.04 with 5.4.0-73-generic kernel all works well (30Hz for each topics)

Thank you for your help

[MartyG-RealSense]

The error Incomplete video frame detected is quite rarely reported. I researched past cases to put together some possible causes to check with you to see whether any of them match your particular circumstances.

• When the website JetsonHacks experienced this error on Jetson in the RealSense Viewer, they apparently suggested recompiling the kernel to resolve it.

IntelRealSense/librealsense#4108

In that same discussion, a RealSense user found that lowering the resolution to 960x540 or lower made the error go away, similar to how you found that the error did not occur at 640x480.

Below I have linked to the point in a JetsonHacks video about RealSense and kernel compilation where the error is generated.

https://www.youtube.com/watch?v=8ckhQP7Af-g&t=242s

• There was a past case of a RealSense user who was experiencing this error with ROS and found that it occurred much more frequently when using low-quality or degraded USB cables than when using high-grade cables that were new. In their particular case, they believed that cable degradation was due to their mobile robot repeatedly bending the cable.

IntelRealSense/librealsense#3189

Are you using the official 1 meter cable supplied with the camera or your own choice of cable please? If it is the official cable, is it in good condition?

• In another case where a few different RealSense users reported the error, two of them were using a Virtual Machine (VM). Is your camera being used with a VM please?

IntelRealSense/librealsense#6444

If none of these possibilities are helpful in resolving your problem then I am happy to continue exploring the issue further with you. Good luck!

[loupdmer]

Thank you for the answer, I will try to recompile the kernel as the video shows and I come back.

I am not using the official 1 meter cable because it was not in good condition so I bought another USB 3.0 cable. The cable seems to be good because I have good results when I launch rs_pointcloud on my ubuntu 18.04 laptop (all frame rates at around 30Hz)

[MartyG-RealSense]

Thanks very much @loupdmer - I look forward to your next update. Good luck!

[MartyG-RealSense]

Hi @loupdmer Do you have an update about this case that you can provide, please? Thanks!

[loupdmer]

Hi @MartyG-RealSense, thank you for taking news,
I have reflashed my card and then rebuilt the kernel according to the jetsonhacks github here :

https://github.com/jetsonhacks/buildJetsonXavierNXKernel

However, I was not able to access the camera with cheese as it was done in the video you linked me (but it was with a JetsonNano and not a JetsonXavier) and it brings a new mistake upon the fact there are held broken packages with unmet dependencies, so I was not able to install libssl-dev package required to build the librealsense sdk.

I tried many solutions given on forums like sudo dpkg --configure -a, removing from source.list or listing all held package but no one was detected as broken and I didn't found solution

For now, I can work on my computer with the realsense so I postpone the issue.

However, I have a new, I am able to acquire only depth and color frames from the camera with rs_camera.launch (and not the rgbd pointcloud), but if I enable pointcloud in the launchfile it doesn't work anymore. Is there any image conversion change when it deals with creating the pointcloud ?

Another thing is that I have a collegue who works on a Jetson Nano with a monocular camera, and he also have issue with acquiring RGB frames, may it would be related to the v4l2 driver of the nvidia cards.

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/jetson_xavier_camera_soft_archi.html#

Thank you for your time

[MartyG-RealSense]

Are you able to obtain a pointcloud if you define one in the roslaunch instruction instead of inside the launch file:

roslaunch realsense2_camera rs_camera.launch filters:=pointcloud

The RealSense ROS documentation provides an example of setting up a pointcloud using this instruction.

https://github.com/IntelRealSense/realsense-ros#point-cloud

I do not have advice that I can offer for diagnosing a setup with a non-RealSense camera.

[MartyG-RealSense]

Hi @loupdmer Do you require further assistance with this case, please? Thanks!

[loupdmer]

Hi @MartyG-RealSense, sorry for the time between my responses
Actually, I succeed in rebuilding kernel according to the previous github
https://github.com/jetsonhacks/buildJetsonXavierNXKernel

My issues with held broken packages was because updates was disabled.
I managed to rebuild, and I have the following results:

• Launching cheese fail.
• Launching demo_pointcloud.launch works, but at a rate of around 6Hz
• Launching rs_camera.launch with filters:=pointcloud fail, it produces the issue I got with rs_pointcloud.launch

I will follow the example you linked me, however, for my project, I only need topics from rs_camera.launch, and I can't spend more time on why a drop of frequency appears with demo_pointcloud.launch.

I hope this topic would help anyone else and I gratefully thank you for your help!

[MartyG-RealSense]

It is no problem at all @loupdmer - thanks very much for the update. I'm pleased to hear that you achieved a solution. Thanks very much for sharing your method for the benefit of the RealSense community. I will close this case now as you achieved a satisfactory outcome. Thanks again!