Performance when publishing pointcloud and align_depth topics using Jetson AGX Xavier L515

[hua-777]

Hardware
Jetson AGX Xavier
Intel RealSense LiDAR Camera L515

Issue
When running $ roslaunch realsense2_camera rs_camera.launch align_depth:=true filters:=pointcloud and using rviz to view the published streams I encounter a major performance hit (less than 1 frame per second) in all published topics. However when running point cloud and align_depth separately everything runs fine. I also tested this on the Jetson TX2 and Jetson Xavier NX platforms resulting with the same issue. However, when ran on an Intel NUC with an i5-6260 there is no performance issue when publishing both pointcloud and align_depth data.

I get a uvc streamer watchdog error that only happens when publishing both pointcloud and align_depth data.
21/07 15:43:09,523 ERROR [546803020160] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 130
21/07 15:43:09,523 ERROR [546777842048] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 131

I suspect that this is an architecture issue as when running realsense on Jetson devices there is a continuous warning stream that isn't present on x86 devices.
21/07 15:48:32,591 WARNING [547516039552] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
21/07 15:48:33,843 WARNING [547516039552] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
21/07 15:48:35,097 WARNING [547516039552] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11

Furthermore it isn't consuming all available CPU resources on the Jetson devices (around 25%), indicating that this likely isn't an issue with computing power.

Has anyone successfully published both pointcloud and align_depth data from a Jetson device? I would appreciate any advice.

Thank you!

[MartyG-RealSense]

Hi @derekhua777 If the control_transfer returned warning is generated continuously then it indicates that there is a serious communication problem with the device.

There is another currently open issue with point cloud generation and latency problems on Jetson at #1977

As a starting point in investigating your case, could you try adding initial_reset:=true to your roslaunch instruction if you are not already doing please, in order to reset the camera at launch. For example:

roslaunch realsense2_camera rs_camera.launch align_depth:=true filters:=pointcloud initial_reset:=true

Does performing a reset at launch reduce the latency, please?

[hua-777]

I tried running with initial_reset:=true but the result is the same. The uvc streamer watchdog error and control_transfer returned error are still both continuously generated and the latency is the same. This issue is present across all the devices I have tested on despite using multiple known good cables and lidars.

[MartyG-RealSense]

Do you have the option to test with a mains electricity powered USB hub on the Jetson boards if you have not done so already?

[hua-777]

I already tested that including plugging the L515 directly into the onboard USB ports on the Jetson Xavier NX and Jetson TX2.

[MartyG-RealSense]

Plugging the camera directly into a port on the Jetson would not count as 'mains powered' because it is drawing from the power supply of the Jetson rather than being supplied independently by the mains power-point. This would increase the chance of USB port instability, because it is a 'passive' USB connection rather than the more stable 'active' USB connection that a hub plugged into a mains power-point provides.

Can you confirm please that you tried a mains-powered USB hub (one with an independent power supply) as well as plugging directly into the Jetson, please?

[MartyG-RealSense]

Hi @derekhua777 Do you require further assistance with this case, please? Thanks!

[admantium-sg]

Hi @derekhua777, check my just opened issue #2014. I systematically tested various starting parameters, could get point cloud data published, albeit at on 3 fps.

[haesolPark]

Hi @MartyG-RealSense, I am going through the same performance issues. I am running L515 connected to Jetson Xavier AGX. Without the 'align_depth' option, the frame rate is about 11-13 fps (it is also too low I think), and when the option is turned on, the frame rate drops to 0.6-0.9... I have tried a 'mains powered' usb hub, you mentioned above, and the behavior is somewhat unreliable; sometimes the frame rate is maintained around 3-4 fps(still too low for my application), but after a few runs of roslaunch, it goes back to 0.6-0.9. Whenever I tried 'realsense_viewer' from the same hardware, however, the frame rate seems to be at least 30 fps whether the rgb option is turned on(the viewer automatically aligns depth and color in this case) or off. It makes me think that something is wrong with the ros wrapper... and I seriously need a help to fix it as soon as possible for my project. I think this issue can easily be reproduced if you use the Jetson (initialized using jetpack) and L515. Thanks.

[MartyG-RealSense]

Hi @haesolPark I recall a case where a RealSense user who had poor performance when generating a pointcloud found that their performance improved if they set RViz to use compressed data instead of raw data.

#1510 (comment)

[haesolPark]

Thank you for your advice @MartyG-RealSense. From the post you mentioned, adjusting the fps of depth stream seems to increase the fps of the pointcloud topic (reducing depth_fps to 5, then I get the pointcloud with fps 5). I think it would be the simplest workaround for now, but I believe there should be a software patch for this or at least technical explanation about this from the realsense development team. Regarding the rviz related things, I don't use rviz for my project since I don't need any visualization (the jetson would locate in a robot). Thank you anyway, and I hope to hear about the updated news about this.

[MartyG-RealSense]

@haesolPark The only supported FPS for depth on the L515 model is 30, so I'm not sure why the launch would have allowed depth_fps to be set at 5 FPS (the D455 model supports 5 FPS as a minimum).

My expectation if rs_camera.launch was used as the launch instruction would be that the launch would decide that a depth_fps definition of '5' was an invalid configuration and instead apply the L515 model's default configuration. You can check whether the launch has applied a custom configuration that you have defined by looking at the Done Setting Dynamic reconfig parameters section near the end of the launch log. If the settings are different from the custom-defined one (for example, you only defined a depth stream but there are both depth and color being published) then this indicates that your custom configuration was declined and overridden.

Below is an example of an L515 launch where the default configuration for it is being applied by the launch.

[MartyG-RealSense]

Hi @haesolPark Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received.