Install RealSense-SDK on the RK3588

[lantianhaohh]

Install realsense-sdk on rk3588, install realsense-sdk through source code compilation, refer to master/doc/installation_jetson.md and master/doc/installation.md documentation installation, after successful installation, connect three D435 cameras, open realsense-viewer debugging, run for a period of time, Realsense-viewer will not detect the camera connection, reopening realsense-viewer will not detect, you must restart rk3588 to detect the camera connection, when connecting a camera is no problem. Then I plugged in three d435 cameras, filled in the serial_no, tried to run roslaunch realsense2_camera rs_multiple_devices.launch, at first it worked fine, and after a while it showed "error handler, failed to set power state.error handler, usb device." disconnected.failed to submit UVC request, finally, No realsense device were found!"

[MartyG-RealSense]

Hi @lantianhaohh Adding additional RealSense cameras places extra processing burden on the computer and consumes more of its computing resources.

If it is possible for your project to be connected to a mains electricity wall socket then using a mains powered USB 3 hub to attach the cameras to can provide additional USB stability.

---

Jetson installation instructions should only be used with Nvidia Jetson boards, even though they are Arm-based like RK3588.

Computing boards that use Rockchip tend to be more like Raspberry Pi than a Jetson. For example, the Odroid-M1 board uses Rockchip, as does Firefly-RK3399.

What is the name of your computer board that has the RK3588 processor, please?

[lantianhaohh]

Hi @MartyG-RealSense ,The motherboard is Firefly's ITX-rk3588J

[lantianhaohh]

@MartyG-RealSense ，Which file should the installation method of RealSense SDK refer to? I feel that the power supply of the USB also has a part, but now it is very unstable, often No realsense device were found!, do you need to use kernel patching? Or adaptability?

[lantianhaohh]

It is relatively stable when using ROS to open one D435, but once you open multiple ones, the kernel version of rk3588 is 5.10.110, Ubuntu 20.04, is it a compatibility problem?

[MartyG-RealSense]

Thank you for the information. This motherboard has a strong hardware specification, and you have stated that the three cameras can run for a while before the Failed to set power state error occurs. This error indicates a problem with accessing a camera, perhaps because it is already being accessed by an application and is in a 'busy' state or the camera cannot be detected because of a problem with the USB connection.

Intel has a dedicated installation guide for the Firefly-RK3399 model at the link below with instructions for Android and Linux. ITX-3588J is also a Firefly model that supports Android and Linux.

https://dev.intelrealsense.com/docs/firefly-rk3399-installation

However, as your board can run normally for a time, it suggests to me that it is primarily a USB issue rather than an SDK installation issue.

Kernel 5.10 is not officially supported by the RealSense SDK. The SDK can work with unsupported kernels but there may be unpredictable consequences in regards to stability. The nearest supported kernel type is 5.11. In situations where there may be kernel instability, an RSUSB build can bypass the kernel, though RSUSB is not suited to multiple camera applications.

[lantianhaohh]

@MartyG-RealSense ,Three D435s can run normally for a short time, only one to two minutes, and most of the time it is impossible to run three at the same time, the logs are error handler, failed to set power state and error: resource temporarily unavailable, number 11.Most require a reboot to guarantee proper startup.Thank you so much.

[lantianhaohh]

I also want to ask a question, RSUSB and kernel patch installation is more stable

[lantianhaohh]

I also want to ask a question, RSUSB and kernel patch installation is more stable

[MartyG-RealSense]

It does strongly sound like a USB power stability issue that is temporarily corrected by resetting the cameras.

The Resource temporarily unavailable warning indicates that there may be a problem with communication between the board and the camera.

Are the cameras able to work if you unplug them from the USB ports and re-insert them instead of rebooting the board? An unplug-replug resets the camera.

RSUSB is typically stable on computers with kernel issues or unusual hardware / software specifications. RSUSB can work with multiple cameras, though a non-RSUSB kernel patched installation is ideal for multicam.

[lantianhaohh]

@MartyG-RealSense ,I just tried it, unplugged the camera from the USB port several times and plugged it back in, and the logs all showed that the No realsense device was found!You must restart to recognize it, but it shows resource temporarily unavailable warning, but RVIZ can display ROS topic messages, and camera image also displays normally, only if there is only one d435.

You said that non-RSUSB kernel patching installation is ideal for multiple cameras, our robot products require multiple D435 cameras, I used the ./scripts/patch-realsense-ubuntu-L4T.sh script in the master/doc/installation_jetson.md document to install the kernel patch was unsuccessful because there was no corresponding file, Maybe this kernel patch installation method does not apply to rk3588, can you teach me how to install kernel patch on rk3588? I didn't succeed for a long time, thank you.

[MartyG-RealSense]

Although RSUSB is not ideally suited for multicam, some RealSense users have found that it worked fine for them with multiple cameras, whilst others had problems. So it would be worth trying it to see how your Firefly board performs with it as RSUSB multicam performance seems to depend on the particular project.

The L4T patch script in the installation_jetson.md document is designed only for patching Jetson boards. However, the patch-realsense-ubuntu-lts.sh patch script in the installation.md instructions likely would not work either with your 5.10 kernel as it is not on the list of supported kernel versions.

In a past case at #2868 (comment) a RealSense team member recommended using -DFORCE_LIBUVC=true with RK devices. LIBUVC is the predecessor of RSUSB, so this statement supports using RSUSB with your board.

[lantianhaohh]

@MartyG-RealSense ,Can the patch-realsense-ubuntu-lts.sh patch script in the installation.md directive be solved by upgrading the kernel version? Here is the command cmake I compiled earlier: /-DFORCE_LIBUVC=true -DCMAKE_BUILD_TYPE=release, -DFORCE_LIBUVC=true has been added, and the only way to try it is the kernel patch. There should be a tutorial on kernel patching,thank you.

[MartyG-RealSense]

If you use kernel 5.11 then patch-realsense-ubuntu-lts.sh should be able to complete its installation. At the time of writing this, 5.11 is the most recent supported kernel, though there is an unofficial test patch for 5.13 and 5.15.

If you use DFORCE_LIBUVC=true or DFORCE_RSUSB=true then you should not patch the kernel afterwards as kernel patching is not required by this build method.

[lantianhaohh]

@MartyG-RealSense ,This script patch-realsense-ubuntu-lts.sh needs to be upgraded to kernel version 5.11, which should be suitable for x86 platforms, right? RK3588 is an ARM platform can be used like this?It is still necessary to compile through the linux SDK kernel, and then flash the firmware to rk3588,thank you.

[lantianhaohh]

I'd like to try the kernel patching method, maybe this one works better

[MartyG-RealSense]

For non-Jetson Arm devices where Ubuntu is beng used as the operating system, LIBUVC without patching is commonly the installation method used.

Odroid boards that are typically Arm-based have a dedicated patch script called patch-realsense-ubuntu-odroid.sh. The script is 3 years old at the time of writing this though.

As mentioned earlier though, your problems with camera detection are more likely related to USB power than the kernel, especially as a single camera can run normally but three do not.

[MartyG-RealSense]

Hi @lantianhaohh Do you require further assistance with this case please as you have a related case at #11050

[lantianhaohh]

Hi @MartyG-RealSense ,Yes, I would like to install the kernel patch to make d435 more stable, because it is very unstable now and often there is no connection.

[MartyG-RealSense]

Is it possible for you to use RSUSB so that the kernel is bypassed and does not need to be patched, please? This will only affect the librealsense part of your project and you can keep using kernel 5.10.110 if other aspects of your project are dependent on it.

[lantianhaohh]

@MartyG-RealSense ,Ok, thanks, I would like to ask, what is the main role of kernel patching? Whether it will have an impact on stability.

[MartyG-RealSense]

It mainly affects support for hardware timestamps. If hardware timestamp support is absent due to non-patching then software timestamps based on the computer's clock ('system time) are still available.

There is also a chance of 'fourcc code' warning messages such as Pixel format ... likely requires patch for fourcc code RW16! occurring, as described at #4971

For further detailed information about using RSUSB or kernel patching, please visit #5212 (comment) and scroll down through the linked-to comment to the section headed What are the advantages and disadvantages of using libuvc vs patched kernel modules?

[MartyG-RealSense]

Hi @lantianhaohh Do you require further assistance with this case, please? Thanks!

[MartyG-RealSense]

Case closed due to no further comments received. I refer readers of this case to #11050