RS utils attempt to open other cameras as rs devices

[jordisolanas-bc]

---

Required Info
Camera Model	D435
Firmware Version	5.15.1
Operating System & Version	Ubuntu 20.04
Kernel Version (Linux Only)	5.10.120
Platform	NVIDIA Jetson
SDK Version	N/A
Language	C++/Python
Segment	Robot

Issue Description

I have a robot with two D435 cameras connected. The robot is running a patched kernel 5.10.120 following Installation_Jetson.md instructions. I have installed librealsense2 and librealsense2-utils (v2.54.2) using the official debians from your repository (not built from source).

If I only have the two D435 cameras connected and I run rs-enumerate-devices -s, I get the following output (no errors):

$> rs-enumerate-devices -s
Device Name                   Serial Number       Firmware Version    
Intel RealSense D435          152222070092        5.15.1              
Intel RealSense D435          153222071259        5.15.1 

But when I connect an RGB camera from a different vendor (e-CON imx412 sensor), while keeping my D435 cameras connected, I get the following error:

$> rs-enumerate-devices -s
 31/10 21:17:39,282 ERROR [281472918759680] (librealsense-exception.h:52) xioctl(VIDIOC_S_EXT_CTRLS) failed Last Error: Invalid argument
Could not create device - xioctl(VIDIOC_S_EXT_CTRLS) failed Last Error: Invalid argument . Check SDK logs for details
Device Name                   Serial Number       Firmware Version    
Intel RealSense D435          152222070092        5.15.1              
Intel RealSense D435          153222071259        5.15.1 

In other reported issues, xioctl(VIDIOC_S_EXT_CTRLS) is considered a kernel conflict. But my D435 are working correctly and the error doesn't show if this other RGB camera is disconnected. Also, using pyrealsense2.context().query_devices() with two realsenses and one e-CON RGB camera returns 3 devices instead of 2 (but if I disconnect the e-CON RGB camera, it reports 2 devices).

How can I ensure rs utils don't try to open other cameras as realsenses? Trying to get rid of the error message.

[MartyG-RealSense]

Hi @jordisolanas-bc Non-RealSense cameras that are attached to a computer are treated as a platform camera by librealsense. With such cameras, only the RGB stream is usually accessible by the librealsense SDK.

The Python script realsense_device_manager.py from the project 'box_dimensioner_multicam.py' provides an example of identifying whether a camera is a platform camera or not by querying the camera device's name.

https://github.com/IntelRealSense/librealsense/blob/master/wrappers/python/examples/box_dimensioner_multicam/realsense_device_manager.py#L44-L50

In the code highlighted in the above link, if an attached camera's name is not 'platform camera' then the script should ignore the non-RealSense camera and only access the RealSense ones.

In regard to tools such as rs-enumerate-devices though, there likely is not anything that can be done to avoid an attached platform camera generating an error unless you alter the tool's C++ source code to ignore platform cameras and then re-compile the customized program. A C++ project at the link below provides an example of querying the camera name.

https://www.codeproject.com/Articles/1233892/FluentRealSense-The-First-Steps-to-a-Simpler-RealS

[jordisolanas-bc]

Hi @MartyG-RealSense Thank you for getting back to me. I tried that same logic on my robot, executing line by line:

>>> import pyrealsense2 as rs
>>> context = rs.context()
>>> for d in context.devices:
...     print(d.get_info(rs.camera_info.name))
... 
Intel RealSense D435
Intel RealSense D435
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: xioctl(VIDIOC_S_EXT_CTRLS) failed Last Error: Invalid argument

As you can see above, the two realsense cameras can be accessed correctly when iterating on context.devices. But the moment I try to iterate on the 3rd device, I get the xioctl failure. I understand there's also no option to avoid that error showing up?

[MartyG-RealSense]

If all of the RealSense cameras that you are using are the D435 model, how about using an If check to print only the devices named D435.

for d in context.devices:
if d.get_info(rs.camera_info.name) = 'D435':
print(d.get_info(rs.camera_info.name))

[jordisolanas-bc]

The error appears by just iterating context.devices. Even if I do nothing inside the for loop, we get the error:

import pyrealsense2 as rs
context=rs.context()
for d in context.devices:
    pass

Result:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: xioctl(VIDIOC_S_EXT_CTRLS) failed Last Error: Invalid argument

So adding the check of name D435 doesn't resolve the issue. context.devices is a pyrealsense2.pyrealsense2.device_list. When I iterate the list, d is type <class 'pyrealsense2.pyrealsense2.device'>. I understand the error comes from the constructor of that class.

Example:

for d in context.devices:
    print(type(d))

Result:

<class 'pyrealsense2.pyrealsense2.device'>
<class 'pyrealsense2.pyrealsense2.device'>
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: xioctl(VIDIOC_S_EXT_CTRLS) failed Last Error: Invalid argument

[MartyG-RealSense]

As you know the serial numbers of your two D435 cameras, you should be able to access only those cameras specifically instead of automatically detecting all attached cameras with a context instruction.

I can appreciate though that this may not be a suitable solution if your project needs to connect D435 camera units other than those two because of the inflexibility of having to manually specify the serial numbers.

#1735 (comment) has a Python script that demonstrates accessing individual cameras by their serial number and defining a separate cfg configuration for each camera if you wish to do so.

[jordisolanas-bc]

Unfortunately, hardcoding the serial number of the cameras is not an option as this is part of a SW release deployed to a fleet of robots. We are implementing the realsense FW update logic.

If there's no option to get rid of the error log, please feel free to close the issue.
Thank you for your support anyways! At least we confirmed that it is expected behavior 👍

[MartyG-RealSense]

I looked over the details of this case again from the beginning.

RealSense cameras and non-RealSense UVC webcams should be able to exist alongside each other attached to the same computer. Conflicts can occur between RealSense cameras and webcams but such cases are rare and usually take the form of either the RealSense camera or the webcam not being detected. That is not the case in this particular situation as all cameras are detected.

I note that you patched the kernel after installing librealsense from Jetson packages. You do not actually need to apply the kernel patch to packages as the patch is already included within the packages.

So it may be worth updating your 5.10 kernel to a slightly newer 5.10 version if possible so that the replacement kernel is unpatched, or doing a fresh installation on your Jetson without patching the kernel this time.

An alternative would be building librealsense from source code with CMake instead of packages and including the CMake build flag -DFORCE_RSUSB_BACKEND=TRUE so that the SDK is built in RSUSB mode. RSUSB is not ideal for use with multiple camera projects but it can be used with multicam and you may not see a problem if only using two RealSense cameras with it. An RSUSB build of the SDK will bypass the kernel and so potentially help to avoid xioctl(VIDIOC_S_EXT_CTRLS) errors.

[MartyG-RealSense]

Hi @jordisolanas-bc Do you require further assistance with this case, please? Thanks!

[jordisolanas-bc]

Hi @MartyG-RealSense. I use kernel 5.10 as it's the one that comes with Jetpack 5.1. That prevents us from using a newer version. On top of the kernel 5.10 published by Nvidia, I have applied the following patches:

• scripts/Tegra/LRS_Patches/05-realsense-powerlinefrequency-control-fix.patch
• scripts/Tegra/LRS_Patches/02-realsense-metadata-L4T-5.0.2.patch
• scripts/Tegra/LRS_Patches/01-realsense-camera-formats-L4T-5.0.2.patch
  Those 3 patches applied successfully on kernel (confirmed original source code didn't contain the fixes already).
  I didn't apply this other patch, because kernel 5.10 already had this fix:
• scripts/Tegra/LRS_Patches/04-media-uvcvideo-mark-buffer-error-where-overflow.patch

Finally, I installed librealsense2 and librealsense2-utils from this repo: "deb https://librealsense.intel.com/Debian/apt-repo $(lsb_release -cs) main" as specified in doc/installation_jetson.md. Could you please clarify what you mean by "You do not actually need to apply the kernel patch to packages as the patch is already included within the packages". Does that mean that steps in doc/installation_jetson.md are not accurate?

Regarding building with -DFORCE_RSUSB_BACKEND=TRUE we would strongly prefer to avoid it due to performance concerns.

[MartyG-RealSense]

Sincere apologies for the delay in responding further.

Packages have the kernel patch bundled within them but a source code installation does not. So the Jetson instruction page is accurate as the package section of the instructions does not mention kernel patches. Only the 'Building from source using native backend' section does.

So if you installed from packages then the ./scripts/patch-realsense-ubuntu-L4T.sh patch script should not be applied to the kernel.

It is also recommended that you only have one type of librealsense installation on your computer. So if you choose to install from packages then you should not also build librealsense from source code on the same computer, otherwise you may have multiple librealsense installations that conflict with one another.

[jordisolanas-bc]

I'm installing librealsense2_2.54.2-0~realsense.2181_arm64.deb and librealsense2-utils_2.54.2-0~realsense.2181_arm64.deb. If I check the content of the package, there's just the .so and all the /usr/bin/rs-* applications. Where's the kernel patch bundled within? Is it a different package?

$> dpkg -c librealsense2_2.54.2-0~realsense.2181_arm64.deb
drwxr-xr-x root/root         0 2023-09-28 00:48 ./
drwxr-xr-x root/root         0 2023-09-28 00:48 ./usr/
drwxr-xr-x root/root         0 2023-09-28 00:48 ./usr/lib/
drwxr-xr-x root/root         0 2023-09-28 00:48 ./usr/lib/aarch64-linux-gnu/
-rw-r--r-- root/root  10304528 2023-09-28 00:48 ./usr/lib/aarch64-linux-gnu/librealsense2.so.2.54.2
drwxr-xr-x root/root         0 2023-09-28 00:48 ./usr/share/
drwxr-xr-x root/root         0 2023-09-28 00:48 ./usr/share/doc/
drwxr-xr-x root/root         0 2023-09-28 00:48 ./usr/share/doc/librealsense2/
-rw-r--r-- root/root      1257 2023-09-28 00:48 ./usr/share/doc/librealsense2/changelog.Debian.gz
-rw-r--r-- root/root      6743 2023-09-28 00:48 ./usr/share/doc/librealsense2/copyright
lrwxrwxrwx root/root         0 2023-09-28 00:48 ./usr/lib/aarch64-linux-gnu/librealsense2.so.2.54 -> librealsense2.so.2.54.2

In any case, I tried removing the kernel patches and rebuilding the kernel as you suggested. I installed librealsense2 and librealsense2-utils, and same issue.

[MartyG-RealSense]

There is no information available unfortunately about how the patch is bundled in the packages or what form it takes, just that it is bundled.

The installation_jetson.md instructions for Jetson have different packages to the distribution_linux.md package instructions for PC computers. Instead of installing the librealsense2 package, the packages that should be installed on Jetson are librealsense2-utils and librealsense2-dev

[MartyG-RealSense]

Hi @jordisolanas-bc Do you require further assistance with this case, please? Thanks!

[jordisolanas-bc]

Hi @MartyG-RealSense,
Apologies for the delay, I was OOO. We install librealsense2-dev in our development environment and librealsense2 in our runtime environment (together with librealsense2-utils in both cases). I checked librealsense2-dev content, and I don't see anything that could be patching the kernel on it either.

In any case, this is not a blocker and cameras are working (it's just annoying to get the errors). If there's no other idea, we can close this issue. Thanks!

[MartyG-RealSense]

I do not have further ideas to suggest, so I will close this issue. Thanks again!