Warnings on Realsense Viewer initialization

[evercoast-smm]

Camera Model: D415
Firmware Version: 05.11.15.00
OS: Ubuntu 18.04 with kernel 4.16.18-041618-generic
Platform: PC
SDK Version: librealsense-2.29.0
Segment: Telecommunications

We're seeing 3 warnings generated by the Realsense Viewer:

02/10 14:06:54,284 WARNING [140512100869888] (backend-v4l2.cpp:1148) Pixel format 20303157-0000-0010-8000-00aa003 likely requires patch for fourcc code W10 !

02/10 14:06:54,303 WARNING [140512100869888] (backend-v4l2.cpp:1148) Pixel format 36315752-1a66-a242-9065-d01814a likely requires patch for fourcc code RW16!

02/10 14:07:05,652 WARNING [140511180482304] (ds5-timestamp.cpp:64) UVC metadata payloads not available. Please refer to the installation chapter for details.

Our primary questions is:

• What is the severity of these warnings? Are these indicative of a problematic machine configuration. And if so, how can this be rectified?

Additional info:

Please note that we've also filed this issue, which could be related: #4970

Also, per (#4815) we note that it's been suggested that these issues could be related to the DKMS, so here's our apt show for the librealsense2-dkms:

evercoast@evercoast:~$ apt show librealsense2-dkms -a
Package: librealsense2-dkms
Version: 1.3.6-0ubuntu0
Priority: extra
Section: kernel
Source: librealsense2
Maintainer: Evgeni Raikhel <evgeni.raikhel@intel.com>
Installed-Size: 22.1 MB
Depends: dkms (>= 2.1.0.0)
Recommends: librealsense2-udev-rules
Homepage: https://github.com/IntelRealSense/librealsense
Download-Size: 4,122 kB
APT-Manual-Installed: yes
APT-Sources: http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo bionic/main amd64 Packages
Description: Modified kernel modules for librealsense2
 uvcvideo - Depth-related 4CC streaming formats definitions and metadata attributes
 videodev - Registering 4CC with ioctl
 hid-sensor-gyro-3d - timestamps for gyro samples
 hid-sensor-accel-3d - timestamps for accelerometer samples
  .
 The package uses the DKMS system to automatically compile and install
 the modified driver on your current and future kernels.
 .
 On Debian and Ubuntu systems, supplied driver takes precedence
 over the existing driver by virtue of module path order.

Package: librealsense2-dkms
Version: 1.3.5-0ubuntu1
Priority: extra
Section: kernel
Source: librealsense2
Maintainer: Evgeni Raikhel <evgeni.raikhel@intel.com>
Installed-Size: 51.7 MB
Depends: dkms (>= 2.1.0.0)
Recommends: librealsense2-udev-rules
Homepage: https://github.com/IntelRealSense/librealsense
Download-Size: 6,583 kB
APT-Sources: http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo bionic/main amd64 Packages
Description: Modified kernel modules for librealsense2
 uvcvideo - Depth-related 4CC streaming formats definitions and metadata attributes
 videodev - Registering 4CC with ioctl
 hid-sensor-gyro-3d - timestamps for gyro samples
 hid-sensor-accel-3d - timestamps for accelerometer samples
  .
 The package uses the DKMS system to automatically compile and install
 the modified driver on your current and future kernels.
 .
 On Debian and Ubuntu systems, supplied driver takes precedence
 over the existing driver by virtue of module path order.

Package: librealsense2-dkms
Version: 1.3.4-0ubuntu1
Priority: extra
Section: kernel
Source: librealsense2
Maintainer: Evgeni Raikhel <evgeni.raikhel@intel.com>
Installed-Size: 51.7 MB
Depends: dkms (>= 2.1.0.0)
Recommends: librealsense2-udev-rules
Homepage: https://github.com/IntelRealSense/librealsense
Download-Size: 6,538 kB
APT-Sources: http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo bionic/main amd64 Packages
Description: Modified kernel modules for librealsense2
 uvcvideo - Depth-related 4CC streaming formats definitions and metadata attributes
 videodev - Registering 4CC with ioctl
 hid-sensor-gyro-3d - timestamps for gyro samples
 hid-sensor-accel-3d - timestamps for accelerometer samples
  .
 The package uses the DKMS system to automatically compile and install
 the modified driver on your current and future kernels.
 .
 On Debian and Ubuntu systems, supplied driver takes precedence
 over the existing driver by virtue of module path order.

Package: librealsense2-dkms
Version: 1.3.3-0ubuntu1
Priority: extra
Section: kernel
Source: librealsense2
Maintainer: Evgeni Raikhel <evgeni.raikhel@intel.com>
Installed-Size: 51.7 MB
Depends: dkms (>= 2.1.0.0)
Recommends: librealsense2-udev-rules
Homepage: https://github.com/IntelRealSense/librealsense
Download-Size: 6,537 kB
APT-Sources: http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo bionic/main amd64 Packages
Description: Modified kernel modules for librealsense2
 uvcvideo - Depth-related 4CC streaming formats definitions and metadata attributes
 videodev - Registering 4CC with ioctl
 hid-sensor-gyro-3d - timestamps for gyro samples
 hid-sensor-accel-3d - timestamps for accelerometer samples
  .
 The package uses the DKMS system to automatically compile and install
 the modified driver on your current and future kernels.
 .
 On Debian and Ubuntu systems, supplied driver takes precedence
 over the existing driver by virtue of module path order.

Package: librealsense2-dkms
Version: 1.3.2-0ubuntu4
Priority: extra
Section: kernel
Source: librealsense2
Maintainer: Evgeni Raikhel <evgeni.raikhel@intel.com>
Installed-Size: 51.7 MB
Depends: dkms (>= 2.1.0.0)
Recommends: librealsense2-udev-rules
Homepage: https://github.com/IntelRealSense/librealsense
Download-Size: 6,537 kB
APT-Sources: http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo bionic/main amd64 Packages
Description: Modified kernel modules for librealsense2
 uvcvideo - Depth-related 4CC streaming formats definitions and metadata attributes
 videodev - Registering 4CC with ioctl
 hid-sensor-gyro-3d - timestamps for gyro samples
 hid-sensor-accel-3d - timestamps for accelerometer samples
  .
 The package uses the DKMS system to automatically compile and install
 the modified driver on your current and future kernels.
 .
 On Debian and Ubuntu systems, supplied driver takes precedence
 over the existing driver by virtue of module path order.

Package: librealsense2-dkms
Version: 1.3.1-0ubuntu3
Priority: extra
Section: kernel
Source: librealsense2
Maintainer: Evgeni Raikhel <evgeni.raikhel@intel.com>
Installed-Size: 41.5 MB
Depends: dkms (>= 2.1.0.0)
Recommends: librealsense2-udev-rules
Homepage: https://github.com/IntelRealSense/librealsense
Download-Size: 5,276 kB
APT-Sources: http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo bionic/main i386 Packages
Description: Modified kernel modules for librealsense2
 uvcvideo - Depth-related 4CC streaming formats definitions and metadata attributes
 videodev - Registering 4CC with ioctl
 hid-sensor-gyro-3d - timestamps for gyro samples
 hid-sensor-accel-3d - timestamps for accelerometer samples
  .
 The package uses the DKMS system to automatically compile and install
 the modified driver on your current and future kernels.
 .
 On Debian and Ubuntu systems, supplied driver takes precedence
 over the existing driver by virtue of module path order.

Package: librealsense2-dkms
Version: 1.3.0-0ubuntu2
Priority: extra
Section: kernel
Source: librealsense2
Maintainer: Evgeni Raikhel <evgeni.raikhel@intel.com>
Installed-Size: 41.5 MB
Depends: dkms (>= 2.1.0.0)
Recommends: librealsense2-udev-rules
Homepage: https://github.com/IntelRealSense/librealsense
Download-Size: 5,275 kB
APT-Sources: http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo bionic/main i386 Packages
Description: Modified kernel modules for librealsense2
 uvcvideo - Depth-related 4CC streaming formats definitions and metadata attributes
 videodev - Registering 4CC with ioctl
 hid-sensor-gyro-3d - timestamps for gyro samples
 hid-sensor-accel-3d - timestamps for accelerometer samples
  .
 The package uses the DKMS system to automatically compile and install
 the modified driver on your current and future kernels.
 .
 On Debian and Ubuntu systems, supplied driver takes precedence
 over the existing driver by virtue of module path order.

Package: librealsense2-dkms
Version: 1.3.0-0ubuntu2
Priority: extra
Section: kernel
Source: librealsense2
Maintainer: Evgeni Raikhel <evgeni.raikhel@intel.com>
Installed-Size: 41.5 MB
Depends: dkms (>= 2.1.0.0)
Recommends: librealsense2-udev-rules
Homepage: https://github.com/IntelRealSense/librealsense
Download-Size: 5,275 kB
APT-Sources: http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo xenial/main i386 Packages
Description: Modified kernel modules for librealsense2
 uvcvideo - Depth-related 4CC streaming formats definitions and metadata attributes
 videodev - Registering 4CC with ioctl
 hid-sensor-gyro-3d - timestamps for gyro samples
 hid-sensor-accel-3d - timestamps for accelerometer samples
  .
 The package uses the DKMS system to automatically compile and install
 the modified driver on your current and future kernels.
 .
 On Debian and Ubuntu systems, supplied driver takes precedence
 over the existing driver by virtue of module path order.

Package: librealsense2-dkms
Version: 1.2.0-0ubuntu5
Priority: extra
Section: kernel
Source: librealsense2
Maintainer: Evgeni Raikhel <evgeni.raikhel@intel.com>
Installed-Size: 30.7 MB
Depends: dkms (>= 2.1.0.0)
Recommends: librealsense2-udev-rules
Homepage: https://github.com/IntelRealSense/librealsense
Download-Size: 3,910 kB
APT-Sources: http://realsense-hw-public.s3.amazonaws.com/Debian/apt-repo xenial/main i386 Packages
Description: Modified kernel modules for librealsense2
 uvcvideo - Depth-related 4CC streaming formats definitions and metadata attributes
 videodev - Registering 4CC with ioctl
 hid-sensor-gyro-3d - timestamps for gyro samples
 hid-sensor-accel-3d - timestamps for accelerometer samples
  .
 The package uses the DKMS system to automatically compile and install
 the modified driver on your current and future kernels.
 .
 On Debian and Ubuntu systems, supplied driver takes precedence
 over the existing driver by virtue of module path order.

[kjacks21]

@evercoast-smm have you tried streaming? I just updated to librealsense-2.29.0 as well and get the following warnings:

 02/10 15:15:27,977 WARNING [139777518278400] (backend-v4l2.cpp:1148) Pixel format 36315752-1a66-a242-9065-d01814a likely requires patch for fourcc code RW16!
 02/10 15:15:27,986 ERROR [139777518278400] (types.h:304) xioctl(VIDIOC_S_CTRL) failed Last Error: Invalid argument
 02/10 15:15:27,986 WARNING [139777518278400] (sensor.cpp:766) Exception was thrown when inspecting RGB Camera property Controls exposure time of color camera. Setting any value will disable auto exposure
 02/10 15:15:27,995 ERROR [139777518278400] (types.h:304) xioctl(VIDIOC_S_CTRL) failed Last Error: Invalid argument
 02/10 15:15:27,995 WARNING [139777518278400] (sensor.cpp:766) Exception was thrown when inspecting RGB Camera property Controls white balance of color image. Setting any value will disable auto white balance

When I try to record data, I get many warnings (example below):

 02/10 15:19:48,683 WARNING [139833687123712] (ds5-timestamp.cpp:64) UVC metadata payloads not available. Please refer to the installation chapter for details.
 02/10 15:19:50,686 WARNING [139833598015232] (ds5-timestamp.cpp:64) UVC metadata payloads not available. Please refer to the installation chapter for details.
 02/10 15:19:53,650 ERROR [139833583331072] (record_device.cpp:210) Failed to downcast snapshot of type librealsense::debug_interface
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Motion
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Video
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Roi
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Depth Frame
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Advanced Mode
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Video Profile
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Disparity Frame
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Motion Profile
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Pose Frame
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Pose Profile
 02/10 15:19:53,651 WARNING [139833583331072] (record_device.cpp:255) Extensions type is unhandled: Tm2
...

I'm able to view the streams and record, but when I try to access the depth frames from the .bag file using the API, I get an error. For example, the trace below:

 depth_sensor = profile.get_device().first_depth_sensor()
RuntimeError: Could not find requested sensor type!

This all used to work on my system but no longer, and the only difference to my system is SDK and firmware updates.

[ev-mp]

@evercoast-smm , the DKMS packages are developed and maintained for Ubuntu LTS kernels only.
The warnings regarding the metadata are indeed due to the lack of the patched kernel. As mentioned in #4970 we do not provide kernel patches for Ubuntu 18 with kernel 4.16. But I think that adopting the patches for Ubuntu 18 can be achieved by creating a copy of existing kernel 4.16 patches (for Xenial) and renaming the file name so instead of xenial it will include bionic token:
realsense-camera-formats_ubuntu-xenial-v4.16.patch
realsense-camera-formats_ubuntu-bionic-v4.16.patch

[ev-mp]

@evercoast-smm, do you need more assistance with 4.16 kernel ?

[evercoast-smm]

Hi @ev-mp, per your suggestion we've upgraded our kernel version to 4.18.20-041820-generic.

We still have issues that are described by ticket (#4972), I'll close the current ticket here, but would very much appreciate help with #4972.

Thank you!