'Unknown symbol in module' when trying to load custom kernel backend in Jetson

[charlielito]

Required Info
Camera Model	435i
Firmware Version	NA
Operating System & Version	Linux Ubuntu 20.04
Kernel Version (Linux Only)	4.9.140 and l4t-r32.4.2
Platform	NVIDIA Jetson AGX
SDK Version	v2.41.0
Language	NA
Segment	Robot

In addition, I am running everything in a Docker container using Balena platform https://www.balena.io/. In a nutshell it runs in the host device a basic Yocto linux distribution with the l4t necessary files and runs a custom Docker image.

Issue Description

I'm trying to build from source using the native backend following this tutorial. I am running this script.

Everything runs smoothly and the kernels are successfully compiled. The resulting .ko files are these:

root@0a57d74:~# ls
tegra-l4t-r32.3.1-hid-sensor-accel-3d.ko    tegra-l4t-r32.3.1-hid-sensor-trigger.ko  tegra-l4t-r32.3.1-videobuf-vmalloc.ko
tegra-l4t-r32.3.1-hid-sensor-gyro-3d.ko     tegra-l4t-r32.3.1-uvcvideo.ko
tegra-l4t-r32.3.1-hid-sensor-iio-common.ko  tegra-l4t-r32.3.1-videobuf-core.ko

Nonetheless, I am having trouble in the last part of the script where the kernel modules are supposed to be loaded with depmod. Since we are running inside a docker container and using Balena, they do not allow the user to use depmod or modprob because /lib/modules/4.9.140-l4t-r32.4.2 is read-only. They propose instead that the user loads the kernel using insmod. So I tried to load manually the .ko. I was able to load all modules except for the ones corresponding to hid-sensor*.

The error for example looks like:

root@0a57d74:~# insmod tegra-l4t-r32.3.1-hid-sensor-trigger.ko 
insmod: ERROR: could not insert module tegra-l4t-r32.3.1-hid-sensor-trigger.ko: Unknown symbol in module

Inspecting the errors with dmesg we get:

[ 1575.470310] USB Video Class driver (1.1.1)
[ 1578.284849] hid_sensor_accel_3d: Unknown symbol hid_sensor_parse_common_attributes (err 0)
[ 1578.284943] hid_sensor_accel_3d: Unknown symbol hid_sensor_write_raw_hyst_value (err 0)
[ 1578.284961] hid_sensor_accel_3d: Unknown symbol hid_sensor_setup_trigger (err 0)
[ 1578.284989] hid_sensor_accel_3d: Unknown symbol sensor_hub_register_callback (err 0)
[ 1578.285012] hid_sensor_accel_3d: Unknown symbol hid_sensor_remove_trigger (err 0)
[ 1578.285030] hid_sensor_accel_3d: Unknown symbol hid_sensor_read_raw_hyst_value (err 0)
[ 1578.285079] hid_sensor_accel_3d: Unknown symbol sensor_hub_remove_callback (err 0)
[ 1578.285141] hid_sensor_accel_3d: Unknown symbol sensor_hub_input_attr_get_raw_value (err 0)
[ 1578.285216] hid_sensor_accel_3d: Unknown symbol sensor_hub_input_get_attribute_info (err 0)
[ 1578.285426] hid_sensor_accel_3d: Unknown symbol hid_sensor_pm_ops (err 0)
[ 1578.285472] hid_sensor_accel_3d: Unknown symbol hid_sensor_power_state (err 0)
[ 1578.285521] hid_sensor_accel_3d: Unknown symbol hid_sensor_format_scale (err 0)
[ 1578.285637] hid_sensor_accel_3d: Unknown symbol hid_sensor_write_samp_freq_value (err 0)
[ 1578.285686] hid_sensor_accel_3d: Unknown symbol hid_sensor_read_samp_freq_value (err 0)
[ 1950.815702] hid_sensor_trigger: Unknown symbol sensor_hub_set_feature (err 0)
[ 1950.815771] hid_sensor_trigger: Unknown symbol hid_sensor_get_usage_index (err 0)
[ 1950.815803] hid_sensor_trigger: Unknown symbol sensor_hub_device_close (err 0)
[ 1950.815823] hid_sensor_trigger: Unknown symbol hid_sensor_read_poll_value (err 0)
[ 1950.815873] hid_sensor_trigger: Unknown symbol sensor_hub_device_open (err 0)
[ 1950.815888] hid_sensor_trigger: Unknown symbol sensor_hub_get_feature (err 0)
[ 2002.158082] hid_sensor_iio_common: Unknown symbol sensor_hub_set_feature (err 0)
[ 2002.158111] hid_sensor_iio_common: Unknown symbol sensor_hub_input_get_attribute_info (err 0)
[ 2002.158133] hid_sensor_iio_common: Unknown symbol sensor_hub_get_feature (err 0)
[ 2196.647894] hid_sensor_gyro_3d: Unknown symbol hid_sensor_parse_common_attributes (err 0)
[ 2196.647978] hid_sensor_gyro_3d: Unknown symbol hid_sensor_write_raw_hyst_value (err 0)
[ 2196.648004] hid_sensor_gyro_3d: Unknown symbol hid_sensor_setup_trigger (err 0)
[ 2196.648042] hid_sensor_gyro_3d: Unknown symbol sensor_hub_register_callback (err 0)
[ 2196.648071] hid_sensor_gyro_3d: Unknown symbol hid_sensor_remove_trigger (err 0)
[ 2196.648092] hid_sensor_gyro_3d: Unknown symbol hid_sensor_read_raw_hyst_value (err 0)
[ 2196.648126] hid_sensor_gyro_3d: Unknown symbol sensor_hub_remove_callback (err 0)
[ 2196.648196] hid_sensor_gyro_3d: Unknown symbol sensor_hub_input_attr_get_raw_value (err 0)
[ 2196.648219] hid_sensor_gyro_3d: Unknown symbol sensor_hub_input_get_attribute_info (err 0)
[ 2196.648241] hid_sensor_gyro_3d: Unknown symbol hid_sensor_pm_ops (err 0)
[ 2196.648264] hid_sensor_gyro_3d: Unknown symbol hid_sensor_power_state (err 0)
[ 2196.648310] hid_sensor_gyro_3d: Unknown symbol hid_sensor_format_scale (err 0)
[ 2196.648347] hid_sensor_gyro_3d: Unknown symbol hid_sensor_write_samp_freq_value (err 0)
[ 2196.648375] hid_sensor_gyro_3d: Unknown symbol hid_sensor_read_samp_freq_value (err 0)

Can anyone guess how to properly load those modules? Or any hint on the other kernel needed to load these?

[charlielito]

Update

I was missing the right Module.symvers because my system didn't have any /usr/src/linux-headers-4.9.140-tegra-ubuntu18.04_aarch64/kernel-4.9 folder. Also the patch script was downloading the wrong L4T version (32.3.1) because there is also no /etc/nv_tegra_release so I set it that manually. I fixed it to download the right version (32.4.2) that matches the uname -r. Also the Module.symvers file was downloaded from https://developer.nvidia.com/embedded/L4T/r32_Release_v4.2/t186ref_release_aarch64/Tegra186_Linux_R32.4.2_aarch64.tbz2. Nonetheless I am having a different problem.

when trying to insert de module I got:

insmod: ERROR: could not insert module tegra-l4t-r32.4.2-hid-sensor-iio-common.ko: Invalid module format

And dmesg shows:

[  180.874804] hid_sensor_iio_common: disagrees about version of symbol module_layout

I really don't understand really well how kernel compilation works so maybe I am doing something wrong. It would be of much help if someone that manages this topic could help. thanks!

[ev-mp]

@charlielito hello,
We do not run and test "containerized" Jetson , so we cannot check and address it directly.
Based on experience with AGX Xavier, the specific error was fired when trying to insert the newly compiled module without performing depmod first. As you've noticed it may be indeed prohibited in Docker.

On the other hand, D435i have received native kernel support with the release of L4T 4.4.1 (rev 32.4.4) and 4.5, as the specific Librealsense patches were integrated into the kernel.
Thus if you upgrade the revision you might not need to perform the patching process. #8254

[charlielito]

Thanks @ev-mp for the helpful information! I tried with L4T 4.4.1 (rev 32.4.4) and it works like a charm!