build px4_fmu-vX_rtps fails at generate_microRTPS_bridge.py using docker image px4-dev-nuttx:latest

[tschiu]

Describe the bug
make px4_fmu-v3_rtps
make px4_fmu-v4_rtps
make px4_fmu-v5 rtps
all fail with the following error

Traceback (most recent call last):
File "/src/Firmware/msg/tools/generate_microRTPS_bridge.py", line 401, in
generate_agent(agent_out_dir)
File "/src/Firmware/msg/tools/generate_microRTPS_bridge.py", line 334, in generate_agent
"fastrtpsgen not found. Specify the location of fastrtpsgen with the -f flag")
Exception: fastrtpsgen not found. Specify the location of fastrtpsgen with the -f flag

To Reproduce
I am running a fresh Ubuntu 16.04 Server VM with Apache2 and MySQL only installed (it's a clone of my standard LAMP server). Server is still on 16.04 and hasn't been upgraded to 18.04 yet.

Install docker using script found here https://dev.px4.io/v1.9.0/en/test_and_ci/docker.html

Exact script below:

curl -fsSL get.docker.com -o get-docker.sh
sudo sh get-docker.sh

Added me to docker group:

Create docker group (may not be required)

sudo groupadd docker

Add your user to the docker group.

sudo usermod -aG docker $USER

Log in/out again before using docker!

logout
...
login with bash

get dev-nuttx image

docker pull px4io/px4-dev-nuttx

get PX4 source

cd ~
mkdir src
cd src
git clone https://github.com/PX4/Firmware.git

run the container with bash shell

docker run -it --privileged --env=LOCAL_USER_ID="$(id -u)" -v ~/src/Firmware:/src/Firmware:rw -v /tmp/.X11-unix:/tmp/.X11-unix:ro -e DISPLAY=:0 -p 14556:14556/udp --name=px4nuttx px4io/px4-dev-nuttx:latest bash

docker container created and are in bash shell

user@6c4a4043cbca:/#

verify fastrtpsgen is available

root@6c4a4043cbca:/src/Firmware# fastrtpsgen
openjdk version "1.8.0_181"
OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-0ubuntu0.18.04.1-b13)
OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode)
ERROR: No input files given
fastrtpsgen usage:
fastrtpsgen [options] [ ...]
where the options are:
-help: shows this help
-version: shows the current version of eProsima Fast RTPS.
-example : Generates a solution for a specific platform (example: x64Win64VS2015)
Supported platforms:
* i86Win32VS2013
* x64Win64VS2013
* i86Win32VS2015
* x64Win64VS2015
* i86Linux2.6gcc
* x64Linux2.6gcc
-replace: replaces existing generated files.
-ppDisable: disables the preprocessor.
-ppPath: specifies the preprocessor path.
-d : sets an output directory for generated files.
-t : sets a specific directory as a temporary directory.
and the supported input files are:
* IDL files.

get fastrtpsgen location

root@6c4a4043cbca:/src/Firmware# which fastrtpsgen
/usr/local/bin/fastrtpsgen

set FASTRTPSGEN_DIR environment variable

export FASTRTPSGEN_DIR=/usr/local/bin

these builds work

make px4_fmu-v3
make px4_fmu-v4
make px4_fmu-v5

these fail

make px4_fmu-v3_rtps
make px4_fmu-v4_rtps
make px4_fmu-v5_rtps

Although the v3_rtps does succeed when I'm running local on the VM after installing using the convenience scripts provided.

Expected behavior

1. Consistent builds (the docker image compile should compile the build versions as my local install -- specifically, the v3_rtps compiles on local, doesn't on docker container)
2. rtps feature should compile on all configurations.

Log Files and Screenshots
user@6c4a4043cbca:/src/Firmware$ make px4_fmu-v5_rtps
-- PX4 config file: /src/Firmware/boards/px4/fmu-v5/rtps.cmake
-- PX4 config: px4_fmu-v5_rtps
-- PX4 platform: nuttx
-- PX4 lockstep: disabled
-- PX4 version: v1.9.2
-- cmake build type: MinSizeRel
-- The CXX compiler identification is GNU 7.2.1
-- The C compiler identification is GNU 7.2.1
-- The ASM compiler identification is GNU
-- Found assembler: /usr/lib/ccache/arm-none-eabi-gcc
-- Check for working CXX compiler: /usr/lib/ccache/arm-none-eabi-g++
-- Check for working CXX compiler: /usr/lib/ccache/arm-none-eabi-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/lib/ccache/arm-none-eabi-gcc
-- Check for working C compiler: /usr/lib/ccache/arm-none-eabi-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found PythonInterp: /usr/bin/python (found version "2.7.15")
-- Found PY_jinja2: /usr/local/lib/python2.7/dist-packages/jinja2
-- PX4 ECL: Very lightweight Estimation & Control Library v1.9.0-rc1
-- Building and including px4_io-v2_default
-- Using C++11
-- Release build type: MinSizeRel
-- Adding UAVCAN STM32 platform driver
-- RTPS send: home_position;wind_estimate;radio_status;satellite_info;adc_report;iridiumsbd_status;vtol_vehicle_status;airspeed;sensor_selection;cpuload;sensor_baro;battery_status;estimator_status;distance_sensor;vehicle_odometry;vehicle_attitude;collision_constraints;sensor_combined
-- RTPS receive: trajectory_waypoint;position_setpoint;camera_capture;collision_report;obstacle_distance;debug_array;debug_key_value;debug_vect;position_setpoint_triplet;debug_value;camera_trigger;vehicle_trajectory_waypoint;optical_flow
-- ROMFS: px4fmu_common
-- ROMFS: Adding rc.board_defaults
-- ROMFS: Adding rc.board_sensors
-- Configuring done
-- Generating done
-- Build files have been written to: /src/Firmware/build/px4_fmu-v5_rtps
[0/1237] git submodule platforms/nuttx/NuttX/nuttx
[2/1237] git submodule platforms/nuttx/NuttX/apps
[5/1237] git submodule src/lib/ecl
[7/1237] git submodule src/drivers/gps/devices
[13/1237] git submodule src/drivers/uavcan/libuavcan
[14/1237] git submodule mavlink/include/mavlink/v2.0
[15/1237] git submodule src/modules/micrortps_bridge/micro-CDR
[27/1237] Generating RTPS topic bridge
FAILED: src/modules/micrortps_bridge/micrortps_client/micrortps_agent/home_position_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/home_position_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/wind_estimate_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/wind_estimate_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/radio_status_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/radio_status_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/satellite_info_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/satellite_info_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/adc_report_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/adc_report_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/iridiumsbd_status_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/iridiumsbd_status_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/vtol_vehicle_status_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/vtol_vehicle_status_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/airspeed_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/airspeed_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/sensor_selection_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/sensor_selection_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/cpuload_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/cpuload_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/sensor_baro_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/sensor_baro_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/battery_status_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/battery_status_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/estimator_status_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/estimator_status_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/distance_sensor_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/distance_sensor_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/vehicle_odometry_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/vehicle_odometry_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/vehicle_attitude_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/vehicle_attitude_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/collision_constraints_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/collision_constraints_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/sensor_combined_Publisher.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/sensor_combined_Publisher.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/trajectory_waypoint_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/trajectory_waypoint_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/position_setpoint_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/position_setpoint_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/camera_capture_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/camera_capture_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/collision_report_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/collision_report_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/obstacle_distance_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/obstacle_distance_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/debug_array_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/debug_array_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/debug_key_value_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/debug_key_value_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/debug_vect_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/debug_vect_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/position_setpoint_triplet_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/position_setpoint_triplet_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/debug_value_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/debug_value_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/camera_trigger_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/camera_trigger_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/vehicle_trajectory_waypoint_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/vehicle_trajectory_waypoint_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_agent/optical_flow_Subscriber.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_agent/optical_flow_Subscriber.h src/modules/micrortps_bridge/micrortps_client/micrortps_client/microRTPS_client.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_client/microRTPS_transport.cpp src/modules/micrortps_bridge/micrortps_client/micrortps_client/microRTPS_transport.h
cd /src/Firmware/build/px4_fmu-v5_rtps/src/modules/micrortps_bridge/micrortps_client && /usr/bin/python /src/Firmware/msg/tools/generate_microRTPS_bridge.py --fastrtpsgen-dir --generate-idl --mkdir-build --generate-cmakelists --topic-msg-dir /src/Firmware/msg --uorb-templates-dir templates/uorb_microcdr --urtps-templates-dir templates/urtps --agent-outdir /src/Firmware/build/px4_fmu-v5_rtps/src/modules/micrortps_bridge/micrortps_client/micrortps_agent --client-outdir /src/Firmware/build/px4_fmu-v5_rtps/src/modules/micrortps_bridge/micrortps_client/micrortps_client --idl-dir /src/Firmware/build/px4_fmu-v5_rtps/src/modules/micrortps_bridge/micrortps_client/micrortps_agent/idl >micrortps_bridge.log > /dev/null
Traceback (most recent call last):
File "/src/Firmware/msg/tools/generate_microRTPS_bridge.py", line 401, in
generate_agent(agent_out_dir)
File "/src/Firmware/msg/tools/generate_microRTPS_bridge.py", line 334, in generate_agent
"fastrtpsgen not found. Specify the location of fastrtpsgen with the -f flag")
Exception: fastrtpsgen not found. Specify the location of fastrtpsgen with the -f flag
[29/1237] Performing configure step for 'px4io_firmware'
-- PX4 config file: /src/Firmware/boards/px4/io-v2/default.cmake
-- PX4 config: px4_io-v2_default
-- PX4 platform: nuttx
-- PX4 lockstep: disabled
-- PX4 version: v1.9.2
-- cmake build type: MinSizeRel
-- The CXX compiler identification is GNU 7.2.1
-- The C compiler identification is GNU 7.2.1
-- The ASM compiler identification is GNU
-- Found assembler: /usr/lib/ccache/arm-none-eabi-gcc
-- Check for working CXX compiler: /usr/lib/ccache/arm-none-eabi-g++
-- Check for working CXX compiler: /usr/lib/ccache/arm-none-eabi-g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /usr/lib/ccache/arm-none-eabi-gcc
-- Check for working C compiler: /usr/lib/ccache/arm-none-eabi-gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found PythonInterp: /usr/bin/python (found version "2.7.15")
-- Found PY_jinja2: /usr/local/lib/python2.7/dist-packages/jinja2
-- PX4 ECL: Very lightweight Estimation & Control Library v1.9.0-rc1
-- Configuring done
-- Generating done
-- Build files have been written to: /src/Firmware/build/px4_fmu-v5_rtps/external/Build/px4io_firmware
ninja: build stopped: subcommand failed.
Makefile:193: recipe for target 'px4_fmu-v5_rtps' failed
make: *** [px4_fmu-v5_rtps] Error 1

Drone (please complete the following information):
N/A

Additional context
I think this is about as plain-vanilla as you can get.
To get xhost to work, I needed to install x11-server-utils
sudo apt-get install x11-xserver-utils

[TSC21]

@tschiu have you tried using px4-dev-nuttx:2019-07-29 instead?

[tschiu]

No. Will do so right now

[tschiu]

#THANK YOU!!!!!

That works perfectly.

I've been pulling my hair out for a week trying to get this to compile.

v3, v4, v5 and v3_rtps, v4_rtps, v5_rtps all compile without errors.

Question: I gather pulling "latest" docker images doesn't guarantee success, and that, if I need to update the docker image, I should use the most recent version with a date tag?

[tschiu]

Using pxio/px4-dev-nuttx:2019-07-29 instead of pxio/px4-dev-nuttx:latest solved the issue.