rsync error when building the px4/firmware

[QingZe0101]

Bug Report

when I input the order "make px4fmu-v2_default" in PX4 Consoles on Windows, rsync error happens like:
"The source and destination cannot both be remote."
I tried the solution "cmake nuttx rsync copy only use relative paths', but I found that the code "px4_impl_nuttx.cmake " is different from the used one, so I am confused and don't know how to fix this problem.

[dagar]

Could you post the verbose build output?

[zjjyyang]

I got the same problem, the message is like:
make px4fmu-v2_default VERBOSE=1
"/C/Program Files/CMake/bin/cmake.exe" -H/D/px4/Fimrware20171112 -B/D/px4/Fimrware20171112/build/px4fmu-v2_default --check-build-system CMakeFiles/Makefile.cmake 0
"/C/Program Files/CMake/bin/cmake.exe" -E cmake_progress_start /D/px4/Fimrware20171112/build/px4fmu-v2_default/CMakeFiles /D/px4/Fimrware20171112/build/px4fmu-v2_default/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make -f NuttX/CMakeFiles/nuttx_configure.dir/build.make NuttX/CMakeFiles/nuttx_configure.dir/depend
make -f msg/CMakeFiles/uorb_headers.dir/build.make msg/CMakeFiles/uorb_headers.dir/depend
make -f src/lib/version/CMakeFiles/ver_gen.dir/build.make src/lib/version/CMakeFiles/ver_gen.dir/depend
"/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/msg /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default/msg /D/px4/Fimrware20171112/build/px4fmu-v2_default/msg/CMakeFiles/uorb_headers.dir/DependInfo.cmake --color=
make -f CMakeFiles/git_ecl.dir/build.make CMakeFiles/git_ecl.dir/depend
"/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/src/lib/version /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default/src/lib/version /D/px4/Fimrware20171112/build/px4fmu-v2_default/src/lib/version/CMakeFiles/ver_gen.dir/DependInfo.cmake --color=
"/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default/CMakeFiles/git_ecl.dir/DependInfo.cmake --color=
make -f msg/CMakeFiles/uorb_headers.dir/build.make msg/CMakeFiles/uorb_headers.dir/build
make -f src/lib/version/CMakeFiles/ver_gen.dir/build.make src/lib/version/CMakeFiles/ver_gen.dir/build
make -f CMakeFiles/git_ecl.dir/build.make CMakeFiles/git_ecl.dir/build
make[3]: Nothing to be done for src/lib/version/CMakeFiles/ver_gen.dir/build'. make[3]: Nothing to be done for CMakeFiles/git_ecl.dir/build'.
make[3]: Nothing to be done for msg/CMakeFiles/uorb_headers.dir/build'. [ 1%] Built target ver_gen make -f src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/build.make src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/depend [ 2%] Built target git_ecl make -f CMakeFiles/git_nuttx.dir/build.make CMakeFiles/git_nuttx.dir/depend "/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/src/modules/systemlib/mixer /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default/src/modules/systemlib/mixer /D/px4/Fimrware20171112/build/px4fmu-v2_default/src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/DependInfo.cmake --color= [ 2%] Built target uorb_headers make -f src/drivers/px4io/CMakeFiles/copy_px4io_bin.dir/build.make src/drivers/px4io/CMakeFiles/copy_px4io_bin.dir/depend "/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default/CMakeFiles/git_nuttx.dir/DependInfo.cmake --color= "/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/src/drivers/px4io /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default/src/drivers/px4io /D/px4/Fimrware20171112/build/px4fmu-v2_default/src/drivers/px4io/CMakeFiles/copy_px4io_bin.dir/DependInfo.cmake --color= make -f src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/build.make src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/build "/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112 /platforms/nuttx/NuttX /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default/Nmake -f CMakeFiles/git_nuttx.dir/build.make CMakeFiles/git_nuttx.dir/buildx_configure.dir/DependInfo.cmake --color= make[3]: Nothing to be done for src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/build'.
make[3]: Nothing to be done for CMakeFiles/git_nuttx.dir/build'. make -f src/drivers/px4io/CMakeFiles/copy_px4io_bin.dir/build.make src/drivers/px4io/CMakeFiles/copy_px4io_bin.dir/build make -f NuttX/CMakeFiles/nuttx_configure.dir/build.make NuttX/CMakeFiles/nuttx_configure.dir/build [ 2%] Built target mixer_gen make -f src/modules/systemlib/param/CMakeFiles/parameters_xml.dir/build.make src/modules/systemlib/param/CMakeFiles/parameters_xml.dir/depend [ 2%] Built target git_nuttx "/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/src/modules/systemlib/param /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default/src/modules/systemlib/param /D/px4/Fimrware20171112/build/px4fmu-v2_default/src/modules/systemlib/param/CMakeFiles/parameters_xml.dir/DependInfo.cmake --color= make -f CMakeFiles/airframes_xml.dir/build.make CMakeFiles/airframes_xml.dir/depend [ 2%] Building px4io-v2 "/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default /D/px4/Fimrware20171112/build/px4fmu-v2_default/CMakeFiles/airframes_xml.dir/DependInfo.cmake --color= cd /D/px4/Fimrware20171112 && make --no-print-directory px4io-v2_default make -f src/modules/systemlib/param/CMakeFiles/parameters_xml.dir/build.make src/modules/systemlib/param/CMakeFiles/parameters_xml.dir/build make -f CMakeFiles/airframes_xml.dir/build.make CMakeFiles/airframes_xml.dir/build make[3]: Nothing to be done for CMakeFiles/airframes_xml.dir/build'.
[ 2%] Built target airframes_xml
make[3]: Nothing to be done for src/modules/systemlib/param/CMakeFiles/parameters_xml.dir/build'. [ 2%] Built target parameters_xml make[4]: warning: jobserver unavailable: using -j1. Add +' to parent make rule.
"/C/Program Files/CMake/bin/cmake.exe" -H/D/px4/Fimrware20171112 -B/D/px4/Fimrware20171112/build/px4io-v2_default --check-build-system CMakeFiles/Makefile.cmake 0
"/C/Program Files/CMake/bin/cmake.exe" -E cmake_progress_start /D/px4/Fimrware20171112/build/px4io-v2_default/CMakeFiles /D/px4/Fimrware20171112/build/px4io-v2_default/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make -f msg/CMakeFiles/uorb_headers.dir/build.make msg/CMakeFiles/uorb_headers.dir/depend
make -f NuttX/CMakeFiles/nuttx_configure.dir/build.make NuttX/CMakeFiles/nuttx_configure.dir/depend
"/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/msg /D/px4/Fimrware20171112/build/px4io-v2_default /D/px4/Fimrware20171112/build/px4io-v2_default/msg /D/px4/Fimrware20171112/build/px4io-v2_default/msg/CMakeFiles/uorb_headers.dir/DependInfo.cmake --color=
make -f CMakeFiles/git_nuttx.dir/build.make CMakeFiles/git_nuttx.dir/depend
make -f src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/build.make src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/depend
"/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/build/px4io-v2_default /D/px4/Fimrware20171112/build/px4io-v2_default /D/px4/Fimrware20171112/build/px4io-v2_default/CMakeFiles/git_nuttx.dir/DependInfo.cmake --color=
"/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/src/modules/systemlib/mixer /D/px4/Fimrware20171112/build/px4io-v2_default /D/px4/Fimrware20171112/build/px4io-v2_default/src/modules/systemlib/mixer /D/px4/Fimrware20171112/build/px4io-v2_default/src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/DependInfo.cmake --color=
make -f msg/CMakeFiles/uorb_headers.dir/build.make msg/CMakeFiles/uorb_headers.dir/build
make -f CMakeFiles/git_nuttx.dir/build.make CMakeFiles/git_nuttx.dir/build
make -f src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/build.make src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/build
make[7]: Nothing to be done for CMakeFiles/git_nuttx.dir/build'. make[7]: Nothing to be done for src/modules/systemlib/mixer/CMakeFiles/mixer_gen.dir/build'.
make[7]: Nothing to be done for msg/CMakeFiles/uorb_headers.dir/build'. [ 1%] Built target git_nuttx make -f NuttX/CMakeFiles/nuttx_builtins.dir/build.make NuttX/CMakeFiles/nuttx_builtins.dir/depend [ 2%] Built target mixer_gen [ 2%] Built target uorb_headers "/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/platforms/nuttx/NuttX /D/px4/Fimrware20171112/build/px4io-v2_default /D/px4/Fimrware20171112/build/px4io-v2_default/NuttX /D/px4/Fimrware20171112/build/px4io-v2_default/NuttX/CMakeFiles/nuttx_builtins.dir/DependInfo.cmake --color= make -f NuttX/CMakeFiles/nuttx_builtins.dir/build.make NuttX/CMakeFiles/nuttx_builtins.dir/build make[7]: Nothing to be done for NuttX/CMakeFiles/nuttx_builtins.dir/build'.
[ 2%] Built target nuttx_builtins
"/C/Program Files/CMake/bin/cmake.exe" -E cmake_depends "MSYS Makefiles" /D/px4/Fimrware20171112 /D/px4/Fimrware20171112/platforms/nuttx/NuttX /D/px4/Fimrware20171112/build/px4io-v2_default /D/px4/Fimrware20171112/build/px4io-v2_default/NuttX /D/px4/Fimrware20171112/build/px4io-v2_default/NuttX/CMakeFiles/nuttx_configure.dir/DependInfo.cmake --color=
make -f NuttX/CMakeFiles/nuttx_configure.dir/build.make NuttX/CMakeFiles/nuttx_configure.dir/build
[ 2%] Copying NuttX\apps to D;C:\px4\toolchain\msys\1.0\px4\Fimrware20171112\build\px4fmu-v2_default\NuttX
[ 3%] Copying NuttX\apps to D;C:\px4\toolchain\msys\1.0\px4\Fimrware20171112\build\px4io-v2_default\NuttX
cd /D/px4/Fimrware20171112/build/px4fmu-v2_default/NuttX && rsync -rp --inplace D:/px4/Fimrware20171112/platforms/nuttx/NuttX/apps D:/px4/Fimrware20171112/build/px4fmu-v2_default/NuttX
[ 2%] Copying NuttX\nuttx to D;C:\px4\toolchain\msys\1.0\px4\Fimrware20171112\build\px4fmu-v2_default\NuttX
cd /D/px4/Fimrware20171112/build/px4fmu-v2_default/NuttX && rsync -rp --inplace D:/px4/Fimrware20171112/platforms/nuttx/NuttX/nuttx D:/px4/Fimrware20171112/build/px4fmu-v2_default/NuttX
The source and destination cannot both be remote.
rsync error: syntax or usage error (code 1) at /usr/src/rsync/rsync-3.0.8/main.c(1148) [Receiver=3.0.8]
make[3]: *** [NuttX/apps_copy.stamp] Error 1
make[3]: *** Waiting for unfinished jobs....
The source and destination cannot both be remote.
rsync error: syntax or usage error (code 1) at /usr/src/rsync/rsync-3.0.8/main.c(1148) [Receiver=3.0.8]
make[3]: *** [NuttX/nuttx_copy.stamp] Error 1
make[2]: *** [NuttX/CMakeFiles/nuttx_configure.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
cd /D/px4/Fimrware20171112/build/px4io-v2_default/NuttX && rsync -rp --inplace D:/px4/Fimrware20171112/platforms/nuttx/NuttX/apps D:/px4/Fimrware20171112/build/px4io-v2_default/NuttX
[ 3%] Copying NuttX\nuttx to D;C:\px4\toolchain\msys\1.0\px4\Fimrware20171112\build\px4io-v2_default\NuttX
The source and destination cannot both be remote.
rsync error: syntax or usage error (code 1) at /usr/src/rsync/rsync-3.0.8/main.c(1148) [Receiver=3.0.8]
cd /D/px4/Fimrware20171112/build/px4io-v2_default/NuttX && rsync -rp --inplace D:/px4/Fimrware20171112/platforms/nuttx/NuttX/nuttx D:/px4/Fimrware20171112/build/px4io-v2_default/NuttX
make[7]: *** [NuttX/apps_copy.stamp] Error 1
make[7]: *** Waiting for unfinished jobs....
The source and destination cannot both be remote.
rsync error: syntax or usage error (code 1) at /usr/src/rsync/rsync-3.0.8/main.c(1148) [Receiver=3.0.8]
make[7]: *** [NuttX/nuttx_copy.stamp] Error 1
make[6]: *** [NuttX/CMakeFiles/nuttx_configure.dir/all] Error 2
make[5]: *** [all] Error 2
make[4]: *** [px4io-v2_default] Error 2
make[3]: *** [../px4io-v2_default/nuttx_px4io-v2_default.elf] Error 2
make[2]: *** [src/drivers/px4io/CMakeFiles/copy_px4io_bin.dir/all] Error 2
make[1]: *** [all] Error 2
make: *** [px4fmu-v2_default] Error 2

[zjjyyang]

It must be the same problem as issue #6332, #6332
But the file ' /cmake/nuttx/px4_impl_nuttx.cmake' has been changed, the original answer is not applied.

[dagar]

It looks like the issue is with the paths being used by rsync. They need to be cygwin paths, or updated to be relative to get around the problem.

[dagar]

Could you try this PR #8284?

[zjjyyang]

I have try the method proposed in #8284, and the problem is still here, the output path:
D;C:\px4\toolchain\msys\1.0\px4\Fimrware20171112\build\px4io-v2_default\NuttX

I think rsync cmd process the var ${CMAKE_CURRENT_SOURCE_DIR} and ${CMAKE_CURRENT_BINARY_DIR}, then make them changed.

I downloaded the source code of rsync, and have no idea currently.

[dagar]

@zjjyyang thanks for testing.

The relatively easy fix will be in PX4 cmake and not rsync itself. We just need to figure out how to get relative paths correctly.

With #8284 could you completely clean, build again, and share the verbose output? https://pastebin.com/

[zjjyyang]

The steps:
1. make clean
2. make distclean
3. update /platforms/nuttx/NuttX/CMakeLists.txt as:
...
\ # copy nuttx to build directory
file(RELATIVE_PATH CP_SRC ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/nuttx)
add_custom_command(OUTPUT nuttx_copy.stamp
# COMMAND ${cp_cmd} ${cp_opts} ${CMAKE_CURRENT_SOURCE_DIR}/nuttx ${CMAKE_CURRENT_BINARY_DIR}
COMMAND ${cp_cmd} ${cp_opts} ${CP_SRC} .
COMMAND cmake -E touch nuttx_copy.stamp
DEPENDS ${copy_nuttx_files}
COMMENT "Copying NuttX/nuttx to ${CMAKE_CURRENT_BINARY_DIR}"
)

set(NUTTX_DIR ${CMAKE_CURRENT_BINARY_DIR}/nuttx)
set(NUTTX_CONFIG_DIR ${PX4_SOURCE_DIR}/nuttx-configs)

# copy apps to build directory
file(RELATIVE_PATH CP_SRC ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/apps)
add_custom_command(OUTPUT apps_copy.stamp
# COMMAND ${cp_cmd} ${cp_opts} ${CMAKE_CURRENT_SOURCE_DIR}/apps ${CMAKE_CURRENT_BINARY_DIR}
COMMAND ${cp_cmd} ${cp_opts} ${CMAKE_CURRENT_SOURCE_DIR}/apps .
COMMAND cmake -E touch apps_copy.stamp
DEPENDS ${copy_apps_files}
COMMENT "Copying NuttX/apps to ${CMAKE_CURRENT_BINARY_DIR}"
)
...
4. make px4fmu-v2_default
5. the output is:
...
-- Configuring done
[ 2%] Built target git_ecl
Scanning dependencies of target mixer_gen
[ 2%] Generating mixer_multirotor.generated.h
[ 2%] Built target mixer_gen
Scanning dependencies of target airframes_xml
[ 2%] Creating airframes.xml
[ 2%] Built target airframes_xml
Scanning dependencies of target parameters_xml
[ 2%] Generating parameters.xml
[ 2%] Built target parameters_xml
[ 2%] Built target uorb_headers
-- Generating done
-- Build files have been written to: D:/px4/Fimrware20171112/build/px4io-v2_default
Scanning dependencies of target uorb_headers
Scanning dependencies of target mixer_gen
Scanning dependencies of target git_nuttx
[ 1%] Generating mixer_multirotor.generated.h
[ 1%] Generating uORB topic headers
[ 2%] git submodule platforms/nuttx/NuttX/nuttx
[ 2%] Built target mixer_gen
Scanning dependencies of target nuttx_configure
Scanning dependencies of target nuttx_builtins
[ 2%] Built target nuttx_builtins
[ 3%] Copying NuttX\apps to D;C:\px4\toolchain\msys\1.0\px4\Fimrware20171112\build\px4io-v2_default\NuttX
[ 2%] Copying NuttX\nuttx to D;C:\px4\toolchain\msys\1.0\px4\Fimrware20171112\build\px4fmu-v2_default\NuttX
[ 3%] Built target git_nuttx
ssh: Could not resolve hostname d: Name or service not known
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.2]
make[3]: *** [NuttX/apps_copy.stamp] Error 12
make[3]: *** Waiting for unfinished jobs....
[ 3%] Built target uorb_headers
[ 3%] Copying NuttX\nuttx to D;C:\px4\toolchain\msys\1.0\px4\Fimrware20171112\build\px4io-v2_default\NuttX
make[2]: *** [NuttX/CMakeFiles/nuttx_configure.dir/all] Error 2
make[2]: *** Waiting for unfinished jobs....
ssh: Could not resolve hostname d: Name or service not known
rsync: connection unexpectedly closed (0 bytes received so far) [Receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(226) [Receiver=3.1.2]
make[7]: *** [NuttX/apps_copy.stamp] Error 12
make[7]: *** Waiting for unfinished jobs....

the output:
[ 3%] Copying NuttX\apps to D;C:\px4\toolchain\msys\1.0\px4\Fimrware20171112\build\px4io-v2_default\NuttX
[ 2%] Copying NuttX\nuttx to D;C:\px4\toolchain\msys\1.0\px4\Fimrware20171112\build\px4fmu-v2_default\NuttX

should be from the "COMMENT "Copying NuttX/nuttx to ${CMAKE_CURRENT_BINARY_DIR}""
and "COMMENT "Copying NuttX/apps to ${CMAKE_CURRENT_BINARY_DIR}""

which means rsync can modify the input path?

[dagar]

@zjjyyang

Because NuttX doesn't have an out of source build we copy the entire thing to each build directory. Due to rsync on windows and path issues we need to be careful and only pass relative paths to the rsync command.

It should ultimately be something like...

rsync ../../NuttX/nuttx .

Are you available on slack (http://slack.px4.io)? We might be able to debug and fix this quickly if we can chat in realtime.

[MaEtUgR]

@QingZe0101 As you are working in some kind of Windws based environment I would be really interested in your experience with that and share with you what I tried in the past already (I had NuttX working well but SITL not working). I have the same name on slack for more realtime communication.

[dagar]

@zjjyyang could you please try this PR? #8397