Fat archives not relocatable

[dhood]

The linux and osx packaging jobs are not currently generating re-locatable prefix.sh scripts.

Output on OS X:

$ source prefix.sh
not found: "/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install/share/ament_cppcheck/package.sh"
not found: "/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install/share/ament_flake8/package.sh"
not found: "/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install/share/ament_pep8/package.sh"
not found: "/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install/share/console_bridge/package.sh"
not found: "/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install/share/fastcdr/package.sh"
not found: "/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install/share/gtest_vendor/package.sh"
...

Contents:

# generated from colcon_core/shell/template/prefix.sh.em

# function to source another script with conditional trace output
# first argument: the name of the script file
colcon_prefix_source_shell_script() {
  # arguments
  _colcon_prefix_source_shell_script="$1"

  # source script with conditional trace output
  if [ -f "$_colcon_prefix_source_shell_script" ]; then
    if [ -n "$COLCON_TRACE" ]; then
      echo ". \"$_colcon_prefix_source_shell_script\""
    fi
    . "$_colcon_prefix_source_shell_script"
  else
    echo "not found: \"$_colcon_prefix_source_shell_script\"" 1>&2
  fi

  unset _colcon_prefix_source_shell_script
}


# since a plain shell script can't determine its own path when being sourced
# it uses the destination set at build time
COLCON_CURRENT_PREFIX=/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install
colcon_prefix_source_shell_script "$COLCON_CURRENT_PREFIX/share/ament_cppcheck/package.sh"

COLCON_CURRENT_PREFIX=/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install
colcon_prefix_source_shell_script "$COLCON_CURRENT_PREFIX/share/ament_flake8/package.sh"

COLCON_CURRENT_PREFIX=/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install
colcon_prefix_source_shell_script "$COLCON_CURRENT_PREFIX/share/ament_pep8/package.sh"

COLCON_CURRENT_PREFIX=/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install
colcon_prefix_source_shell_script "$COLCON_CURRENT_PREFIX/share/console_bridge/package.sh"

COLCON_CURRENT_PREFIX=/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install
colcon_prefix_source_shell_script "$COLCON_CURRENT_PREFIX/share/fastcdr/package.sh"

COLCON_CURRENT_PREFIX=/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install
colcon_prefix_source_shell_script "$COLCON_CURRENT_PREFIX/share/gtest_vendor/package.sh"

COLCON_CURRENT_PREFIX=/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install
colcon_prefix_source_shell_script "$COLCON_CURRENT_PREFIX/share/orocos_kdl/package.sh"

COLCON_CURRENT_PREFIX=/Users/osrf/jenkins-agent/workspace/packaging_osx/ws/install
colcon_prefix_source_shell_script "$COLCON_CURRENT_PREFIX/share/osrf_pycommon/package.sh"

...

unset COLCON_CURRENT_PREFIX
unset colcon_prefix_source_shell_script

Contents for linux is similar but uses COLCON_CURRENT_PREFIX=/home/rosbuild/ci_scripts/ws/install

[dhood]

(I labelled this as a bug since users wanting to use archives can't currently do so; there might be technical limitations why this isn't possible, or there might be work might be underway to address this already, but to the user wanting to use nightly packaging jobs this is a bug IMO)

[dirk-thomas]

A plain sh script has no way of determining the path of "itself". Therefore .sh files can't be relocatable. That has nothing to do with the build tool and is not "fixable".

If a user wants to source the .sh file rather .bash the file either needs to be edited or the environment variable needs to be set manually beforehand.

[dhood]

should there be a .bash file included in the archives? there is not

[dirk-thomas]

Well, there should be... I didn't install these non-primary shell packages since they are not necessary to run CI. But for the fat archives they should certainly be present to generate these artifacts. See #158.

[dhood]

I noticed that RobotWebTools/rclnodejs#320 referenced this issue.
Other projects are still expecting a local_setup.bash to be in the archives. Has there been any work done to let users know about this change? (I know most of the user-facing colcon switch is targeted for bouncy, but this impacts users using archives today)

[dirk-thomas]

The wiki changes are the next item on the meta ticket.

[dirk-thomas]

colcon/colcon-core#38 is adding an error message if a .sh script is relying on the build time prefix but it doesn't exist.

[minggangw]

I downloaded the latest package (#1067) for linux from https://ci.ros2.org/view/packaging/job/packaging_linux/lastSuccessfulBuild/ and executed source local_setup.bash to compile some code, but still met the path problem:

CMake Error in CMakeLists.txt:
  Imported target "rmw_fastrtps_cpp::rmw_fastrtps_cpp" includes non-existent
  path

    "/home/rosbuild/ci_scripts/ws/install/include"

  in its INTERFACE_INCLUDE_DIRECTORIES.  Possible reasons include:

  * The path was deleted, renamed, or moved to another location.

  * An install or uninstall procedure did not complete successfully.

  * The installation package was faulty and references files it does not
  provide.

[mikaelarguedas]

@minggangw Thanks for reporting this.

It seems to be a regression introduced in ros2/rmw_fastrtps#198, it's been reverted in ros2/rmw_fastrtps#200 pending a better solution.

Next nightlies should be relocatable. In the meantime you can use the artifact produced in https://ci.ros2.org/job/ci_packaging_linux/93/ for Linux testing.