EGLIBC.cross-testing

                                                        -*- mode: text -*-

                      Cross-Testing With EGLIBC
                  Jim Blandy <jimb@codesourcery.com>


Introduction

Developers writing software for embedded systems often use a desktop
or other similarly capable computer for development, but need to run
tests on the embedded system, or perhaps on a simulator.  When
configured for cross-compilation, the stock GNU C library simply
disables running tests altogether: the command 'make tests' builds
test programs, but does not run them.  EGLIBC, however, provides
facilities for compiling tests and generating data files on the build
system, but running the test programs themselves on a remote system or
simulator.


Test environment requirements

The test environment must meet certain conditions for EGLIBC's
cross-testing facilities to work:

- Shared filesystems.  The 'build' system, on which you configure and
  compile EGLIBC, and the 'host' system, on which you intend to run
  EGLIBC, must share a filesystem containing the EGLIBC build and
  source trees.  Files must appear at the same paths on both systems.

- Remote-shell like invocation.  There must be a way to run a program
  on the host system from the build system, passing it properly quoted
  command-line arguments, setting environment variables, and
  inheriting the caller's standard input and output.


Usage

To use EGLIBC's cross-testing support, provide values for the
following Make variables when you invoke 'make':

- cross-test-wrapper

  This should be the name of the cross-testing wrapper command, along
  with any arguments.

- cross-localedef

  This should be the name of a cross-capable localedef program, like
  that included in the EGLIBC 'localedef' module, along with any
  arguments needed.

These are each explained in detail below.


The Cross-Testing Wrapper

To run test programs reliably, the stock GNU C library takes care to
ensure that test programs use the newly compiled dynamic linker and
shared libraries, and never the host system's installed libraries.  To
accomplish this, it runs the tests by explicitly invoking the dynamic
linker from the build tree, passing it a list of build tree
directories to search for shared libraries, followed by the name of
the executable to run and its arguments.

For example, where one might normally run a test program like this:

    $ ./tst-foo arg1 arg2

the GNU C library might run that program like this:

    $ $objdir/elf/ld-linux.so.3 --library-path $objdir \
      ./tst-foo arg1 arg2

(where $objdir is the path to the top of the build tree, and the
trailing backslash indicates a continuation of the command).  In other
words, each test program invocation is 'wrapped up' inside an explicit
invocation of the dynamic linker, which must itself execute the test
program, having loaded shared libraries from the appropriate
directories.

To support cross-testing, EGLIBC allows the developer to optionally
set the 'cross-test-wrapper' Make variable to another wrapper command,
to which it passes the entire dynamic linker invocation shown above as
arguments.  For example, if the developer supplies a wrapper of
'my-wrapper hostname', then EGLIBC would run the test above as
follows:

    $ my-wrapper hostname \
      $objdir/elf/ld-linux.so.3 --library-path $objdir \
      ./tst-foo arg1 arg2

The 'my-wrapper' command is responsible for executing the command
given on the host system.

Since tests are run in varying directories, the wrapper should either
be in your command search path, or 'cross-test-wrapper' should give an
absolute path for the wrapper.

The wrapper must meet several requirements:

- It must preserve the current directory.  As explained above, the
  build directory tree must be visible on both the build and host
  systems, at the same path.  The test wrapper must ensure that the
  current directory it inherits is also inherited by the dynamic
  linker (and thus the test program itself).

- It must preserve environment variables' values.  Many EGLIBC tests
  set environment variables for test runs; in native testing, it
  invokes programs like this:

    $ GCONV_PATH=$objdir/iconvdata \
      $objdir/elf/ld-linux.so.3 --library-path $objdir \
      ./tst-foo arg1 arg2

  With the cross-testing wrapper, that invocation becomes:

    $ GCONV_PATH=$objdir/iconvdata \
      my-wrapper hostname \
      $objdir/elf/ld-linux.so.3 --library-path $objdir \
      ./tst-foo arg1 arg2

  Here, 'my-wrapper' must ensure that the value it sees for
  'GCONV_PATH' will be seen by the dynamic linker, and thus 'tst-foo'
  itself.  (The wrapper supplied with GLIBC simply preserves the
  values of *all* enviroment variables, with a fixed set of
  exceptions.)

  If your wrapper is a shell script, take care to correctly propagate
  environment variables whose values contain spaces and shell
  metacharacters.

- It must pass the command's arguments, unmodified.  The arguments
  seen by the test program should be exactly those seen by the wrapper
  (after whatever arguments are given to the wrapper itself).  The
  EGLIBC test framework performs all needed shell word splitting and
  expansion (wildcard expansion, parameter substitution, and so on)
  before invoking the wrapper; further expansion may break the tests.


The 'cross-test-ssh.sh' script

If you want to use 'ssh' (or something sufficiently similar) to run
test programs on your host system, EGLIBC includes a shell script,
'scripts/cross-test-ssh.sh', which you can use as your wrapper
command.  This script takes care of setting the test command's current
directory, propagating environment variable values, and carrying
command-line arguments, all across an 'ssh' connection.  You may even
supply an alternative to 'ssh' on the command line, if needed.

For more details, pass 'cross-test-ssh.sh' the '--help' option.


The Cross-Compiling Locale Definition Command

Some EGLIBC tests rely on locales generated especially for the test
process.  In a native configuration, these tests simply run the
'localedef' command built by the normal EGLIBC build process,
'locale/localedef', to process and install their locales.  However, in
a cross-compiling configuration, this 'localedef' is built for the
host system, not the build system, and since it requires quite a bit
of memory to run (we have seen it fail on systems with 64MiB of
memory), it may not be practical to run it on the host system.

If set, EGLIBC uses the 'cross-localedef' Make variable as the command
to run on the build system to process and install locales.  The
localedef program built from the EGLIBC 'localedef' module is
suitable.

The value of 'cross-localedef' may also include command-line arguments
to be passed to the program; if you are using EGLIBC's 'localedef',
you may include endianness and 'uint32_t' alignment arguments here.


Example

In developing EGLIBC's cross-testing facility, we invoked 'make' with
the following script:

    #!/bin/sh

    srcdir=...
    test_hostname=...
    localedefdir=...
    cross_gxx=...-g++

    wrapper="$srcdir/scripts/cross-test-ssh.sh $test_hostname"
    localedef="$localedefdir/localedef --little-endian --uint32-align=4"

    make cross-test-wrapper="$wrapper" \
         cross-localedef="$localedef" \
         CXX="$cross_gxx" \
         "$@"


Other Cross-Testing Concerns

Here are notes on some other issues which you may encounter in running
the EGLIBC tests in a cross-compiling environment:

- Some tests require a C++ cross-compiler; you should set the 'CXX'
  Make variable to the name of an appropriate cross-compiler.

- Some tests require access to libstdc++.so.6 and libgcc_s.so.1; we
  simply place copies of these libraries in the top EGLIBC build
  directory.