Skip to content

Conference call notes 20160111

Jump to bottom
Kenneth Hoste edited this page Jan 11, 2016 · 9 revisions

(back to Conference calls)

Notes on the EasyBuild conference call dedicated to discussing implementation of RPATH support, Monday January 11th 2016 (5pm CET)

Attendees

Alphabetical list of attendees (10):

  • Xavier Besseron (Uni.lu, Luxembourg)
  • Timothy Brown (Uni. of Colorado @ Boulder, US)
  • Pablo Escobar (UniBas, Switzerland)
  • Elisabeth Fisher (NASA, US)
  • Todd Gamblin (LLNL, US)
  • Fotis Georgatos (freelancer)
  • Kenneth Hoste (HPC-UGent, Belgium)
  • Lumír Jasiok (IIT4Innovations, Czech Republic)
  • Alan O'Cais (JSC, Germany)
  • Ward Poelmans (HPC-UGent, Belgium)

Agenda

  • implementation of support for RPATH linking in EasyBuild
    • different available options?
    • overall approach for implementing option of choice
    • who does what?

Notes

Options

  • compiler wrapper script (cfr. Spack)

  • ld wrapper script (cfr. Spack, script from Swedish colleagues)

    • should be sufficient (?)

  • patchelf

    • doesn't work on OS X
    • 'dummy' values need to be included that are long enough to hold actual path

  • $ORIGIN

    • can be used to support relative RPATH'ing
    • allows relocating installations later
    • advantage of Conda over Nix
    • not used in Spack today

  • sanity check?

    • match ldd & readelf
    • Todd has something that works on Linux

  • RPATH support means sort of re-architecting EB design?

    • RPATH support is step towards dropping modules as part of build process?
    • modules are still useful for users

  • $LD_LIBRARY_PATH may still be needed

    • e.g. for compilers (depends on how they were built)
    • Intel compilers are patched at LLNL to use RPATH

  • use of RPATH will fix problems caused by loading modules

    • loading CMake (which loads ncurses) breaks htop on the system
    • also allows using tools built with different toolchains/versions
      • linking to two different versions of a library in the same binary may not work though...
    • comes close to 'toolchain-neutral' software (cfr. https://github.com/hpcugent/easybuild-framework/issues/570)
    • can also helps with keeping size of environment under controle
      • important for MPI, job launchers, performance (library lookup)

  • RPATH vs RUNPATH

    • if both are set, RUNPATH wins
      • if you do RUNPATH, ld will actually do both (by default)
    • $LD_LIBRARY_PATH wins over RUNPATH, so to much potential issues
    • supporting both should be fairly trivial?

  • default on OS X is to use full paths to libraries

  • RPATH & CMake (Todd)

    • cmake by default will re-write RPATHs on install.
    • so if you use an ld wrapper it will not cover CMake in the standard case
    • (ironic b/c cmake has really good rpath support, but it requires a special case if YOU want to implement good RPATH support)
    • so you have to set CMAKE_INSTALL_RPATH on your cmake builds to tell it to write the right thing:
    • https://github.com/LLNL/spack/blob/develop/lib/spack/spack/build_environment.py#L215
    • the reason it does that is b/c it sets the RPATH at build time to the build directory (nice for developers)
    • but it rewrites on install so that the install works
Implementation
  • specify in easyconfigs which libraries to RPATH against it?
  • --rpath-deps?
  • also support to specify "RPATH all the things"
  • concern of reproducibility of build where RPATH was used
  • which deps were RPATH'ed should be dumped to the easyconfig copied in the install dir/archive
Inspiration
Who?
  • Rob
  • Ward (help with wrapper script)
Follow-up
Clone this wiki locally