Telcon: 2016 03 24

Participants

1. Todd Gamblin (LLNL)
2. Benedikt Hegner (CERN)
3. Ben Boeckel (Kitware)
4. D'Jay (Kitware)
5. James Amundsen (Fermi)
6. Matt Legendre (LLNL)
7. Mike Collette (LLNL)
8. Patrick Gartung (Fermilab)
9. Peter Scheibel (LLNL)
10. Greg Lee (LLNL)
11. Mario Melara (NERSC)
12. Gregory Becker (LLNL)

News

1. Module customization is in Spack thanks to Massimiliano Culpo (EPFL)

• Packages can define:
  1. setup_environment: set up this package's compile and run environments
  2. setup_dependent_environment: set up part of compile and run environments for dependencies
  3. setup_dependent_package: set up module-scope things for dependent packages.
  • may be wise to deprecate some of this and put it in a namespace, but it's here for now.
• New EnvironmentModifications object allows:
  • set, unset, set_path, prepend_path, append_path, etc.
  • changes are recorded and used both for Spack's environment and for adding to modules.
    • packager does not have to write the same code twice for build and for module.
• New modules.yaml config
  • currently allows things like disabling dotkit generation
  • will allow site-specific customization, overrides, etc.
  • Looking into naming customization, so that things like PR #498 can be set up with an expression in the config file

1. Welcome to the folks from deal.ii, who are starting to use Spack

• See interesting discussion of package managers here
• Lots of mac contributions, hardening of numerical lib builds.

1. Peter Scheibel is working on better source archive caching

• Implements a mirror in var/cache in the spack install
• allows packages to be cached locally, built again w/o downloading
• Still needs to cover resources.

1. Status on RPM generation (Peter Scheibel)

• Lots of work on supporting DESTDIR recently.
• About to start work on an RPM naming scheme
  • scheme will allow packagers to specify arbitrary degrees of combinatorial-ness for installed RPMs.
  • from no combinations -> spack hashes.

1. Binary install caching (Benedikt Hegner)

• Going well, working except for platform support.
• Needs platform support from Greg Becker to move forward.

1. Status of OS + Cray support (Greg Becker)

• Merged, currently resolving conflicts and broken tests.

1. Status of build/link/run dependencies (Ben Boeckel)

• Nearly ready to merge, caught more bugs.
• This plus OS support will allow better concretization of build deps and compilers on cross-compiled machines like Titan.

1. lapack/blas rework #485

Discussion

1. require directive needed by ROSE team, e.g. require('%gcc@4.9.3', when='=chaos_5_x86_64_ib')
2. How to mix fortran + clang compilers
3. How to build gcc and gfortran on Mac OS X

• packages need some hardening
• no luck building gcc on macs so far
• LLNL working on getting CI set up to include macs
  • Jim: bad experience with Mac Minis
  • Ben, D'Jay: Kitware has good experience with Mac minis... go figure

1. Benedikt:

• High Energy Physics community is evaluating Spack to replace existing solutions at CERN
• planning a hackathon 1st week of May to see how far they can get
• Benedikt has drafted an email with a list of features that would be useful
  • Will send to the mailing list to see if anyone else is working on these features, and to plan the sprint at CERN.
    1. binary packaging
    2. how to do relocation of binaries for binary packaging.
    • don't want to depend on absolute RPATH
    • CERN does "environment chaining"
      • uses RUNPATH + environment
      • when RUNPATH doesn't work, fall back on the environment
        • LD_LIBRARY_PATH overrides RUNPATH
    • LLNL typically falls back on RPATH
      • this is why Spack currently uses RPATH
      • RUNPATH/RPATH differences are subtle, search order is different:
        • RUNPATH only searches RPATH of loading object
          • doesn't look in parent objects' RPATHs
        • RPATH searches loading object, then its parent, then its parent, and so on.
          • falls back to LD_LIBRARY_PATH only if no objects in the build have the path
        • Interesting description of differences on the Qt blog
        • Need to think about what this means for Spack.
    1. best effort builds of packages (bottom-up DAG, parallelize)
    • Needed for integration builds, where they want things to fail as last
    • Spack needs finer-grained locking on packages.
  • other items TBD in email.
    • please send Benedikt feedback on email:
      • what's being done, what's not