Telcon: 2015 12 17

Participants

1. Todd Gamblin
2. Mark Miller
3. Mike Collette
4. François Bissey
5. Peter Scheibel
6. Jim Galarowicz
7. David Shrader
8. Greg Becker
9. Ray Loy
10. Matt Legendre
11. Ben Boeckel
12. Adam Moody
13. Mario Melara
14. Greg Lee

News & Contributions

1. Lots of new contributors after SC (thanks everyone!)

• many PRs from new users (Erik Schnetter submitted many PRs right after the talk!)
• Need to merge his fix for lib order in cc wrappers (195)
  • Need to test and merge it with a similar internal fix we have

1. New users on the mailing list (from EPFL, ANL, many others)
2. We moved to github.com/llnl/spack

• update your remotes!

1. Greg Becker & Mario Melara report that Cray integration is working well on NERSC machines.

• NERSC is making packages for Cori now.
  • this is going well, need module-based externals
• Will come with module-based compiler detection & support.
• Mario had a "spackdown" at NERSC
  • Gave Spack tutorial to NERSC users
  • notified people of the google group and telcon

To be merged

1. PR 210: multiple repositories is going to be merged soon.

• restructures /var/spack/packages to /var/spack/repos/builtin
• allows for separate repository directories & package namespaces
• can directly import one package from another via namespaces (e.g., import spack.pkg.builtin.mpich)

2. PR 120: External dependencies

• Depend on system MPI, other packages.

2. Many updates and conversations from the folks at EPFL

• PR 208: multiple tarballs per package
  • Looks good -- needs testing and will be merged.
• Many package updates (mvapich2, fftw, gnuplot, others...)

3. Spack test XML output with nose 202

• Spack test-install command already merged

4. Externals now added directly to sys.path (Peter Scheibel)

• needed for nose
• no more from external import argparse

Already merged

1. PR 86: Open|SpeedShop and associated packages
2. Qt and paraview updates from Ben Boeckel (164, 165, 166, 168)

• Build Paraview on catalyst
• Working on vulcan with xlc
  • Ran into issues with libtool + xl (see below)
• Working on OS X version

1. Scogland submitted shell arg handling fix for zsh (205)

• I need to test with bash -- if someone could do that and sign off before I get to it, it would be nice.

1. Greg Lee added -D flag: run Spack under pdb

Bug fixes

1. Package cache hashing
2. Issue 206, 238, 217: issues with uninstall failing.

• Most of these were related to the way specs are read in from the database.

1. Issue 209: Fixed bug in some virtual dependency handling

• satisfies() was not working right with multiple vdeps.

1. Issue 203: Figured out some issues with gcc builds

• 5.3.0 should now build properly
• Issues were with RPATHs passed to the internal bootstrap in gcc

Other Bugs

• Issues with libtool using a compiler's name to deduce flags (François Bissey)
  • Similar to prior distutils issues (which we patched)
  • Todd working on a fix for this -- see proposed solution in this thread
• Issue 244
  • Remaining issue relates to handling packages that conditionally extend others.

Work in Progress (WIP)

• Greg and Mario to work on:
  • Using modules as external packages (on cray)
    • auto-load modules to load system MPI
  • Adding sourceme files to compilers (e.g. load Intel's compilervars.sh automatically)
• Custom compiler flags (Greg Becker authored, Todd thinking about merging)
• Better PGI & XL compiler support in Boost (Peter Scheibel)
• Paraview (Kitware, Ben Boeckel)

Discussion

• Sorry if I failed to mention anything above -- community continues to grow!
• People want us to test on Macs -- I'd like to
  • Seeing about getting some Mac build agents at LLNL -- likely not happening short-term
  • Mark M.: AWS may offer mac build bots
  • Todd: also want Ubuntu build bots; could do those over AWS
• Discuss improving package debug-ability
  • Matt: spack packages can be difficult to debug
    • source code is gone after build, so DWARF points at source that doesn't exist
    • Things Matt is working on:
      1. Make more packages support +debug option.
      • support in a common way for autotools, cmake
      • allow override for arbitrary Makefile builds.
      • would enable debug flags per package
      • Matt: Need to make some default policy decisions about what comes with +debug
        • allow configuration w/reasonable defaults
      2. Figure out how best to manage source location & storage

1. Supporting +debug

   • Mark: is this just about flags or also other package options?
     • Matt: other options, too
       • many packages have extra features they turn on with, e.g., -DDEBUG
       • want to allow package author to support these.
   • Matt: if we support Windows, +debug flag needs to propagate to all dependencies there.
   • Matt:
     • To do standard stuff, we need to know whether a package is using autotools or cmake.
     • Options for how to implement this:
       1. Extend a subclass of Package, e.g. AutotoolsPackage
       • weakness: some packages have 2 builds. need this to be conditional like everything else
       2. In-package directive, with when clauses like other directives
       • could function like a mix-in in Ruby (kind of like other directives)
   • Mark: Make sure that +opt and +debug are separate.
     • Matt: that is the intent, just need to decide default for each.

2. Managing source for debugging.

• Want to automatically save the source somehow after build
  • Don't want to leave the object files around
  • need to build with source in a way so that DWARF paths are valid after install
    • Can build out of $prefix/.spack but it's not fast like /tmp
    • Ben: strip can change the paths used in debug information
      • may need to build with artificially long paths so you can trim later.
• Proposal for how to do this right:
  1. Create longer stage directory path.
  2. Build & install out of stage.
  3. Scan DWARF of binaries in prefix for source paths.
  4. Copy all Referenced source into $prefix/.spack/src
  5. Use strip to rename paths in the installed binaries