Telcon: 2015 10 15

Participants

• Todd Gamblin (LLNL)
• Greg Lee (LLNL)
• Peter Scheibel (LLNL)
• Jim Galarowicz (Krell)
• Matt Legendre (LLNL)
• Mike Collette (LLNL)

News

1. Todd and Greg Becker went to NERSC/LBL on Tuesday to given an overview of Spack to NERSC USG, and to outline Cray support plans.

• Mario Melara at NERSC is working 100% time on the Spack Cray port, will be working with Greg Becker.
• Joaquin Correa, Rebecca Hartman-Baker, Karen Ng were also there.
• Mario and Greg Becker worked at LLNL Wed. to get some of this done.

2. Cray plan (getting done now):

3. Spack architecture will be a class, not a Python string. * Will be a class, but class will provide string identifiers for build targets on that type of machine.

   • Most arch identifiers will be based on uname output.
   • On special platforms like BG/Q and Cray, we'll define our own (e.g. cray-xc40, bgq, etc.)
   • Architecture class will describe:
     • How to detect if I am on this type of architecture
     • What build targets are available on this machine?
       • e.g., on an XC40, there would be build targets for the front-end, back-end haswell, and back-end MIC.

4. Architecture will specify how to find compilers for each target. * Will have multiple strategy classes for finding compilers

   • Can find from the PATH as Spack does now.
   • Can find from modules, for Cray PrgEnv and compiler support.
   • May add more (e.g., known locations of backend compilers in /opt/ibmcmp on BG/Q) * These "finding" strategies could be extended to find external packages like MPI.
   • Would work with Matt Legendre's external packages support
   • Would eventually provide a way for Spack to discover some of its other build dependencies and minimize builds.
     • e.g., finding git, cmake, svn, etc. and building if not found.

5. Definition of a "compiler" will expand.

   • No longer just a set of 4 paths to CC, CXX, F77, FC
   • Also includes modules to load before entering install at build time.
     • we'll add a module attribute in the compilers.yaml file.
     • may also take this opportunity to add lines for finding sourceme scripts like icc uses, for when modules aren't used.
     • both of these are just mechanisms for bundling some env settings with a Spack compiler.

6. spack.build_environment.fork() will change:

   • Will now spawn a sub-shell first.
   • sub-shell loads required modules for each compiler.
   • sub-shell launches another spack to do the real install.
     • "sub-spack" takes a spec file as YAML and does not concretize or re-init the build environment -- goes straight to install
     • means we'll add a new hidden command, spack yaml-install, to be called by builds
   • Note that adding this support will make it easy to implement parallel builds in SLURM.

7. Joaquin Correa also mentioned that Cray support in craype 2.0.0 or later could potentially be done via pkg-config and not by modules.

• This is an interesting option, but we went with modules because:
  • Older PE's didn't use pkg-config
  • Not clear whether Cray would change the underlying PrgEnv mechanism again
  • Interested in following up with Cray on the pkg-config support
  • With pkg-config could potentially make Spack's cc just emulate cc/CC/ftn
• pkg-config may be a great way to detect external dependencies on a Cray.
  • should look into this after that feature is merged (see PR 120)

3. Spack testing update

• Peter Scheibel has created spack testinstall
  • This does a build and generates JUnit XML output
  • Can be read by Bamboo.
  • Creates a test case for each dependency of a package on install.

4. Kitware update.

• Ben Boeckel has gotten familiar with the Spack code, has a basic ParaView build working.
• Submitted a few PRs:
  • use setdefault()
  • updated netcdf)
  • multiple spack roots

Discussion

• TBD