Conference call notes 20211124

(back to Conference calls)

Notes on the 186th EasyBuild conference call, Wednesday Nov 24th 2021 (09:00 UTC)

Attendees

Alphabetical list of attendees (8):

• Alex Domingo (Vrije Universiteit Brussel, Belgium)
• Kenneth Hoste (HPC-UGent, Belgium)
• Adam Huffman (Big Data Institute, Oxford, UK)
• Robert Mijakovic (LuxProvide)
• Sam Moors (Vrije Universiteit Brussel, Belgium)
• Jurij Pečar (EMBL, Germany)
• Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)
• Victor Holanda (CSCS, Switzerland)

Agenda

• overview of recent developments
• Q&A

Recent developments

• release timeline
  • latest release: EasyBuild v4.5.0 (Oct 29th 2021)
  • ETA next release: end of Nov'21?
• recent changes
  • framework
    • bug fixes
      • only remove lock if it was created in the same EasyBuild session (not if it existed already) (PR #3889)
      • don't try to ensure absolute path for path part of repositorypath (PR #3889)
        • required to fully fix regression w.r.t. using remote Git repository as easyconfigs archive (issue #3892)
    • enhancements
      • determine which extensions can be skipped in parallel (if --parallel-extensions-install is enabled) (PR #3890)
    • changes
      • ...
  • easyblocks
    • bug fixes
      • remove newline from package description (required to avoid installation error with setuptools>=59.0.0) (PR #2623)
      • update CP2K easyblock to use -D__MKL for CP2K versions newer than 2.3 (PR #2586)
    • enhancements
      • update Inspector easyblock for recent versions (>= 2021, oneAPI versions) (PR #2624)
      • update SAMtools easyblock to support installation of SAMtools 1.14 (PR #2620)
    • changes
      • ...
  • easyconfigs
    • >50 easyconfig PRs merged since last conf call
    • bug fixes
      • set $DALTON_TMPDIR to build directory to run Dalton tests PR #14341)
      • add patch for libfabric v1.12.1 to fix 'unknown link width 0x10' warning (PR #14387)
      • add patch for Kent_tools v422 to ensure OpenSSL dependency is used + add missing freetype dependency (PR #14325)
      • add pkg-config build dependency to Python easyconfigs, to ensure OpenSSL wrapper is detected (PR #14316)
    • (noteworthy) new software
      • see also overview at https://github.com/easybuilders/easybuild-easyconfigs/issues?q=is%3Amerged+label%3Anew
      • ...
    • noteworthy software updates
      • PyTorch 1.10.0 with foss/2021a + CUDA-11.3.1 (PR #14233)
      • CUDA 11.5.0 (PR #14372)
    • changes
      • ...
• to merge/fix/tackle soon
  • framework
    • reported bugs / bug fixes
      • fix builddir value when --sanity-check-only is used and buildininstalldir is enabled (PR #3900)
        • fixes (issue #3895)
      • git_config confuses branches and tags (issue #3902)
    • enhancements
      • ...
  • easyblocks
    • bug fixes
      • set wm_* class variable in OpenFOAM easyblock when using --sanity-check-only (PR #2616)
    • enhancements
      • enhance Amber easyblock for AmberTools (PR #2618)
      • add option to control MKL-DNN in jaxlib (PR #2619)
      • add option to ignore PLUMED version check for GROMACS (PR #2621)
      • update PyTorch easyblock to avoid RPATH linking to CUDA stubs library in libcaffe2_nvrtc.so (PR #2622)
    • changes
      • allow oversubscription of MPI ranks w.r.t. available cores in sanity check
        • see also issue #13945: OpenMPI does not build on machine with six cores
        • allow oversubscription in OpenMPI MPI execution check (PR #2611)
          • see also PR #2570: prevent OpenMPI from failing on machines with less than 8 cores
        • skip motorBike tutorial case in OpenFOAM sanity check if available cores < 6 (PR #2615)
        • it would be better to have a central fix for this in framework, or at least something that can be leveraged easily from easyblocks/easyconfigs?
          • self.toolchain.allow_mpi_oversubscription() to call in sanity_check_step of specific easyblocks?
          • support for using sanity_check_allow_mpi_oversubscription = True in easyconfigs?
          • build option, configuration parameter?
    • new software
      • add custom easyblock for Clang-AOMP (PR #2617)
        • required for dependencies of HIP/ROCm 4.5.0
  • easyconfigs
    • new software
      • see https://github.com/easybuilders/easybuild-easyconfigs/labels/new
      • HIP (PR #14339)
      • ROCm (PR #14355
    • noteworthy software updates
      • ...
    • changes
      • remove imkl builddependency from FlexiBLAS (PR #14335)
        • don't belong in foss toolchain, also requires accepting accepting the Intel oneAPI EULA
      • avoid Rust build dependency for Python by installing pre-built wheel for cryptography (issue #14320)
        • Sam: could/should we use our own pre-built wheel for cryptography?
        • could be uploaded to https://sources.easybuild.io
      • replace pkg-config with pkgconf (issue #14390)

Q&A

• Victor: any updates on EasyBuild User Meeting?
  • still being planned, will likely be planned in week Jan 24-28 2021 (more news soon...)
  • online only, about half time
  • hackathon session to focus effort on specific aspects (like generating toolchain easyconfigs)
• any updates on easystack files?
  • still same status as it was a while ago
  • plans in EESSI project to actively work on enhancing the support (and making it stable)
• any efforts of testing installation of and installed easyconfigs?
  • a shared location for ReFrame tests for installations would be nice
  • should we set up a dedicated repository on easybuild-testing (or under EESSI organization?)
  • important aspect here is to avoid system-specific stuff in tests (like # cores, # GPUs, etc.)
    • Victor: recent developments in ReFrame to facilitate this
      • see ReFrame documentation:
        • https://reframe-hpc.readthedocs.io/en/stable/tutorial_advanced.html?highlight=variable#writing-a-run-only-regression-test
        • https://reframe-hpc.readthedocs.io/en/stable/manpage.html?highlight=variable#cmdoption-S
    • Sam: also reference values for performance are very system-specific
    • related tool by Victor to do performance monitoring over time
    • Kenneth: building up a "database" for measured performance values
  • see also task #6 in EESSI hackathon: https://github.com/EESSI/meetings/wiki/EESSI-hackathon-Dec'21