Skip to content

Conference call notes 20210721

Jump to bottom
ocaisa edited this page Jul 21, 2021 · 7 revisions

(back to Conference calls)

Notes on the 177th EasyBuild conference call, Wednesday July 21st 2021 (08:00 UTC)

Attendees

Alphabetical list of attendees (9):

  • Alan O'Cais (JSC)
  • Jörg Saßmannshausen (NIHR Biomedical Research Centre, UK)
  • Fotis Georgatos (EPFL)
  • Alexander Grund (TU Dresden, Germany)
  • Davide Vanzo (Microsoft)
  • Adam Huffman (Big Data Institute, Oxford, UK)
  • Mikael Öhman (Chalmers University of Technology, Sweden)

Agenda

  • overview of recent developments
  • Q&A

Recent developments

  • last release: EasyBuild v4.4.1 (July 6)
  • ETA next release: TBD
  • recent changes
    • framework
      • bug fixes
        • Fix ignore_test_failure not set for Extensions (PR #3782)
        • Fix forgotten renaming of 'l' to 'char' (PR #3758)
        • minor fixes to output of test reports when using multiple PRs (PR #3773)
      • enhancements
        • Don't add empty PATH or LD_LIBRARY_PATH (PR #3769)
        • Improve warning about missing intel fftw libs (PR #3776)
      • changes
        • remove Python 2.6 from list of supported Python versions in setup.py (PR #3767)
    • easyblocks
      • bug fixes
        • Use correct GTK+ for OpenCV (PR #2520)
        • Handle failure of running nvidia-smi in TensorFlow tests (PR #2506)
      • new easyblocks
      • changes
        • update sanity check in ESMF easyblock to take into account new binary names from version 8.1.0 onwards (PR #2512)
        • remove Python 2.6 from list of supported Python versions in setup.py (PR #2507)
        • fixed sanity check for later versions of molpro (PR #2499)
        • Use report_test_failure() in TF (PR #2519)
    • easyconfigs
      • over 50 easyconfig PRs merged since last conf call!
      • bug fixes
        • Avoid dependency on system GTK+ in GTK+ themes (PR #13449)
        • avoid downloading old numpy version during install for h5py (PR #13428)
        • Fix order of cffi in Python ECs (PR #13400)
        • Fix PyQt 5.15.1 ECs (PR #13371)
        • Fix Rust v1.52.1 EC (PR #13399)
      • enhancements
        • Nothing of note
      • new software
      • noteworthy software updates
        • (nothing major)
      • noteworthy changes
        • (nothing major) pull/13188))
    • documentation
      • (none)
  • to merge/fix/tackle soon
    • framework
      • reported bugs / bug fixes
        • Make logdir writable also when --stop/--fetch is given (PR #3771)
        • Delete all included easyblock modules before importing them one by one (PR #3780)
          • Should only have impacted Python2 (at least according to the test case added)
        • Flexiblas doesn't have static libs but we still populate environment variables (without checking them)
          • Can Flexiblas even support static libs?
            • Looking at the build scripts, it should probably work
          • Only reason codes use these is because that's the only way we provide a list of those libraries
            • Should supply an equivalent list for the shared libs
          • Issue opened at https://github.com/easybuilders/easybuild-framework/issues/3783 but there is a lot of discussion already in Slack
      • enhancements
        • Filter out duplicate paths added to module files (PR #3770)
          • Can get duplication from easyconfigs/easyblocks
        • support additional features in easystack files
          • support for filtering via labels (PR #3620)
        • avoid using a priority in prepend_module_path (Lmod) to avoid costly module calls (PR #3636)
        • add support for installing extensions in parallel (WIP) (PR #3667)
        • finding modules with multiple modulepaths and HMNS (issue #3703)
        • add support for NVHPC-based toolchains (PR #3736)
        • add --review-pr-max and --review-pr-filter options to limit easyconfigs shown in multi-diff (WIP) (PR #3754)
      • changes
        • also enable static analysis for Python 2.7 (PR #3725)
          • catches accidentally overwriting local variables in list comprehensions (like --from-pr --include-easyblocks-from-pr bug)
        • add per-extension timing (PR #3734)
        • raise error for patches without .patch extension and no destination given (WIP) PR #3738)
    • easyblocks
      • bug fixes
        • Fix sanity-check and debug builds of TensorFlow (PR #2522)
        • Honor --ignore-test-failure in PythonPackage (PR #2516)
        • treat files/directories of unpacked sources equally in PackedBinary (PR #2306)
        • --module-only doesn't always work as expected
          • we need a better way of catching this in tests
          • problem is that you typically need an actual installation to catch these problems, so can't be done in easyconfigs or easyblocks test suite run in CI
          • test installations done on generoso via boegelbot could be enhanced to catch problems with --module-only?
        • explicitly use only OpenBLAS for PyTorch if MKL is not used (PR #2448)
      • enhancements
        • Remove redundant CPATH entry of tbb modules (PR #2511)
        • Simplify Boost easyblock (PR #2513)
          • Still waiting for feedback
        • Reduce the number of command line args for CMake configure (PR #2514)
        • Add Ampere to Lammps (PR #2510)
          • Another open PR for later LAMMPS versions exists (but doesn't need to hold this back)
        • Clean up installation of Tkinter (PR #2510)
          • Little concern about backwards compatability
        • enhance test and install step of CMakePythonPackage easyblock (PR #2318)
        • add support for installing R extensions in parallel (WIP) (PR #2408)
        • allow for Perl modules being part of other, already installed Perl modules (PR #2386)
        • including FlexiBLAS as the default BLAS in foss will require easyblock changes (issue #2421)
        • should set BLA_VENDOR in CMakeMake easyblock if BLAS is in the toolchain (PR #2420)
        • enable make check and sanity check exec for OpenMPI (PR #2444)
      • changes
        • (nothing major)
      • new software
    • easyconfigs
      • bug fixes
        • (nothing major)
      • enhancements
        • (nothing major)
      • new software
        • (nothing major)
      • software updates
        • SciPy-bundle with intel/2021a (PR #12964)
          • need to look into handful of failing tests...

2021a update of common toolchains

Q&A

  • Should we archive some of the older stuff?
    • Intel 2017 for example is a struggle due to licence changes, FOSS 2017 still works
    • Archiving doesn't mean deleting, so may not need a major version bump
  • Work ongoing on Alphafold, deps on FOSS level are done. Next step is an easyconfig for Alphafold itself.
    • They now have a release and build instructions for a non-docker use case
    • Can use a later version of kalign according to the author of that (which you can get from GitHub)
    • As long as we use tagged releases, we should be fine
      • Running the test step is of course a good idea
      • Does the test suite need all the data files? They are huge...
    • Final "build" is just a set of python scripts, so should be reasonably straightforward
      • Some versions seem to unnecessarily strict in the requirements (like SciPy)
  • Python-bare and Perl-minimal exist now
    • Should we separate out the Python/Perl packages and use a bundle for these instead?
    • Could take a similar approach to Boost and have Python.bare and a bundle called Python
      • Little weird with easyblocks used by these, but it works
    • Could do it in the next toolchain?
  • Do we support the latest version of Spack in EasyBuild?
    • Older versions work fine
    • Later versions moved to using setenv scripts which doesn't work well with Lmod (e.g., 0.16)
    • Lmod has a translation tool for environment scripts, but any logic would be lost
Clone this wiki locally