test on mvapich

[simonbyrne]

No description provided.

[giordano]

Probably need to increase the timeout, 20 minutes isn't even enough for building the package in spack. This spack environment for Ubuntu 22.04 (you should change the runs-on below, this also requires installing the Ubuntu package rdma-core):

# This is a Spack Environment file.
#
# It describes a set of packages to be installed, along with
# configuration settings.
spack:
  # add package specs to the `specs` list
  specs: [mvapich2]
  view: true
  concretizer:
    unify: true
  'compilers:':
  - compiler:
      spec: clang@12.0.1
      paths:
        cc: /usr/bin/clang-12
        cxx: /usr/bin/clang++-12
        f77: null
        fc: null
      flags: {}
      operating_system: ubuntu22.04
      target: x86_64
      modules: []
      environment: {}
      extra_rpaths: []
  - compiler:
      spec: clang@13.0.1
      paths:
        cc: /usr/bin/clang-13
        cxx: /usr/bin/clang++-13
        f77: null
        fc: null
      flags: {}
      operating_system: ubuntu22.04
      target: x86_64
      modules: []
      environment: {}
      extra_rpaths: []
  - compiler:
      spec: clang@14.0.0
      paths:
        cc: /usr/bin/clang
        cxx: /usr/bin/clang++
        f77: null
        fc: null
      flags: {}
      operating_system: ubuntu22.04
      target: x86_64
      modules: []
      environment: {}
      extra_rpaths: []
  - compiler:
      spec: gcc@9.5.0
      paths:
        cc: /usr/bin/gcc-9
        cxx: /usr/bin/g++-9
        f77: /usr/bin/gfortran-9
        fc: /usr/bin/gfortran-9
      flags: {}
      operating_system: ubuntu22.04
      target: x86_64
      modules: []
      environment: {}
      extra_rpaths: []
  - compiler:
      spec: gcc@10.4.0
      paths:
        cc: /usr/bin/gcc-10
        cxx: /usr/bin/g++-10
        f77: /usr/bin/gfortran-10
        fc: /usr/bin/gfortran-10
      flags: {}
      operating_system: ubuntu22.04
      target: x86_64
      modules: []
      environment: {}
      extra_rpaths: []
  - compiler:
      spec: gcc@11.3.0
      paths:
        cc: /usr/bin/gcc
        cxx: /usr/bin/g++
        f77: /usr/bin/gfortran
        fc: /usr/bin/gfortran
      flags: {}
      operating_system: ubuntu22.04
      target: x86_64
      modules: []
      environment: {}
      extra_rpaths: []
  - compiler:
      spec: gcc@12.1.0
      paths:
        cc: /usr/bin/gcc-12
        cxx: /usr/bin/g++-12
        f77: /usr/bin/gfortran-12
        fc: /usr/bin/gfortran-12
      flags: {}
      operating_system: ubuntu22.04
      target: x86_64
      modules: []
      environment: {}
      extra_rpaths: []
  'packages:':
    bison:
      externals:
      - spec: bison@3.8.2
        prefix: /usr
    cmake:
      externals:
      - spec: cmake@3.25.1
        prefix: /usr/local
    findutils:
      externals:
      - spec: findutils@4.8.0
        prefix: /usr
    flex:
      externals:
      - spec: flex@2.6.4+lex
        prefix: /usr
    gettext:
      externals:
      - spec: gettext@0.21
        prefix: /usr
    libtool:
      externals:
      - spec: libtool@2.4.6
        prefix: /usr
    m4:
      externals:
      - spec: m4@1.4.18
        prefix: /usr
    perl:
      externals:
      - spec: perl@5.34.0~cpanm+shared+threads
        prefix: /usr
    python:
      externals:
      - spec: python@3.10.6+bz2+crypt+ctypes+dbm+lzma+nis+pyexpat+pythoncmd+readline+sqlite3+ssl~tkinter+uuid+zlib
        prefix: /usr
    rdma-core:
      externals:
      - spec: rdma-core@39.0
        prefix: /usr
    tar:
      externals:
      - spec: tar@1.34
        prefix: /usr
    xz:
      externals:
      - spec: xz@5.2.5
        prefix: /usr

but honestly I'm not sure it's easy to build mvapich on a system without Infiniband.

[giordano]

Uh, you also need the Ubuntu package libibverbs-dev.

[giordano]

With the above spack environment and the Ubuntu packages rdma-core and libibverbs-dev, I was able to build mvapich on GitHub Actions

==> mvapich2: Successfully installed mvapich2-2.3.7-dmysxjspupkgl5giqs4pjkuqdzpwlzpp
  Stage: 0.81s.  Autoreconf: 0.00s.  Configure: 1m 50.69s.  Build: 30m 23.42s.  Install: 22.95s.  Total: 32m 37.95s

It takes about 32-33 minutes just to build it (excluding the other non-external dependencies, but they're pretty quick)

[simonbyrne]

might be easier if you just push to this branch?

[giordano]

Small progress: I managed to build and cache MVAPICH. However I didn't manage to make MPIPreferences find it, but with warm cache this should finally be easier to debug.

Edit: it turned out I had typed lib/ as lin/ in LD_LIBRARY_PATH 😣

[giordano]

Ok, this is finally working as in "MPI.jl tests start", but initialzing MVAPICH crashes immediately.

[giordano]

Spawn tests are failing as reported in #684 (comment).

[simonbyrne]

Fantastic, thanks @giordano!

[giordano]

MVAPICH tests (after excluding spawn) are now successful! Most of the other failures on julia nightly are related to JuliaLang/julia#48039

[simonbyrne]

Thanks for your help!