Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SDSC: PKG - expanse/0.17.3/cpu/b - Missing Abinit (example application) #33

Closed
nwolter opened this issue Mar 7, 2023 · 25 comments
Closed

SDSC: PKG - expanse/0.17.3/cpu/b - Missing Abinit (example application) #33

nwolter opened this issue Mar 7, 2023 · 25 comments

Comments

@nwolter
Copy link

nwolter commented Mar 7, 2023

No description provided.

@nwolter
Copy link
Author

nwolter commented Mar 8, 2023

Missing Abinit in 0.17.3 software stack
This is one of our example applications, /cm/shared/exapmles/sdsc

@nwolter nwolter changed the title SDSC: PKG - expanse/0.17.3/cpu/a - Missing Abinit SDSC: PKG - expanse/0.17.3/cpu/a - Missing Abinit (example application) Mar 9, 2023
@mkandes
Copy link
Member

mkandes commented Apr 7, 2023

@mkandes - This is an open source package with a Spack pacakge ready for build testing.

@mkandes mkandes changed the title SDSC: PKG - expanse/0.17.3/cpu/a - Missing Abinit (example application) SDSC: PKG - expanse/0.17.3/cpu/b - Missing Abinit (example application) May 5, 2023
@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Let's again begin building our spec build script by inspecting the spack info about the package.

[mkandes@login02 ~]$ spack info abinit
AutotoolsPackage:   abinit

Description:
    ABINIT is a package whose main program allows one to find the total
    energy, charge density and electronic structure of systems made of
    electrons and nuclei (molecules and periodic solids) within Density
    Functional Theory (DFT), using pseudopotentials and a planewave or
    wavelet basis. ABINIT also includes options to optimize the geometry
    according to the DFT forces and stresses, or to perform molecular
    dynamics simulations using these forces, or to generate dynamical
    matrices, Born effective charges, and dielectric tensors, based on
    Density-Functional Perturbation Theory, and many more properties.
    Excited states can be computed within the Many-Body Perturbation Theory
    (the GW approximation and the Bethe-Salpeter equation), and Time-
    Dependent Density Functional Theory (for molecules). In addition to the
    main ABINIT code, different utility programs are provided.

Homepage: https://www.abinit.org/

Externally Detectable: 
    False

Tags: 
    None

Preferred version:  
    9.4.2     https://www.abinit.org/sites/default/files/packages/abinit-9.4.2.tar.gz

Safe versions:  
    9.4.2     https://www.abinit.org/sites/default/files/packages/abinit-9.4.2.tar.gz
    8.10.3    https://www.abinit.org/sites/default/files/packages/abinit-8.10.3.tar.gz
    8.10.2    https://www.abinit.org/sites/default/files/packages/abinit-8.10.2.tar.gz
    8.8.2     https://www.abinit.org/sites/default/files/packages/abinit-8.8.2.tar.gz
    8.6.3     https://www.abinit.org/sites/default/files/packages/abinit-8.6.3.tar.gz
    8.2.2     https://www.abinit.org/sites/default/files/packages/abinit-8.2.2.tar.gz
    8.0.8b    https://www.abinit.org/sites/default/files/packages/abinit-8.0.8b.tar.gz

Deprecated versions:  
    None

Variants:
    Name [Default]                    When    Allowed values          Description
    ==============================    ====    ====================    ================

    libxml2 [off]                     --      on, off                 Enable libxml2
                                                                      support, used by
                                                                      multibinit
    mpi [on]                          --      on, off                 Builds with MPI
                                                                      support.
                                                                      Requires MPI2+
    openmp [off]                      --      on, off                 Enables OpenMP
                                                                      threads. Use
                                                                      threaded FFTW3
    optimization-flavor [standard]    --      safe, standard,         Select the
                                              aggressive              optimization
                                                                      flavor to use.
    scalapack [off]                   --      on, off                 Enables
                                                                      scalapack
                                                                      support.
                                                                      Requires MPI
    wannier90 [off]                   --      on, off                 Enables the
                                                                      Wannier90
                                                                      library

Installation Phases:
    autoreconf    configure    build    install

Build Dependencies:
    atompaw  fftw-api   hdf5    libxc    mpi       netcdf-fortran  wannier90
    blas     gnuconfig  lapack  libxml2  netcdf-c  scalapack

Link Dependencies:
    atompaw  fftw-api  lapack  libxml2  netcdf-c        scalapack
    blas     hdf5      libxc   mpi      netcdf-fortran  wannier90

Run Dependencies:
    None

Virtual Packages: 
    None

[mkandes@login02 ~]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

We should also note the current version available on the system in the older expanse/0.15.4/cpu Spack instance.

[mkandes@login01 ~]$ module spider abinit

------------------------------------------------------------------------------------------------------------------------------------------------------------------
  abinit: abinit/8.10.3-openblas
------------------------------------------------------------------------------------------------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "abinit/8.10.3-openblas" module is available to load.

      cpu/0.15.4  gcc/9.2.0  mvapich2/2.3.6
 
    Help:
      ABINIT is a package whose main program allows one to find the total
      energy, charge density and electronic structure of systems made of
      electrons and nuclei (molecules and periodic solids) within Density
      Functional Theory (DFT), using pseudopotentials and a planewave or
      wavelet basis. ABINIT also includes options to optimize the geometry
      according to the DFT forces and stresses, or to perform molecular
      dynamics simulations using these forces, or to generate dynamical
      matrices, Born effective charges, and dielectric tensors, based on
      Density-Functional Perturbation Theory, and many more properties.
      Excited states can be computed within the Many-Body Perturbation Theory
      (the GW approximation and the Bethe-Salpeter equation), and Time-
      Dependent Density Functional Theory (for molecules). In addition to the
      main ABINIT code, different utility programs are provided.
      


 

[mkandes@login01 ~]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Create a spec build script in your shared instance configuration for build tests.

[mkandes@login02 ~]$ cd .spack/0.17.3/cpu/b/etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/
[mkandes@login02 specs]$ ls
elpa@2021.05.001.o24794597.exp-15-01  wannier90@3.1.0.o24775410.exp-15-40
elpa@2021.05.001.o24795000.exp-15-01  wannier90@3.1.0.o24776006.exp-15-01
elpa@2021.05.001.sh                   wannier90@3.1.0.o24800871.exp-15-02
spark@3.4.0.o22902088.exp-15-02       wannier90@3.1.0.sh
spark@3.4.0.sh
[mkandes@login02 specs]$ cp wannier90@3.1.0.sh abinit@9.4.2.sh
[mkandes@login02 specs]$

And then select the initial spec configuration to run.

[mkandes@login01 ~]$ spack spec -l abinit@9.4.2 % gcc@10.2.0 ~libxml2 +mpi ~openmp +scalapack +wannier90 ^fftw@3.3.10/$(spack find --format '{hash:7}' fftw@3.3.10 % gcc@10.2.0 ~mpi ~openmp) ^netlib-scalapack@2.1.0/$(spack find --format '{hash:7}' netlib-scalapack@2.1.0 % gcc@10.2.0 ^openmpi@4.1.3) ^wannier90@3.1.0/$(spack find --format '{hash:7}' wannier90@3.1.0 % gcc@10.2.0 ^openmpi@4.1.3)
Input spec
--------------------------------
abinit@9.4.2%gcc@10.2.0~libxml2+mpi~openmp+scalapack+wannier90
    ^fftw@3.3.10%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~mpi~openmp~pfft_patches precision=double,float arch=linux-rocky8-zen2
    ^netlib-scalapack@2.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo+pic+shared build_type=Release patches=1c9ce5fee1451a08c2de3cc87f446aeda0b818ebbce4ad0d980ddf2f2a0b2dc4,f2baedde688ffe4c20943c334f580eb298e04d6f35c86b90a1f4e8cb7ae344a2 arch=linux-rocky8-zen2
        ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2
        ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2
            ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2
                ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
                ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2
                    ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2
                    ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2
                    ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2
                ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2
            ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2
            ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
            ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2
            ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2
            ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2
            ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2
                ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2
    ^wannier90@3.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +shared arch=linux-rocky8-zen2
        ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr+ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2
        ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2
            ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2
                ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
                ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2
                    ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2
                    ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2
                    ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2
                ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2
            ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2
            ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
            ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2
            ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2
            ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2
            ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2
                ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2

Concretized
--------------------------------
==> Error: abinit@9.4.2%gcc@10.2.0~libxml2+mpi~openmp+scalapack+wannier90 ^fftw@3.3.10%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~mpi~openmp~pfft_patches precision=double,float arch=linux-rocky8-zen2 ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2 ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2 ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2 ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2 ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2 ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2 ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2 ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2 ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2 ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2 ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2 ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2 ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2 ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2 ^netlib-scalapack@2.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo+pic+shared build_type=Release patches=1c9ce5fee1451a08c2de3cc87f446aeda0b818ebbce4ad0d980ddf2f2a0b2dc4,f2baedde688ffe4c20943c334f580eb298e04d6f35c86b90a1f4e8cb7ae344a2 arch=linux-rocky8-zen2 ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2 ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2 ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2 ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr+ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2 ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2 ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2 ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2 ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2 ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2 ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2 ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2 ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2 ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2 ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2 ^wannier90@3.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +shared arch=linux-rocky8-zen2 ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2 ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2 ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2 ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2 is unsatisfiable, errors are:

    To see full clingo unsat cores, re-run with `spack --show-cores=full`
    For full, subset-minimal unsat cores, re-run with `spack --show-cores=minimized
    Warning: This may take (up to) hours for some specs
[mkandes@login01 ~]$

But there is a conflict on the first try.

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Hmm. It looks like the deployed version of wannier90 may have picked up OpenBLAS with +ilp64.

^wannier90@3.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +shared arch=linux-rocky8-zen2
        ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr+ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Yup. This was likely left in by accident. 56ae69a

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Performing hotfix within production instance.

. /cm/shared/apps/spack/0.17.3/cpu/b/share/spack/setup-env.sh
[spack_cpu@exp-15-56 openmpi@4.1.3]$ spack find -lvd wannier90
==> 1 installed package
-- linux-rocky8-zen2 / gcc@10.2.0 -------------------------------
wrzoklo wannier90@3.1.0+shared
w4zdqc3     openblas@0.3.18~bignuma~consistent_fpcsr+ilp64+locking+pic+shared threads=none
oq3qvsv     openmpi@4.1.3~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm
7rqkdv4         hwloc@2.6.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared
ykynzrw             libpciaccess@0.16
mgovjpj             libxml2@2.9.12~python
zduoj2d                 libiconv@1.16 libs=shared,static
paz7hxz                 xz@5.2.5~pic libs=shared,static
ws4iari                 zlib@1.2.11+optimize+pic+shared
5lhvslt             ncurses@6.2~symlinks+termlib abi=none
bimlmtn         libevent@2.1.8~openssl
fy2cjdg         lustre@2.15.2
ckhyr5e         numactl@2.0.14 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296
dpvrfip         pmix@3.2.1~docs+pmi_backwards_compatibility~restful
4kvl3fd         slurm@21.08.8~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc
dnpjjuc         ucx@1.10.1~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none
xjr3cuj             rdma-core@43.0~ipo build_type=RelWithDebInfo

[spack_cpu@exp-15-56 openmpi@4.1.3]$ spack uninstall wannier90@3.1.0/wrzoklo
==> The following packages will be uninstalled:

    -- linux-rocky8-zen2 / gcc@10.2.0 -------------------------------
    wrzoklo wannier90@3.1.0

==> Do you want to proceed? [y/N] y
==> Successfully uninstalled wannier90@3.1.0%gcc@10.2.0+shared arch=linux-rocky8-zen2/wrzoklo
[spack_cpu@exp-15-56 openmpi@4.1.3]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Set ~ilp64 in OpenBLAS constraint and re-run spec build script.

[spack_cpu@exp-15-56 openmpi@4.1.3]$ vi wannier90@3.1.0.sh 
[spack_cpu@exp-15-56 openmpi@4.1.3]$ sbatch wannier90@3.1.0.sh 
Submitted batch job 24815821
[spack_cpu@exp-15-56 openmpi@4.1.3]$ squeue -u $USER
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
          24815821 ind-share wannier9 spack_cp  R       0:06      1 exp-15-56
          24815803 ind-share     bash spack_cp  R       3:14      1 exp-15-56
[spack_cpu@exp-15-56 openmpi@4.1.3]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Fixed.

-rw-r--r-- 1 spack_cpu spack  33K Aug 23 19:07 wannier90@3.1.0.o24808571.exp-15-56
-rw-r--r-- 1 spack_cpu spack 2.3K Aug 24 09:17 wannier90@3.1.0.sh
drwxr-sr-x 2 spack_cpu spack  118 Aug 24 09:17 .
-rw-r--r-- 1 spack_cpu spack  33K Aug 24 09:19 wannier90@3.1.0.o24815821.exp-15-56
[spack_cpu@exp-15-56 openmpi@4.1.3]$ less wannier90@3.1.0.o24815821.exp-15-56
[spack_cpu@exp-15-56 openmpi@4.1.3]$ spack find -lvd wannier90
==> 1 installed package
-- linux-rocky8-zen2 / gcc@10.2.0 -------------------------------
4b3fvlj wannier90@3.1.0+shared
fgk2tlu     openblas@0.3.18~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none
oq3qvsv     openmpi@4.1.3~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm
7rqkdv4         hwloc@2.6.0~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared
ykynzrw             libpciaccess@0.16
mgovjpj             libxml2@2.9.12~python
zduoj2d                 libiconv@1.16 libs=shared,static
paz7hxz                 xz@5.2.5~pic libs=shared,static
ws4iari                 zlib@1.2.11+optimize+pic+shared
5lhvslt             ncurses@6.2~symlinks+termlib abi=none
bimlmtn         libevent@2.1.8~openssl
fy2cjdg         lustre@2.15.2
ckhyr5e         numactl@2.0.14 patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296
dpvrfip         pmix@3.2.1~docs+pmi_backwards_compatibility~restful
4kvl3fd         slurm@21.08.8~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc
dnpjjuc         ucx@1.10.1~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none
xjr3cuj             rdma-core@43.0~ipo build_type=RelWithDebInfo

[spack_cpu@exp-15-56 openmpi@4.1.3]$

mkandes added a commit that referenced this issue Aug 24, 2023
@mkandes
Redploy wannier90@3.1.0 with openblas@0.3.18 ~ilp64
faf8b3c 
See #33 for more information.
mkandes added a commit that referenced this issue Aug 24, 2023
@mkandes
Redeploy wannier90@3.1.0 with openblas@0.3.18 ~ilp64
0897147 
See #33 for more information.
@mkandes
Copy link
Member

mkandes commented Aug 24, 2023 •
edited
Loading

0897147

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Previous abinit spec now concretizes.

[mkandes@login02 ~]$ !998
spack spec -l abinit@9.4.2 % gcc@10.2.0 ~libxml2 +mpi ~openmp +scalapack +wannier90 ^fftw@3.3.10/$(spack find --format '{hash:7}' fftw@3.3.10 % gcc@10.2.0 ~mpi ~openmp) ^netlib-scalapack@2.1.0/$(spack find --format '{hash:7}' netlib-scalapack@2.1.0 % gcc@10.2.0 ^openmpi@4.1.3) ^wannier90@3.1.0/$(spack find --format '{hash:7}' wannier90@3.1.0 % gcc@10.2.0 ^openmpi@4.1.3)
Input spec
--------------------------------
abinit@9.4.2%gcc@10.2.0~libxml2+mpi~openmp+scalapack+wannier90
    ^fftw@3.3.10%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~mpi~openmp~pfft_patches precision=double,float arch=linux-rocky8-zen2
    ^netlib-scalapack@2.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo+pic+shared build_type=Release patches=1c9ce5fee1451a08c2de3cc87f446aeda0b818ebbce4ad0d980ddf2f2a0b2dc4,f2baedde688ffe4c20943c334f580eb298e04d6f35c86b90a1f4e8cb7ae344a2 arch=linux-rocky8-zen2
        ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2
        ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2
            ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2
                ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
                ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2
                    ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2
                    ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2
                    ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2
                ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2
            ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2
            ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
            ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2
            ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2
            ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2
            ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2
                ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2
    ^wannier90@3.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +shared arch=linux-rocky8-zen2
        ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2
        ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2
            ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2
                ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
                ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2
                    ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2
                    ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2
                    ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2
                ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2
            ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2
            ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
            ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2
            ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2
            ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2
            ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2
                ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2

Concretized
--------------------------------
qznmdw2  abinit@9.4.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~libxml2+mpi~openmp+scalapack+wannier90 optimization-flavor=standard patches=6c7e501a4d957c8506c6bf9d6db2f201bc3ac156ba3e998ef5fcc3cf1b581ad1 arch=linux-rocky8-zen2
2c4dttz      ^atompaw@4.1.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=152deb68e20577489faac871c27d92734ad08301a72733dd951f41f3d63d3021 arch=linux-rocky8-zen2
2yb6cx6          ^libxc@5.1.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cuda+shared cuda_arch=none arch=linux-rocky8-zen2
aqkfsad              ^perl@5.32.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +cpanm+shared+threads arch=linux-rocky8-zen2
thj4acg                  ^berkeley-db@18.1.40%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +cxx~docs+stl patches=b231fcc4d5cff05e5c3a4814f6a5af0e9a966428dc2176540d2c05aff41de522 arch=linux-rocky8-zen2
pulggjv                  ^bzip2@1.0.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~debug~pic+shared arch=linux-rocky8-zen2
bphm3rh                      ^diffutils@3.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
zduoj2d                          ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2
clf6bmr                  ^gdbm@1.19%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
clxlnwz                      ^readline@8.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
5lhvslt                          ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2
ws4iari                  ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2
fgk2tlu          ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2
qogw3ss      ^fftw@3.3.10%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~mpi~openmp~pfft_patches precision=double,float arch=linux-rocky8-zen2
2hpbqwg      ^hdf5@1.10.7%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cxx~fortran+hl~ipo~java+mpi+shared~szip~threadsafe+tools api=default build_type=RelWithDebInfo arch=linux-rocky8-zen2
vroxb66          ^cmake@3.21.4%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~doc+ncurses~openssl+ownlibs~qt build_type=Release arch=linux-rocky8-zen2
oq3qvsv          ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2
7rqkdv4              ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2
ykynzrw                  ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
mgovjpj                  ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2
paz7hxz                      ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2
bimlmtn              ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2
fy2cjdg              ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
ckhyr5e              ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2
dpvrfip              ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2
4kvl3fd              ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2
dnpjjuc              ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2
xjr3cuj                  ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2
qftywfd          ^pkgconf@1.8.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
ihrcufv      ^netcdf-c@4.8.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~dap~fsync~hdf4~jna+mpi~parallel-netcdf+pic+shared arch=linux-rocky8-zen2
yb23vii          ^m4@1.4.19%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +sigsegv patches=9dc5fbd0d5cb1037ab1e6d0ecc74a30df218d0a94bdd5a02759a97f62daca573,bfdffa7c2eb01021d5849b36972c069693654ad826c1a20b53534009a4ec7a89 arch=linux-rocky8-zen2
ig3ykvk              ^libsigsegv@2.13%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
rf4ostx      ^netcdf-fortran@4.5.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~doc+pic+shared arch=linux-rocky8-zen2
pywku55      ^netlib-scalapack@2.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo+pic+shared build_type=Release patches=1c9ce5fee1451a08c2de3cc87f446aeda0b818ebbce4ad0d980ddf2f2a0b2dc4,f2baedde688ffe4c20943c334f580eb298e04d6f35c86b90a1f4e8cb7ae344a2 arch=linux-rocky8-zen2
4b3fvlj      ^wannier90@3.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +shared arch=linux-rocky8-zen2

[mkandes@login02 ~]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Adding netcdf-c as final constraint for build spec.

[mkandes@login02 ~]$ spack spec -l abinit@9.4.2 % gcc@10.2.0 ~libxml2 +mpi ~openmp +scalapack +wannier90 ^fftw@3.3.10/$(spack find --format '{hash:7}' fftw@3.3.10 % gcc@10.2.0 ~mpi ~openmp) ^netlib-scalapack@2.1.0/$(spack find --format '{hash:7}' netlib-scalapack@2.1.0 % gcc@10.2.0 ^openmpi@4.1.3) ^netcdf-c@4.8.1/$(spack find --format '{hash:7}' netcdf-c@4.8.1 % gcc@10.2.0 +mpi ^openmpi@4.1.3) ^wannier90@3.1.0/$(spack find --format '{hash:7}' wannier90@3.1.0 % gcc@10.2.0 ^openmpi@4.1.3)
Input spec
--------------------------------
abinit@9.4.2%gcc@10.2.0~libxml2+mpi~openmp+scalapack+wannier90
    ^fftw@3.3.10%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~mpi~openmp~pfft_patches precision=double,float arch=linux-rocky8-zen2
    ^netcdf-c@4.8.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~dap~fsync~hdf4~jna+mpi+parallel-netcdf+pic+shared arch=linux-rocky8-zen2
        ^hdf5@1.10.7%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +cxx+fortran+hl~ipo+java+mpi+shared+szip~threadsafe+tools api=default build_type=RelWithDebInfo arch=linux-rocky8-zen2
            ^libaec@1.0.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2
            ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2
            ^openjdk@11.0.12_7%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
            ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2
                ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2
                    ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
                    ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2
                        ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2
                        ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2
                        ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2
                    ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2
                ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2
                ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
                ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2
                ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2
                ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2
                    ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2
            ^pkgconf@1.8.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
        ^parallel-netcdf@1.12.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~burstbuffer+cxx+fortran+pic+shared arch=linux-rocky8-zen2
    ^netlib-scalapack@2.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo+pic+shared build_type=Release patches=1c9ce5fee1451a08c2de3cc87f446aeda0b818ebbce4ad0d980ddf2f2a0b2dc4,f2baedde688ffe4c20943c334f580eb298e04d6f35c86b90a1f4e8cb7ae344a2 arch=linux-rocky8-zen2
        ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2
        ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2
            ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2
                ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
                ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2
                    ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2
                    ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2
                    ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2
                ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2
            ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2
            ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
            ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2
            ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2
            ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2
            ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2
                ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2
    ^wannier90@3.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +shared arch=linux-rocky8-zen2
        ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2
        ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2
            ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2
                ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
                ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2
                    ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2
                    ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2
                    ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2
                ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2
            ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2
            ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
            ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2
            ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2
            ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2
            ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2
                ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2

Concretized
--------------------------------
q47zuuw  abinit@9.4.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~libxml2+mpi~openmp+scalapack+wannier90 optimization-flavor=standard patches=6c7e501a4d957c8506c6bf9d6db2f201bc3ac156ba3e998ef5fcc3cf1b581ad1 arch=linux-rocky8-zen2
2c4dttz      ^atompaw@4.1.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=152deb68e20577489faac871c27d92734ad08301a72733dd951f41f3d63d3021 arch=linux-rocky8-zen2
2yb6cx6          ^libxc@5.1.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cuda+shared cuda_arch=none arch=linux-rocky8-zen2
aqkfsad              ^perl@5.32.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +cpanm+shared+threads arch=linux-rocky8-zen2
thj4acg                  ^berkeley-db@18.1.40%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +cxx~docs+stl patches=b231fcc4d5cff05e5c3a4814f6a5af0e9a966428dc2176540d2c05aff41de522 arch=linux-rocky8-zen2
pulggjv                  ^bzip2@1.0.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~debug~pic+shared arch=linux-rocky8-zen2
bphm3rh                      ^diffutils@3.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
zduoj2d                          ^libiconv@1.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  libs=shared,static arch=linux-rocky8-zen2
clf6bmr                  ^gdbm@1.19%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
clxlnwz                      ^readline@8.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
5lhvslt                          ^ncurses@6.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~symlinks+termlib abi=none arch=linux-rocky8-zen2
ws4iari                  ^zlib@1.2.11%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +optimize+pic+shared arch=linux-rocky8-zen2
fgk2tlu          ^openblas@0.3.18%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~bignuma~consistent_fpcsr~ilp64+locking+pic+shared threads=none arch=linux-rocky8-zen2
qogw3ss      ^fftw@3.3.10%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~mpi~openmp~pfft_patches precision=double,float arch=linux-rocky8-zen2
5o4oibc      ^hdf5@1.10.7%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +cxx+fortran+hl~ipo+java+mpi+shared+szip~threadsafe+tools api=default build_type=RelWithDebInfo arch=linux-rocky8-zen2
nqhtzbx          ^libaec@1.0.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2
ckhyr5e          ^numactl@2.0.14%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  patches=4e1d78cbbb85de625bad28705e748856033eaafab92a66dffd383a3d7e00cc94,62fc8a8bf7665a60e8f4c93ebbd535647cebf74198f7afafec4c085a8825c006,ff37630df599cfabf0740518b91ec8daaf18e8f288b19adaae5364dc1f6b2296 arch=linux-rocky8-zen2
27cv2ps          ^openjdk@11.0.12_7%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
oq3qvsv          ^openmpi@4.1.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~atomics~cuda~cxx~cxx_exceptions~gpfs~internal-hwloc~java+legacylaunchers+lustre~memchecker+pmi+pmix+romio~rsh~singularity+static+vt+wrapper-rpath cuda_arch=none fabrics=ucx schedulers=slurm arch=linux-rocky8-zen2
7rqkdv4              ^hwloc@2.6.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~cairo~cuda~gl~libudev+libxml2~netloc~nvml~opencl+pci~rocm+shared arch=linux-rocky8-zen2
ykynzrw                  ^libpciaccess@0.16%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
mgovjpj                  ^libxml2@2.9.12%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~python arch=linux-rocky8-zen2
paz7hxz                      ^xz@5.2.5%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~pic libs=shared,static arch=linux-rocky8-zen2
bimlmtn              ^libevent@2.1.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~openssl arch=linux-rocky8-zen2
fy2cjdg              ^lustre@2.15.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
dpvrfip              ^pmix@3.2.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~docs+pmi_backwards_compatibility~restful arch=linux-rocky8-zen2
4kvl3fd              ^slurm@21.08.8%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~gtk~hdf5~hwloc~mariadb~pmix+readline~restd sysconfdir=PREFIX/etc arch=linux-rocky8-zen2
dnpjjuc              ^ucx@1.10.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~assertions~cm+cma~cuda+dc~debug+dm~gdrcopy+ib-hw-tm~java~knem~logging+mlx5-dv+optimizations~parameter_checking+pic+rc~rocm+thread_multiple+ud~xpmem cuda_arch=none arch=linux-rocky8-zen2
xjr3cuj                  ^rdma-core@43.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo build_type=RelWithDebInfo arch=linux-rocky8-zen2
qftywfd          ^pkgconf@1.8.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native"  arch=linux-rocky8-zen2
2u4nujb      ^netcdf-c@4.8.1%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~dap~fsync~hdf4~jna+mpi+parallel-netcdf+pic+shared arch=linux-rocky8-zen2
njcetkt          ^parallel-netcdf@1.12.2%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~burstbuffer+cxx+fortran+pic+shared arch=linux-rocky8-zen2
i35rncs      ^netcdf-fortran@4.5.3%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~doc+pic+shared arch=linux-rocky8-zen2
pywku55      ^netlib-scalapack@2.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" ~ipo+pic+shared build_type=Release patches=1c9ce5fee1451a08c2de3cc87f446aeda0b818ebbce4ad0d980ddf2f2a0b2dc4,f2baedde688ffe4c20943c334f580eb298e04d6f35c86b90a1f4e8cb7ae344a2 arch=linux-rocky8-zen2
4b3fvlj      ^wannier90@3.1.0%gcc@10.2.0 cflags="-O2 -march=native" cxxflags="-O2 -march=native" fflags="-O2 -march=native" +shared arch=linux-rocky8-zen2

[mkandes@login02 ~]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Construct spec build script and then run it.

#!/usr/bin/env bash

#SBATCH --job-name=abinit@9.4.2
#SBATCH --account=use300
##SBATCH --reservation=rocky8u7_testing
#SBATCH --partition=ind-shared
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=16
#SBATCH --mem=32G
#SBATCH --time=00:30:00
#SBATCH --output=%x.o%j.%N

declare -xr LOCAL_TIME="$(date +'%Y%m%dT%H%M%S%z')"
declare -xir UNIX_TIME="$(date +'%s')"

declare -xr LOCAL_SCRATCH_DIR="/scratch/${USER}/job_${SLURM_JOB_ID}"
declare -xr TMPDIR="${LOCAL_SCRATCH_DIR}"

declare -xr SYSTEM_NAME='expanse'

declare -xr SPACK_VERSION='0.17.3'
declare -xr SPACK_INSTANCE_NAME='cpu'
declare -xr SPACK_INSTANCE_VERSION='b'
declare -xr SPACK_INSTANCE_DIR="/cm/shared/apps/spack/${SPACK_VERSION}/${SPACK_INSTANCE_NAME}/${SPACK_INSTANCE_VERSION}"

declare -xr SLURM_JOB_SCRIPT="$(scontrol show job ${SLURM_JOB_ID} | awk -F= '/Command=/{print $2}')"
declare -xr SLURM_JOB_MD5SUM="$(md5sum ${SLURM_JOB_SCRIPT})"

declare -xr SCHEDULER_MODULE='slurm'

echo "${UNIX_TIME} ${SLURM_JOB_ID} ${SLURM_JOB_MD5SUM} ${SLURM_JOB_DEPENDENCY}" 
echo ""

cat "${SLURM_JOB_SCRIPT}"

module purge
module load "${SCHEDULER_MODULE}"
module list
. "${SPACK_INSTANCE_DIR}/share/spack/setup-env.sh"
shopt -s expand_aliases
source ~/.bashrc

declare -xr SPACK_PACKAGE='abinit@9.4.2'
declare -xr SPACK_COMPILER='gcc@10.2.0'
declare -xr SPACK_VARIANTS='~libxml2 +mpi ~openmp +scalapack +wannier90'
declare -xr SPACK_DEPENDENCIES="^fftw@3.3.10/$(spack find --format '{hash:7}' fftw@3.3.10 % ${SPACK_COMPILER} ~mpi ~openmp) ^netlib-scalapack@2.1.0/$(spack find --format '{hash:7}' netlib-scalapack@2.1.0 % ${SPACK_COMPILER} ^openmpi@4.1.3) ^netcdf-c@4.8.1/$(spack find --format '{hash:7}' netcdf-c@4.8.1 % ${SPACK_COMPILER} +mpi ^openmpi@4.1.3) ^wannier90@3.1.0/$(spack find --format '{hash:7}' wannier90@3.1.0 % ${SPACK_COMPILER} ^openmpi@4.1.3)"
declare -xr SPACK_SPEC="${SPACK_PACKAGE} % ${SPACK_COMPILER} ${SPACK_VARIANTS} ${SPACK_DEPENDENCIES}"

printenv

spack config get compilers
spack config get config
spack config get mirrors
spack config get modules
spack config get packages
spack config get repos
spack config get upstreams

time -p spack spec --long --namespaces --types --reuse "${SPACK_SPEC}"
if [[ "${?}" -ne 0 ]]; then
  echo 'ERROR: spack concretization failed.'
  exit 1
fi

time -p spack install --jobs "${SLURM_CPUS_PER_TASK}" --fail-fast --yes-to-all --reuse "${SPACK_SPEC}"
if [[ "${?}" -ne 0 ]]; then
  echo 'ERROR: spack install failed.'
  exit 1
fi

spack module lmod refresh -y

#sbatch --dependency="afterok:${SLURM_JOB_ID}" ''

sleep 30

[mkandes@login02 specs]$ vi abinit@9.4.2.sh 
[mkandes@login02 specs]$ pwd
/home/mkandes/.spack/0.17.3/cpu/b/etc/spack/sdsc/expanse/0.17.3/cpu/b/specs
[mkandes@login02 specs]$ sbatch abinit@9.4.2.sh 
Submitted batch job 24816025
[mkandes@login02 specs]$ squeue -u $USER
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
  24578139_[10-19]   compute tar-ilsv  mkandes PD       0:00      1 (JobArrayTaskLimit)
        24578139_9   compute tar-ilsv  mkandes  R   15:00:52      1 exp-1-17
          24816025 ind-share abinit@9  mkandes  R       0:03      1 exp-15-44
[mkandes@login02 specs]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Build succeeded!

==> Installing abinit-9.4.2-q47zuuw6yvhcbzfgxtyvphv5w5qxm42e
==> No binary for abinit-9.4.2-q47zuuw6yvhcbzfgxtyvphv5w5qxm42e found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/d4/d40886f5c8b138bb4aa1ca05da23388eb70a682790cfe5020ecce4db1b1a76bc.tar.gz
==> Applied patch /cm/shared/apps/spack/0.17.3/cpu/b/var/spack/repos/builtin/packages/abinit/rm_march_settings_v9.patch
==> abinit: Executing phase: 'autoreconf'
==> abinit: Executing phase: 'configure'
==> abinit: Executing phase: 'build'
==> abinit: Executing phase: 'install'
==> abinit: Successfully installed abinit-9.4.2-q47zuuw6yvhcbzfgxtyvphv5w5qxm42e
  Fetch: 4.91s.  Build: 18m 24.65s.  Total: 18m 29.55s.
[+] /home/mkandes/.spack/0.17.3/cpu/b/opt/spack/linux-rocky8-zen2/gcc-10.2.0/abinit-9.4.2-q47zuuw6yvhcbzfgxtyvphv5w5qxm42e
real 1396.79
user 952.51
sys 105.22
==> Regenerating lmod module files

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Modified example abinit job from /cm/shared/examples/sdsc/abinit on Expanse ran successfully.

[mkandes@login01 abinit]$ pwd
/home/mkandes/software/abinit
[mkandes@login01 abinit]$ cat run-abinit.sh 
#!/usr/bin/env bash

#SBATCH --job-name=abinit-si-tgw1
#SBATCH --account=use300
#SBATCH --partition=debug
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=4
#SBATCH --cpus-per-task=1
#SBATCH --mem=8G
#SBATCH --time=00:15:00
#SBATCH --output=%x.o%j.%N

module reset
module load gcc/10.2.0
module load openmpi/4.1.3
shopt -s expand_aliases
source ~/.bashrc
module load openmpi/4.1.3-oq3qvsv/gcc/10.2.0/abinit/9.4.2/q47zuuw
module list
export OMP_STACKSIZE='512M'
export OMP_NUM_THREADS=1
printenv

time -p srun --mpi=pmi2 -n "${SLURM_NTASKS}" abinit < tgw1_x.files 
[mkandes@login01 abinit]$ ls
14si.pspnc                         tgw1o_DS1_EIG.nc      tgw1o_DS3_SCR
abinit-si-tgw1.o24817369.exp-9-55  tgw1o_DS1_GSR.nc      tgw1o_DS4_GW
run-abinit.sh                      tgw1o_DS1_WFK         tgw1o_DS4_GWDIAG
tgw1_1.in                          tgw1o_DS2_DEN         tgw1o_DS4_SGR
tgw1_1.out                         tgw1o_DS2_EBANDS.agr  tgw1o_DS4_SGR_SIGC
tgw1o_DS1_DDB                      tgw1o_DS2_EIG         tgw1o_DS4_SIG
tgw1o_DS1_DEN                      tgw1o_DS2_EIG.nc      tgw1o_DS4_SIGRES.nc
tgw1o_DS1_EBANDS.agr               tgw1o_DS2_GSR.nc      tgw1o_OUT.nc
tgw1o_DS1_EIG                      tgw1o_DS2_WFK         tgw1_x.files
[mkandes@login01 abinit]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Resync personal fork of sdsc/spack repo.

[mkandes@login02 mkandes]$ git fetch --tags upstream
remote: Enumerating objects: 28, done.
remote: Counting objects: 100% (28/28), done.
remote: Compressing objects: 100% (15/15), done.
remote: Total 28 (delta 7), reused 26 (delta 5), pack-reused 0
Unpacking objects: 100% (28/28), 25.63 KiB | 14.00 KiB/s, done.
From https://github.com/sdsc/spack
   67e74f0e08..0897147105  sdsc-0.17.3 -> upstream/sdsc-0.17.3
[mkandes@login02 mkandes]$ git merge upstream/sdsc-0.17.3
Updating 67e74f0e08..0897147105
Fast-forward
 .../wannier90@3.1.0.o24815821.exp-15-56            | 690 +++++++++++++++++++++
 .../gcc@10.2.0/openmpi@4.1.3/wannier90@3.1.0.sh    |  76 +++
 var/spack/repos/sdsc/packages/wannier90/make.sys   |   7 +
 var/spack/repos/sdsc/packages/wannier90/package.py | 199 ++++++
 4 files changed, 972 insertions(+)
 create mode 100644 etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/gcc@10.2.0/openmpi@4.1.3/wannier90@3.1.0.o24815821.exp-15-56
 create mode 100644 etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/gcc@10.2.0/openmpi@4.1.3/wannier90@3.1.0.sh
 create mode 100644 var/spack/repos/sdsc/packages/wannier90/make.sys
 create mode 100644 var/spack/repos/sdsc/packages/wannier90/package.py
[mkandes@login02 mkandes]$ git push
Username for 'https://github.com': mkandes
Password for 'https://mkandes@github.com': 
Enumerating objects: 51, done.
Counting objects: 100% (51/51), done.
Delta compression using up to 64 threads
Compressing objects: 100% (20/20), done.
Writing objects: 100% (28/28), 12.30 KiB | 4.10 MiB/s, done.
Total 28 (delta 12), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (12/12), completed with 7 local objects.
To https://github.com/mkandes/spack.git
   67e74f0e08..0897147105  sdsc-0.17.3 -> sdsc-0.17.3
[mkandes@login02 mkandes]$ pwd
/home/mkandes/software/spack/repos/mkandes
[mkandes@login02 mkandes]$ git log
commit 089714710505385c44bb6cb3e8136d55b6b852ce (HEAD -> sdsc-0.17.3, upstream/sdsc-0.17.3, origin/sdsc-0.17.3, origin/HEAD)
Author: Marty Kandes <mkandes@sdsc.edu>
Date:   Thu Aug 24 09:22:42 2023 -0700

    Redeploy wannier90@3.1.0 with openblas@0.3.18 ~ilp64
    
    See https://github.com/sdsc/spack/issues/33 for more information.

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023 •
edited
Loading

Create a spec branch off sdsc-0.17.3 deployment (trunk) branch.

[mkandes@login02 mkandes]$ git branch sdsc-0.17.3-gh-33-spec-abinit
[mkandes@login02 mkandes]$ git checkout sdsc-0.17.3-gh-33-spec-abinit
Switched to branch 'sdsc-0.17.3-gh-33-spec-abinit'
[mkandes@login02 mkandes]$ git branch
  sdsc-0.17.3
* sdsc-0.17.3-gh-33-spec-abinit
  sdsc-0.17.3-gh-46-pkg-spec-wannier90
[mkandes@login02 mkandes]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Commit spec build script to spec branch and push back to personal fork.

[mkandes@login02 mkandes]$ cd etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/gcc@10.2.0/openmpi@4.1.3/
[mkandes@login02 openmpi@4.1.3]$ cp /home/mkandes/.spack/0.17.3/cpu/b/etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/abinit@9.4.2.sh ./
[mkandes@login02 openmpi@4.1.3]$ vi abinit@9.4.2.sh 
[mkandes@login02 openmpi@4.1.3]$ vi wannier90@3.1.0.sh 
[mkandes@login02 openmpi@4.1.3]$ git add wannier90@3.1.0.sh 
[mkandes@login02 openmpi@4.1.3]$ git add abinit@9.4.2.sh 
[mkandes@login02 openmpi@4.1.3]$ git status
On branch sdsc-0.17.3-gh-33-spec-abinit
Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	new file:   abinit@9.4.2.sh
	modified:   wannier90@3.1.0.sh


It took 5.53 seconds to enumerate untracked files. 'status -uno'
may speed it up, but you have to be careful not to forget to add
new files yourself (see 'git help status').
[mkandes@login02 openmpi@4.1.3]$ git commit
[sdsc-0.17.3-gh-33-spec-abinit 873d6f5d2e] Add abinit@9.4.2 % gcc@10.2.0 ^openmpi@4.1.3 to expanse/0.17.3/cpu/b
 2 files changed, 77 insertions(+), 1 deletion(-)
 create mode 100644 etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/gcc@10.2.0/openmpi@4.1.3/abinit@9.4.2.sh
[mkandes@login02 openmpi@4.1.3]$ git log
commit 873d6f5d2ec79acf562a2c1e578024788b5b4280 (HEAD -> sdsc-0.17.3-gh-33-spec-abinit)
Author: Marty Kandes <mkandes@sdsc.edu>
Date:   Thu Aug 24 11:59:22 2023 -0700

    Add abinit@9.4.2 % gcc@10.2.0 ^openmpi@4.1.3 to expanse/0.17.3/cpu/b

commit 089714710505385c44bb6cb3e8136d55b6b852ce (upstream/sdsc-0.17.3, origin/sdsc-0.17.3, origin/HEAD, sdsc-0.17.3)
Author: Marty Kandes <mkandes@sdsc.edu>
Date:   Thu Aug 24 09:22:42 2023 -0700

    Redeploy wannier90@3.1.0 with openblas@0.3.18 ~ilp64
    
    See https://github.com/sdsc/spack/issues/33 for more information.

commit 2b4e0bc03bff7d6741c2d288049932da39ece92e
Author: Marty Kandes <mkandes@sdsc.edu>
Date:   Wed Aug 23 19:15:32 2023 -0700

    Deploy wannier90@3.1.0 % gcc@10.2.0 ^openmpi@4.1.3 into exp/0.17.3/cpu/b

commit 56ae69a027390e8f6d80ca45f1f631a266aba598
[mkandes@login02 openmpi@4.1.3]$ git push
fatal: The current branch sdsc-0.17.3-gh-33-spec-abinit has no upstream branch.
To push the current branch and set the remote as upstream, use

    git push --set-upstream origin sdsc-0.17.3-gh-33-spec-abinit

To have this happen automatically for branches without a tracking
upstream, see 'push.autoSetupRemote' in 'git help config'.

[mkandes@login02 openmpi@4.1.3]$ git push --set-upstream origin sdsc-0.17.3-gh-33-spec-abinit
Enumerating objects: 26, done.
Counting objects: 100% (26/26), done.
Delta compression using up to 64 threads
Compressing objects: 100% (10/10), done.
Writing objects: 100% (14/14), 2.01 KiB | 2.01 MiB/s, done.
Total 14 (delta 6), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (6/6), completed with 6 local objects.
remote: 
remote: Create a pull request for 'sdsc-0.17.3-gh-33-spec-abinit' on GitHub by visiting:
remote:      https://github.com/mkandes/spack/pull/new/sdsc-0.17.3-gh-33-spec-abinit
remote: 
To https://github.com/mkandes/spack.git
 * [new branch]            sdsc-0.17.3-gh-33-spec-abinit -> sdsc-0.17.3-gh-33-spec-abinit
branch 'sdsc-0.17.3-gh-33-spec-abinit' set up to track 'origin/sdsc-0.17.3-gh-33-spec-abinit'.
[mkandes@login02 openmpi@4.1.3]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Pull request created and merged into sdsc/spack sdsc-0.17.3 deployment branch. #95

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Pull changes into expanse/0.17.3/cpu/b production instance.

[mkandes@login02 ~]$ !900
sudo -u spack_cpu ssh spack_cpu@login.expanse.sdsc.edu
PIN+Yubi: 
Welcome to Bright release         9.0

                                                         Based on Rocky Linux 8
                                                                    ID: #000002

--------------------------------------------------------------------------------

                                 WELCOME TO
                  _______  __ ____  ___    _   _______ ______
                 / ____/ |/ // __ \/   |  / | / / ___// ____/
                / __/  |   // /_/ / /| | /  |/ /\__ \/ __/
               / /___ /   |/ ____/ ___ |/ /|  /___/ / /___
              /_____//_/|_/_/   /_/  |_/_/ |_//____/_____/

--------------------------------------------------------------------------------

Use the following commands to adjust your environment:

'module avail'            - show available modules
'module add <module>'     - adds a module to your environment for this session
'module initadd <module>' - configure module to be loaded at every login

-------------------------------------------------------------------------------
Last login: Wed Aug 23 18:51:59 2023 from 198.202.100.14
[spack_cpu@login02 ~]$ srun --partition=ind-shared --reservation=root_73  --account=use300 --nodes=1 --nodelist=exp-15-56 --ntasks-per-node=1 --cpus-per-task=16 --mem=32G --time=12:00:00 --pty --wait=0 /bin/bash
[spack_cpu@exp-15-56 ~]$ cd /cm/shared/apps/spack/0.17.3/cpu/b/
[spack_cpu@exp-15-56 b]$ ls
bin              DEPLOYMENT.md   LICENSE-MIT     pytest.ini   var
CHANGELOG.md     etc             NOTICE          README.md
CONTRIBUTING.md  lib             opt             SECURITY.md
COPYRIGHT        LICENSE-APACHE  pyproject.toml  share
[spack_cpu@exp-15-56 b]$ git stash
No local changes to save
[spack_cpu@exp-15-56 b]$ git pull
remote: Enumerating objects: 15, done.
remote: Counting objects: 100% (15/15), done.
remote: Compressing objects: 100% (5/5), done.
remote: Total 15 (delta 7), reused 13 (delta 6), pack-reused 0
Unpacking objects: 100% (15/15), 2.49 KiB | 182.00 KiB/s, done.
From https://github.com/sdsc/spack
   0897147105..c8c44dd1dd  sdsc-0.17.3 -> origin/sdsc-0.17.3
Updating 0897147105..c8c44dd1dd
Fast-forward
 .../specs/gcc@10.2.0/openmpi@4.1.3/abinit@9.4.2.sh | 76 ++++++++++++++++++++++
 .../gcc@10.2.0/openmpi@4.1.3/wannier90@3.1.0.sh    |  2 +-
 2 files changed, 77 insertions(+), 1 deletion(-)
 create mode 100644 etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/gcc@10.2.0/openmpi@4.1.3/abinit@9.4.2.sh
[spack_cpu@exp-15-56 b]$ git stash pop
No stash entries found.
[spack_cpu@exp-15-56 b]$ git log
commit c8c44dd1dd31c9b9f64fe530a32597ff1890d39e (HEAD -> sdsc-0.17.3, origin/sdsc-0.17.3, origin/HEAD)
Merge: 0897147105 873d6f5d2e
Author: Marty Kandes <mkandes@sdsc.edu>
Date:   Thu Aug 24 12:01:35 2023 -0700

    Merge pull request #95 from mkandes/sdsc-0.17.3-gh-33-spec-abinit
    
    Add abinit@9.4.2 % gcc@10.2.0 ^openmpi@4.1.3 to expanse/0.17.3/cpu/b

commit 873d6f5d2ec79acf562a2c1e578024788b5b4280
Author: Marty Kandes <mkandes@sdsc.edu>
Date:   Thu Aug 24 11:59:22 2023 -0700

    Add abinit@9.4.2 % gcc@10.2.0 ^openmpi@4.1.3 to expanse/0.17.3/cpu/b

commit 089714710505385c44bb6cb3e8136d55b6b852ce
Author: Marty Kandes <mkandes@sdsc.edu>
Date:   Thu Aug 24 09:22:42 2023 -0700

    Redeploy wannier90@3.1.0 with openblas@0.3.18 ~ilp64
    
    See https://github.com/sdsc/spack/issues/33 for more information.
[spack_cpu@exp-15-56 b]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Run spec build.

[spack_cpu@exp-15-56 b]$ cd etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/gcc@10.2.0/openmpi@4.1.3/
[spack_cpu@exp-15-56 openmpi@4.1.3]$ sbatch abinit@9.4.2.sh 
Submitted batch job 24818063
[spack_cpu@exp-15-56 openmpi@4.1.3]$ squeue -u $USER
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
          24818063 ind-share abinit@9 spack_cp PD       0:00      1 (None)
          24817869 ind-share     bash spack_cp  R       2:44      1 exp-15-56
[spack_cpu@exp-15-56 openmpi@4.1.3]$ squeue -u $USER
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
          24818063 ind-share abinit@9 spack_cp  R       0:09      1 exp-15-56
          24817869 ind-share     bash spack_cp  R       2:53      1 exp-15-56
[spack_cpu@exp-15-56 openmpi@4.1.3]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Build succeeded!

==> Installing abinit-9.4.2-q47zuuw6yvhcbzfgxtyvphv5w5qxm42e
==> No binary for abinit-9.4.2-q47zuuw6yvhcbzfgxtyvphv5w5qxm42e found: installing from source
==> Fetching https://mirror.spack.io/_source-cache/archive/d4/d40886f5c8b138bb4aa1ca05da23388eb70a682790cfe5020ecce4db1b1a76bc.tar.gz
==> Applied patch /cm/shared/apps/spack/0.17.3/cpu/b/var/spack/repos/builtin/packages/abinit/rm_march_settings_v9.patch
==> abinit: Executing phase: 'autoreconf'
==> abinit: Executing phase: 'configure'
==> abinit: Executing phase: 'build'
==> abinit: Executing phase: 'install'
==> abinit: Successfully installed abinit-9.4.2-q47zuuw6yvhcbzfgxtyvphv5w5qxm42e
  Fetch: 0.32s.  Build: 7m 5.60s.  Total: 7m 5.92s.
[+] /cm/shared/apps/spack/0.17.3/cpu/b/opt/spack/linux-rocky8-zen2/gcc-10.2.0/abinit-9.4.2-q47zuuw6yvhcbzfgxtyvphv5w5qxm42e
real 474.42
user 957.77
sys 88.54
==> Regenerating lmod module files
==> OpenFOAM bashrc env: /cm/shared/apps/spack/0.17.3/cpu/b/opt/spack/linux-rocky8-zen2/aocc-3.2.0/openfoam-2106-jz42us227mirxrhqjvojlaut2giuh74j/etc/bashrc

[mkandes@login02 ~]$ module spider abinit/9.4.2

--------------------------------------------------------------------------------------------------------------------------------------
  abinit/9.4.2: abinit/9.4.2/q47zuuw
--------------------------------------------------------------------------------------------------------------------------------------

    You will need to load all module(s) on any one of the lines below before the "abinit/9.4.2/q47zuuw" module is available to load.

      cpu/0.17.3b  gcc/10.2.0/npcyll4  openmpi/4.1.3/oq3qvsv
 
    Help:
      ABINIT is a package whose main program allows one to find the total
      energy, charge density and electronic structure of systems made of
      electrons and nuclei (molecules and periodic solids) within Density
      Functional Theory (DFT), using pseudopotentials and a planewave or
      wavelet basis. ABINIT also includes options to optimize the geometry
      according to the DFT forces and stresses, or to perform molecular
      dynamics simulations using these forces, or to generate dynamical
      matrices, Born effective charges, and dielectric tensors, based on
      Density-Functional Perturbation Theory, and many more properties.
      Excited states can be computed within the Many-Body Perturbation Theory
      (the GW approximation and the Bethe-Salpeter equation), and Time-
      Dependent Density Functional Theory (for molecules). In addition to the
      main ABINIT code, different utility programs are provided.
[mkandes@login02 ~]$

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Re-run test job with production version now deployed.

[mkandes@login02 abinit]$ pwd
/home/mkandes/software/abinit
[mkandes@login02 abinit]$ cat run-abinit.sh 
#!/usr/bin/env bash

#SBATCH --job-name=abinit-si-tgw1
#SBATCH --account=use300
#SBATCH --partition=debug
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=4
#SBATCH --cpus-per-task=1
#SBATCH --mem=8G
#SBATCH --time=00:15:00
#SBATCH --output=%x.o%j.%N

module reset
module load gcc/10.2.0
module load openmpi/4.1.3
module load abinit/9.4.2/q47zuuw
module list
export OMP_STACKSIZE='512M'
export OMP_NUM_THREADS=1
printenv

time -p srun --mpi=pmi2 -n "${SLURM_NTASKS}" abinit < tgw1_x.files 
[mkandes@login02 abinit]$ sbatch run-abinit.sh 
Submitted batch job 24818333
[mkandes@login02 abinit]$ vi run-abinit.sh 
[mkandes@login02 abinit]$ squeue -u $USER
             JOBID PARTITION     NAME     USER ST       TIME  NODES NODELIST(REASON)
  24578139_[10-19]   compute tar-ilsv  mkandes PD       0:00      1 (JobArrayTaskLimit)
        24578139_9   compute tar-ilsv  mkandes  R   17:49:56      1 exp-1-17
[mkandes@login02 abinit]$ ls
14si.pspnc                         tgw1_1.out            tgw1o_DS1_EIG.nc      tgw1o_DS2_EIG     tgw1o_DS4_GW        tgw1o_DS4_SIGRES.nc
abinit-si-tgw1.o24817369.exp-9-55  tgw1o_DS1_DDB         tgw1o_DS1_GSR.nc      tgw1o_DS2_EIG.nc  tgw1o_DS4_GWDIAG    tgw1o_OUT.nc
abinit-si-tgw1.o24818333.exp-9-55  tgw1o_DS1_DEN         tgw1o_DS1_WFK         tgw1o_DS2_GSR.nc  tgw1o_DS4_SGR       tgw1_x.files
run-abinit.sh                      tgw1o_DS1_EBANDS.agr  tgw1o_DS2_DEN         tgw1o_DS2_WFK     tgw1o_DS4_SGR_SIGC
tgw1_1.in                          tgw1o_DS1_EIG         tgw1o_DS2_EBANDS.agr  tgw1o_DS3_SCR     tgw1o_DS4_SIG
[mkandes@login02 abinit]$

Success!

@mkandes
Copy link
Member

mkandes commented Aug 24, 2023

Commit standard out from spec build job for the record.

[spack_cpu@exp-15-56 openmpi@4.1.3]$ git add abinit@9.4.2.o24818063.exp-15-56 
[spack_cpu@exp-15-56 openmpi@4.1.3]$ git commit
[sdsc-0.17.3 bccf259f4f] Deploy abinit@9.4.2 % gcc@10.2.0 ^openmpi@4.1.3 into exp/0.17.3/cpu/b
 1 file changed, 761 insertions(+)
 create mode 100644 etc/spack/sdsc/expanse/0.17.3/cpu/b/specs/gcc@10.2.0/openmpi@4.1.3/abinit@9.4.2.o24818063.exp-15-56
[spack_cpu@exp-15-56 openmpi@4.1.3]$ git push
Username for 'https://github.com': mkandes
Password for 'https://mkandes@github.com': 
Enumerating objects: 24, done.
Counting objects: 100% (24/24), done.
Delta compression using up to 128 threads
Compressing objects: 100% (9/9), done.
Writing objects: 100% (13/13), 11.36 KiB | 11.36 MiB/s, done.
Total 13 (delta 5), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (5/5), completed with 5 local objects.
To https://github.com/sdsc/spack.git
   c8c44dd1dd..bccf259f4f  sdsc-0.17.3 -> sdsc-0.17.3
[spack_cpu@exp-15-56 openmpi@4.1.3]$

@mkandes mkandes closed this as completed Aug 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mkandes @nwolter