Improved developer experience

[aidanheerdegen]

[Updated 2023/11/24 - by harshula]

We need a good standard method for using spack to build software at NCI, ideally with the following attributes

A general user (read-only):

• Don't need to git clone spack
• ACCESS-NRI spack_packages are available

An advanced user (in addition to a general user's requirements, also needs to be able to write/create new packages):

• System libraries are available and easy to use
• Built packages are installed in the users own /g/data/ space
• Read-only build cache available to speed up development time

[aidanheerdegen]

Currently the steps to use spack require a number of steps, including cloning spack

ACCESS-NRI/ACCESS-OM2#6 (comment)

[aidanheerdegen]

An example, and possible starting point:

https://forum.access-hive.org.au/t/shared-access-spack-configuration/227

[harshula]

V2 of the Spack config has been merged and should help improve the developer experience: PR #6

[harshula]

Improved documentation in PR #10

[harshula]

According to the Spack documentation, chaining may not be applicable to us because we have an install_tree outside of the Spack base directory:

https://spack.readthedocs.io/en/latest/chain.html

You can point your Spack installation to another installation to use any packages that are installed there. To register the other Spack instance, you can add it as an entry to upstreams.yaml:

upstreams:
  spack-instance-1:
    install_tree: /path/to/other/spack/opt/spack
  spack-instance-2:
    install_tree: /path/to/another/spack/opt/spack

install_tree must point to the opt/spack directory inside of the Spack base directory.

[harshula]

[23/04/2024: Refer to instructions on the ACCESS-Hive Forum]
https://forum.access-hive.org.au/t/how-to-build-access-om2-on-gadi/1545

[harshula]

Notes

I've noticed a subtle difference between spack dev-build and spack install. dev-build appears to obtain a stale path for the Spack instance. It's not from ~/.spack.