{bio}[foss/2023a] AlphaFold v2.3.2, dm-haiku v0.0.12, tensorstore v0.1.65 w/ CUDA v12.1.1

[ThomasHoffmann77]

(created using eb --new-pr)
requires:

• {tools}[gfbf/2023a] jax v0.4.25 w/ CUDA 12.1.1 #20119
• {bio}[foss/2023a] OpenMM v8.0.0 w/ CUDA 12.1.1 #20362

[migueldiascosta]

fwiw, I'm getting error: HWCAP_NEON was not declared in this scope when building OpenMM from this PR on NVIDIA Grace-Hopper

it looks like openmm-8.0.0/cmake_modules/TargetArch.cmake is detecting Grace-Hopper as arm instead of armv8,

which then leads to openmm-8.0.0/CMakeLists.txt setting -D__ARM__=1 instead of -D__ARM64__=1,

which in turn leads openmm-8.0.0/openmmapi/include/openmm/internal/vectorize_neon.h to use HWCAP_NEON instead of HWCAP_ASIMD

forcing TARGET_ARCH to be armv8 in openmm-8.0.0/CMakeLists.txt fixed the issue for me

[ThomasHoffmann77]

fwiw, I'm getting error: HWCAP_NEON was not declared in this scope when building OpenMM from this PR on NVIDIA Grace-Hopper

it looks like openmm-8.0.0/cmake_modules/TargetArch.cmake is detecting Grace-Hopper as arm instead of armv8,

which then leads to openmm-8.0.0/CMakeLists.txt setting -D__ARM__=1 instead of -D__ARM64__=1,

which in turn leads openmm-8.0.0/openmmapi/include/openmm/internal/vectorize_neon.h to use HWCAP_NEON instead of HWCAP_ASIMD

forcing TARGET_ARCH to be armv8 in openmm-8.0.0/CMakeLists.txt fixed the issue for me

#18911

[VRehnberg]

Fyi, I've got a draft #20421 that might become relevant for this one as well. Perhaps you'll have opinions :).

[VRehnberg]

Thanks for adding this.

So typical use would be(?):

1. CPU only job to get features (gpu not detected)
2. Job-array to GPUs to run predictions (features.pkl found and --only-model-pred="${SLURM_ARRAY_TASK_ID}")
3. Single job with GPU to run relaxation (possibly in parallel, [How is this launched, or is it not run separately???])

[ThomasHoffmann77]

Thanks for adding this.

So typical use would be(?):

1. CPU only job to get features (gpu not detected)
2. Job-array to GPUs to run predictions (features.pkl found and --only-model-pred="${SLURM_ARRAY_TASK_ID}")

yes, for monomer jobs.
For multimer, you need to translate the array ID to X,Y with X in [1..5], Y in [0..4] (if you run with --num_multimer_predictions_per_model=5)

3. Single job with GPU to run relaxation (possibly in parallel, [How is this launched, or is it not run separately???])

In order to get the ranking, you can run a quick CPU job after running the predictions.
--models_to_relax default is changed from best to none. Therefore the pipeline stops after the predictions.
You can resume with the relaxation by restarting with --models_to_relax=all (or best).

[ThomasHoffmann77]

accidentally closed

requested changes done

[boegel]

lgtm

[boegel]

@boegelbot please test @ jsc-zen3-a100

[boegelbot]

@boegel: Request for testing this PR well received on jsczen3l1.int.jsc-zen3.fz-juelich.de

PR test command 'if [[ develop != 'develop' ]]; then EB_BRANCH=develop ./easybuild_develop.sh 2> /dev/null 1>&2; EB_PREFIX=/home/boegelbot/easybuild/develop source init_env_easybuild_develop.sh; fi; EB_PR=19942 EB_ARGS= EB_CONTAINER= EB_REPO=easybuild-easyconfigs EB_BRANCH=develop /opt/software/slurm/bin/sbatch --job-name test_PR_19942 --ntasks=8 --partition=jsczen3g --gres=gpu:1 ~/boegelbot/eb_from_pr_upload_jsc-zen3.sh' executed!

• exit code: 0
• output:

Submitted batch job 5066

Test results coming soon (I hope)...

- notification for comment with ID 2407894839 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

[boegel]

Test report by @boegel
SUCCESS
Build succeeded for 3 out of 3 (3 easyconfigs in total)
node3901.accelgor.os - Linux RHEL 8.8, x86_64, AMD EPYC 7413 24-Core Processor, 1 x NVIDIA NVIDIA A100-SXM4-80GB, 545.23.08, Python 3.6.8
See https://gist.github.com/boegel/cb450cdab5ed9c44eb1dd6e80e9541d2 for a full test report.

[boegelbot]

Test report by @boegelbot
SUCCESS
Build succeeded for 4 out of 4 (3 easyconfigs in total)
jsczen3g1.int.jsc-zen3.fz-juelich.de - Linux Rocky Linux 9.4, x86_64, AMD EPYC-Milan Processor (zen3), 1 x NVIDIA NVIDIA A100 80GB PCIe, 555.42.06, Python 3.9.18
See https://gist.github.com/boegelbot/c87f706d4ad650a5227d1ab0da8297d1 for a full test report.

[boegel]

Going in, thanks @ThomasHoffmann77!