Running as part of global-workflow fails in exgrid2grid_step2.sh with srun: no record for task id 1

[DWesl]

Running for a C768 run as part of global-workflow produces a specification with nodes=1, ppn=4, and tpp=1.
Running with ush/run_verif_global_in_global_workflow.sh produces a job with nproc=${npe_node_metp_gfs}=1.
When run on HERA, scripts/exgrid2grid_step1.sh launches the METplus job with srun --multi-prog /path/to/task-file, where task-file has nproc lines detailing commands to execute. srun then fails because it can't find as many tasks as it wants; I think it is defaulting to four tasks.

Changing scripts/exgrid2grid_step1.sh to specify --ntasks ${nproc} as part of the srun command allows the process to finish. A better solution probably involves changing how ush/run_verif_global_in_global_workflow.sh determines nproc: man sbatch suggests SLURM_NTASKS, but global-workflow probably has a variable to specify the number of threads that would be less closely tied to the job manager.

[DavidHuber-NOAA]

@DWesl This was recently fixed in the global-workflow as part of an overhaul of the resource configuration system. The job now runs with a single task by default. See NOAA-EMC/global-workflow#2804 and let me know if updating your global-workflow resolves the issue.

[DWesl]

The new setting in verif-global (should have checked this earlier) references nproc and defaults to one:

EMC_verif-global/ush/run_verif_global_in_global_workflow.sh

Lines 277 to 279 in 92904d2

	## Run settings for machines
	export MPMD="YES"
	export nproc=${nproc:-1}

and global-workflow sets nproc in config.metp, which should solve the problem more generally.