diff --git a/docs/2D-decomp.md b/docs/2D-decomp.md index 105a28cf0..16fa468ac 100644 --- a/docs/2D-decomp.md +++ b/docs/2D-decomp.md @@ -17,5 +17,51 @@ The structural changes needed for the 2D decomposition are implemented in MPI_FI Both MPI_FIRST.f and EXCH.f contain significant additional test code to exchange arrays containing grid coordinates and ensure EXACT matches for all exchanges before the domain exchanges are performed. This is intended to trap errors in the larger variety of 2D decomposition layouts that are possible and most of it can eventually be removed or made conditional at build and run time. -Indices and variables to facilitate the 2D decomposition are found in CTLBLK.f and shared in the rest of UPP through use of CTLBLK.mod. +The following is found in CTLBLK.f and shared in the rest of UPP through use of CTLBLK.mod: + +| Variable | Type | Description | +|---------|------|-------------| +| im | integer | full longitude domain| +| jm | integer | full latitude domain| +| | | | +| jsta | integer | start latitude on a task subdomain| +| jend | integer | end latitude on a task subdomain| +| ista | integer | start longitude on a task subdomain| +| iend | integer | end longitude on a task subdomain| +| | | | +| ista_2l | integer |start longitude -2 of the subdomain| +| iend_2u | integer |end longitude +2 of the subdomain| +| jsta_2l | integer |start latitude -2 of the subdomain| +| jend_2u | integer |end latitude +2 of the subdomain| + +The shape of the subdomain is ista_2l:iend_2u,jsta_2l:jend_2u so it includes the halos although the halos are not populated until exchange is done in EXCH.f. Because of halos we need more bounds defined: + +| Variable | Type | Description | +|---------|------|-------------| +| jsta_m | integer | Beginning latitude loop index in subdomain for halo depth 1 | +| jend_m | integer | ending latitude loop index in subdomain for halo depth 1 | +| jsta_m2 | integer | second latitude below begin latitude of subdomain for halo depth 2 (in NGMFLD.f) | +| jend_m2 | integer | second latitude above end latitude of subdomain for halo depth 2 ( in NGMFLD.f) | + +Note: + +| Variable | Type | Description | +|---------|------|-------------| +| ista_m | integer | begining longitude loop index in subdomain for halo depth 1| +| iend_m | integer | end longitude loop index in subdomain for halo depth 1 | +| ista_m2 | integer | second longitude before begin longitude for halo depth 2 (not used as of 6/22)| +| iend_m2 | integer | second longitude after end longitude for halo depth 2 (not used as of 6/22) | + +Note: + + +| Variable | Type | Description | +|---------|------|-------------| +| ileft | integer | MPI rank containing the last longitude before ista +| iright | integer | MPI rank containing the first longitude after iend +| iup | integer | MPI rank containing the first latitude after jend +| idn | integer | MPI rank containing the last latitude before jsta +| | | | +| ileftb | integer |MPI rank containing the last longitude before ista but for cyclic boundary conditions where "last" at the beginning is the other end of the domain (apparently unused and replaced with local calculation) | +| irightb | integer | MPI rank containing the first longitude after iend but for cyclic boundary conditions where "first" at the beginning is the other end of the domain (apparently unused and replaced with local calculation) | diff --git a/sorc/ncep_post.fd/MPI_FIRST.f b/sorc/ncep_post.fd/MPI_FIRST.f index 1b399877c..37f21e48c 100644 --- a/sorc/ncep_post.fd/MPI_FIRST.f +++ b/sorc/ncep_post.fd/MPI_FIRST.f @@ -130,14 +130,17 @@ SUBROUTINE MPI_FIRST() ! global loop ranges ! -! para_range2 supports a 2D decomposition. The rest of the post -! supports 1D still and the call here is the special case where each -! processor gets all of the longitudes in the latitude 1D subdomain -! jsta:jend. The X decomposition will be specified by the third -! argument (currently 1) and the Y decoposition will be specified by -! the fourth argument (currently all of the ranks) When X is -! subdivided the third and fourth arguments will have to be integral -! factors of num_procs +! para_range2 supports a 2D decomposition. +! The X decomposition is specified by the third +! argument and the Y decoposition is specified by +! the fourth argument. The product of the third and fourth arguments +! must be num_procs and the third and fourth arguments must be integral +! factors of num_procs. +! +! for the special case of 1D decomposition, numx is set to 1 and the +! fourth argument becomes the number of MPI ranks for the job. numx=1 +! makes the code fully compatible with the old 1D decomposition. + call para_range2(im,jm,numx,num_procs/numx,me,ista,iend,jsta,jend)