Tpetra: improve performance of TAFC (Transfer and fillComplete)

[jhux2]

Enhancement

@trilinos/tpetra

TAFC currently is a host-based operation. This makes it potentially very expensive, for example, in a matrix product such as $R*(AP)$ in multigrid setup. Within Tpetra::CrsMatrix::transferAndFillComplete(), I've measured the tall poles to be

• unpack, count, and resize (phase 3, timer label "TAFC unpack-count-resize")
  Tpetra: improve performance of unpackAndCombineWithOwningPIDsCount #11694
• Copying same/permute/remote data into CSR arrays (phase 3, timer label "TAFC copy same-perm-remote data") Tpetra: improve performance of unpackAndCombineIntoCrsArrays #11693
• Making the column map (phase 4, timer label "TAFC makeColMap")
  Tpetra: improve performance of lowCommunicationMakeColMapAndReindex #11695
• sortAndMergeCrsEntries (phase 5, timer label "TAFC sortAndMergeCrsEntries")
  Tpetra: improve performance of sortAndMergeCrsEntries #11696
• remove ArrayRCP usage to create local data ahead of setAllValues() in TAFC
  • first PR to call setAll Values on Kokkos::View instead of ArrayRCP, also remove copies to host after sort and merge of CSR views Tpetra: remove unnecessary copy to host in TAFC #12262

[jhux2]

Sample timer data:

|   |   |   |   |   |   |   TpetraExt MueLu::R*(AP)-implicit-1: MMM-T exportAndFillComplete: 0.451469 - 85.9368% [1] {min=0.450672, max=0.452267, std dev=0.00112805} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC All:MMLegacy: 0.451126 - 99.9239% [1] {min=0.450328, max=0.451924, std dev=0.00112886} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC getOwningPIDs same map:               0.00026664 -  0.0591054% [1] {min=0.000259792, max=0.000273488, std dev=9.68453e-06} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC reallocate buffers:                   7.8373e-05 -  0.0173727% [1] {min=7.1575e-05, max=8.5171e-05, std dev=9.61382e-06} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC pack and prepare:                     0.00181816 -  0.403027% [1] {min=0.00172944, max=0.00190688, std dev=0.000125466} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC getOwningPIDs exchange remote data:   0.00127957 -  0.28364% [1] {min=0.00117301, max=0.00138613, std dev=0.000150701} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC unpack-count-resize:                  0.0733156  - 16.2517% [1] {min=0.0728005, max=0.0738308, std dev=0.000728582} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC copy same-perm-remote data:           0.109153   - 24.1956% [1] {min=0.109, max=0.109305, std dev=0.000215957} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC makeColMap:                           0.079964   - 17.7254% [1] {min=0.079285, max=0.080643, std dev=0.000960279} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC restrict colmap:                      2.2995e-06 -  0.000509724% [1] {min=2.074e-06, max=2.525e-06, std dev=3.18905e-07} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC sortAndMergeCrsEntries:               0.148774   - 32.9783% [1] {min=0.147311, max=0.150236, std dev=0.00206834} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC setAllValues:                         0.0200524  -  4.44497% [1] {min=0.0198734, max=0.0202314, std dev=0.000253164} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>
|   |   |   |   |   |   |   |   |   Tpetra MueLu::R*(AP)-implicit-1: TAFC build importer and esfc:              0.00323561 -  0.717229% [1] {min=0.00159667, max=0.00487454, std dev=0.0023178} <1, 0, 0, 0, 0, 0, 0, 0, 0, 1>

[jhux2]

@lucbv

[cgcgcg]

Why is it running the Legacy code path? There is an MMOpt, if I remember correctly.

[jhux2]

My understanding is that MMOpt comes into play after the various phases above.

See

Trilinos/packages/tpetra/core/src/Tpetra_CrsMatrix_def.hpp

Line 7710 in 99e0860

bool isMM = false; // optimize for matrix-matrix ops.

Trilinos/packages/tpetra/core/src/Tpetra_CrsMatrix_def.hpp

Lines 7756 to 7757 in 99e0860

	if(isMM) os<<":MMOpt";
	else os<<":MMLegacy";

Trilinos/packages/tpetra/core/src/Tpetra_CrsMatrix_def.hpp

Line 8639 in 99e0860

if( isMM ) {

[cgcgcg]

Looking at the code, that seems to be correct.

[jhux2]

closing as complete