Feature request : Support multiple rhs in SGS efficiently

[srajama1]

Feature request : Support multiple rhs in MT-SGS efficiently. Currently, it is implemented but it could be better if we pass the multivector to Kokkos Kernels, which accepts and iterates on all the vectors. Based on comment by @mhoemmen serial SGS also does the same.

Customer: Exawind

@brian-kelley : One more thing when you are at SGS.

@iyamaza found this was not implemented. @lucbv

[mhoemmen]

It's implemented, but it's slow (it just loops over columns on the outside). I'm pretty sure Ichi was talking about the not-multithreaded case.

[brian-kelley]

@mhoemmen Are you talking about inside Ifpack2 Relaxation? It looks like Relaxation::MTGaussSeidel() does a loop over vectors. There would be a significant win there from moving that loop all the way inside the KK SGS implementation, from fewer kernel launches and better locality with the matrix accesses.

[lucbv]

@brian-kelley You would probably gain time on a GPU by not having to load the matrix in memory multiple times.

[mhoemmen]

@iyamazaki Are we talking about MTGaussSeidel or the non-MT Gauss-Seidel?

[iyamazaki]

Hi, @mhoemmen. I was looking at MTGaussSeidel, which seems to loop over the vectors in Ifpack2? But, I did not look at what the KokkosKernel does.

[srajama1]

I checked the MT-SGS case and I should have been clear in the original posting. It is implemented but it not in an optimal way. I will edit the description above.

[mhoemmen]

Non-MT Gauss-Seidel has the same issue but it's separate code.

[brian-kelley]

Within KokkosKernels MT-SGS multiple RHS doesn't exist at all. Within Ifpack2 the inefficient implementation (outermost loop over vectors) exists.