ODE: BDF methods (#1930)

* ODE: adding BDF algorithms

Implementing BDF formula for stiff ODEs.
Orders 1 to 5 are available and tested.
The integrators can be called on GPU to
solve multiple systems in parallel.

* ODE: fixing storage handling for start-up RK stack

* ODE: clang-format

* ODE: first adaptive version of BDF

The current implementation only allows for adaptivity in time,
at this point the BDF Step actually converges as expected with
first order integration!

* ODE: fixing issues with adaptive BDF

The unit-test BDF_adaptive now shows the integration
of the logistic equation using adaptive time steps and
increasing integration order from 1 to 5.

* ODE: running BDF on StiffChemistry problem

The problem runs fine and is solved but there are oscillations
while the behavior of the solution is smooth. More investigation
is needed...

* BDF: fixing types and template parameters in batched calls

Bascially we need template parameters to be more versatile
and cannot assume that all rank1 views will have the exact
same underlying type, for instance layouts can be different.

* More fixes for GPUs only in tests this time.

* ODE: BDF adaptive, fix small bug

After adding rhs and update vectors to temp the subviews taken for
other variables need to be offset appropriately...

* Revert "More fixes for GPUs only in tests this time."

This reverts commit 2f70432.

* Revert "Revert "More fixes for GPUs only in tests this time.""

This reverts commit 836012b.

* ODE: BDF small change to temporarily avoid compile time issue

True fix involving a KOKKOS_VERSION check is upcoming after more
tests on GPU side...

* ODE: BDF fix for some printf statements that will go away soon...

* ODE: adding benchmark for BDF

The benchmark helps us monitor the performance of the BDF
implementaiton across multiple platforms as well as impact of
changes over time.

* ODE: improve benchmark interface...

* ODE: BDF changes to use RMS norm and change some default values

Small changes to compare more closely with reference implementation.
Some of these might be reverted eventually but that's fine for now.

* ODE: BDF convergence more stable and results look pretty good now!

Changing the Newton solver convergence criteria as well as changing
a few default input parameters leads to a more stable algorithms
which can now integrate the stiff Henderson autocatalytic example
well in 66 time steps instead of 200k for fixed order integration...

* ODE: BDF fix bug in initial time step calculation

The initial step routine was overwriting the initial right hand side
which led to obvious issues further down the road... now things should
work fine. Need to figure out if I can re-initialize the variables in
the perf test while excluding that time from each iteration.

* ODE: BDF removing bad print statement...

std::cout in device code

* ODE - BDF: improving perf test

Basically adding new untimed setup within the main loop of the
benchmark to reset the intial conditions, buffers and vectors
ahead of each iteration.

* Modifying unit-test to catch proper return type

* Applying clang-format

batched/dense/impl/KokkosBatched_Gesv_Impl.hpp

@@ -366,20 +366,24 @@ KOKKOS_INLINE_FUNCTION void TeamVectorHadamard1D(const MemberType &member,
 /// ===========
 template <>
 struct SerialGesv<Gesv::StaticPivoting> {
-  template <typename MatrixType, typename VectorType>
+  template <typename MatrixType, typename XVectorType, typename YVectorType>
   KOKKOS_INLINE_FUNCTION static int invoke(const MatrixType A,
-                                           const VectorType X,
-                                           const VectorType Y,
+                                           const XVectorType X,
+                                           const YVectorType Y,
                                            const MatrixType tmp) {
 #if (KOKKOSKERNELS_DEBUG_LEVEL > 0)
     static_assert(Kokkos::is_view<MatrixType>::value,
                   "KokkosBatched::gesv: MatrixType is not a Kokkos::View.");
-    static_assert(Kokkos::is_view<VectorType>::value,
-                  "KokkosBatched::gesv: VectorType is not a Kokkos::View.");
+    static_assert(Kokkos::is_view<XVectorType>::value,
+                  "KokkosBatched::gesv: XVectorType is not a Kokkos::View.");
+    static_assert(Kokkos::is_view<YVectorType>::value,
+                  "KokkosBatched::gesv: YVectorType is not a Kokkos::View.");
     static_assert(MatrixType::rank == 2,
                   "KokkosBatched::gesv: MatrixType must have rank 2.");
-    static_assert(VectorType::rank == 1,
-                  "KokkosBatched::gesv: VectorType must have rank 1.");
+    static_assert(XVectorType::rank == 1,
+                  "KokkosBatched::gesv: XVectorType must have rank 1.");
+    static_assert(YVectorType::rank == 1,
+                  "KokkosBatched::gesv: YVectorType must have rank 1.");
 
     // Check compatibility of dimensions at run time.
 
@@ -462,20 +466,24 @@ struct SerialGesv<Gesv::StaticPivoting> {
 
 template <>
 struct SerialGesv<Gesv::NoPivoting> {
-  template <typename MatrixType, typename VectorType>
+  template <typename MatrixType, typename XVectorType, typename YVectorType>
   KOKKOS_INLINE_FUNCTION static int invoke(const MatrixType A,
-                                           const VectorType X,
-                                           const VectorType Y,
+                                           const XVectorType X,
+                                           const YVectorType Y,
                                            const MatrixType /*tmp*/) {
 #if (KOKKOSKERNELS_DEBUG_LEVEL > 0)
     static_assert(Kokkos::is_view<MatrixType>::value,
                   "KokkosBatched::gesv: MatrixType is not a Kokkos::View.");
-    static_assert(Kokkos::is_view<VectorType>::value,
-                  "KokkosBatched::gesv: VectorType is not a Kokkos::View.");
+    static_assert(Kokkos::is_view<XVectorType>::value,
+                  "KokkosBatched::gesv: XVectorType is not a Kokkos::View.");
+    static_assert(Kokkos::is_view<YVectorType>::value,
+                  "KokkosBatched::gesv: YVectorType is not a Kokkos::View.");
     static_assert(MatrixType::rank == 2,
                   "KokkosBatched::gesv: MatrixType must have rank 2.");
-    static_assert(VectorType::rank == 1,
-                  "KokkosBatched::gesv: VectorType must have rank 1.");
+    static_assert(XVectorType::rank == 1,
+                  "KokkosBatched::gesv: XVectorType must have rank 1.");
+    static_assert(YVectorType::rank == 1,
+                  "KokkosBatched::gesv: YVectorType must have rank 1.");
 
     // Check compatibility of dimensions at run time.
 

batched/dense/src/KokkosBatched_Gesv.hpp

@@ -63,11 +63,18 @@ struct Gesv {
 
 template <typename ArgAlgo>
 struct SerialGesv {
-  template <typename MatrixType, typename VectorType>
+  template <typename MatrixType, typename XVectorType, typename YVectorType>
   KOKKOS_INLINE_FUNCTION static int invoke(const MatrixType A,
-                                           const VectorType X,
-                                           const VectorType Y,
+                                           const XVectorType X,
+                                           const YVectorType Y,
                                            const MatrixType tmp);
+
+  template <typename MatrixType, typename VectorType>
+  [[deprecated]] KOKKOS_INLINE_FUNCTION static int invoke(
+      const MatrixType A, const VectorType X, const VectorType Y,
+      const MatrixType tmp) {
+    return invoke(A, X, Y, tmp);
+  }
 };
 
 /// \brief Team Batched GESV: