[WIP] Extending the BP4 engine to allow data transfers through SCR

[anagainaru]

Adding over PR #3294

Test for ADIOS2 with SCR for 1D, 2D, 10D, Sync and Deferred. The new API will need to initialize and finalize SCR and nothing else. Example:

#include "scr.h"

MPI_Init();
SCR_Init();

adios2::IO io = adios.DeclareIO("TestIO");
io.SetParameters({{"UseSCR", "1"}});

adios2::Engine engine = io.Open(fname, adios2::Mode::Write);

for (step=0; step<total_steps; step++)
{
  engine.Put(var, data);
}

engine.Close();

SCR_Finalize();
MPI_Finalize();

SCR works only for MPI codes. The library needs to be initialized after MPI_Init and finalized before MPI_Finalize.
If the flag is set without SCR being initialized an error will occur. If SCR is initialized before MPI was initialized an error will occur.

Currently supported only in BP4

[eisenhauer]

Do you have a FindSCR.cmake? Also, does the test actually test proper functioning of SCR? That may be what the scr_complete variable represents, but it isn't actually examined in the test proper.

[anagainaru]

Do you have a FindSCR.cmake?

Oops I forgot to add it to the first commit

Also, does the test actually test proper functioning of SCR? That may be what the scr_complete variable represents, but it isn't actually examined in the test proper.

I am actually not sure how to examine if SCR is doing anything.
Following the code Adam has in his example, I am calling SCR functions before Put and I assume all files are handled by SCR.

SCR_Start_output(fname.c_str(), SCR_FLAG_CHECKPOINT);
// Engine Open
...
// Engine Close
SCR_Complete_output(scr_valid);

All the variables between the two calls are tracked with SCR inside ADIOS2.

We might want to do some smarter things (like detect is SCR_Start_output has been called and not error in case it has not but just fall back to not using it or be able to decide for which variable we want to use SCR), but we can discuss them today.

[williamfgc]

Just minor suggestions. Looks great, once this PR is merged it would be good to add documentation (./docs) for the readthedocs user guide.

[williamfgc]

#include <scr.h> would help identify it as an external header.

[williamfgc]

Similarly here #include <scr.h> would help identify it as an external header. Thanks for adding a test, BTW.

[anagainaru]

Just minor suggestions. Looks great, once this PR is merged it would be good to add documentation (./docs) for the readthedocs user guide.

We are looking at correctness and performance for now. Once we decide to merge it, I will add documentation for it. We are not there yet.

Sure about the header. If we decide to merge it, I'll change it.