Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue272 + Issue265 mpi and main in separate process #275

Merged
merged 34 commits into from
Jul 11, 2019

Conversation

mikebentley15
Copy link
Collaborator

Fixes #265 and fixes #272

Description:
Allow calling a main()-like function from a FLiT test. Also, be able to call a main()-like function that uses MPI from a FLiT test.

The main()-like functionality is the following:

  • Done with two components:
    1. REGISTER_FLIT_MAIN() that takes the function pointer, and
    2. call_main() that takes the function pointer and command-line arguments
  • The main()-like function is called from a child process (that is the same executable, but with special flags)

The MPI functionality is the following:

  • Remove all MPI functionality from before (except for the compiling with MPI)
  • When FLiT tests are called, they are not under MPI
  • Instead, the user calls call_mpi_main() from the test where they specify everything the call_main() function requires plus an mpirun command with flags.
  • The new subprocess(es) is(are) now launched with the user's mpirun command and everything just works.
  • The downside is that tests using MPI must use a main()-like function. There is no other way. I think this is a reasonable downside, and worth the exxtra stability our current approach affords
  • Add utility functions in src/subprocess.cpp

Documentation:

  • Remove documentation to removed MPI functionality
  • Add many sections to "Writing Test Cases" page describing how to write a test that wraps around a main() function and how to write MPI tests with examples.

Tests:

  • Update the MPI tests
  • Test everything in src/subprocess.cpp
  • Add unit tests for call_main()
  • Add tests for the TestFactory class
  • Add tests for the registerTest() function used by the REGISTER_TEST() macro

mikebentley15 and others added 20 commits June 12, 2019 00:21
One function added: call_with_output()
Need to do FLIT_REGISTER_MAIN() first, but it is all implemented,
and tested!
- Debugging with TravisCI
- Fixed tst_subprocess.  sh does not output the same error string for all versions
This strips out most of the infrastructure from before of how MPI worked.
Now we will onlly support MPI with a main()-like function in which you
are expected to call MPI_Init() and MPI_Finalize().
This demonstrates how it can be done to run an MPI test with float,
double, and long double without having to write three different main
functions.  Not easy...
To do this, functions which() and realpath() were implemented in fsutil.
This should allow for the user to still call call_main() and
call_mpi_main() after changing to a different directory.

Also, documentation was added for call_main() and call_mpi_main().
allow it to be called more than once with the same args
@mikebentley15 mikebentley15 requested a review from IanBriggs July 8, 2019 22:48
@mikebentley15 mikebentley15 added c++ Involves touching c++ code documentation Involves touching documentation make Involves touching GNU Makefiles python Involves touching python code tests Involves touching tests labels Jul 8, 2019
Bug: if the test file itself is found to be variable, then the symbol bisect
will fail because of duplicate flit test registrations.  This is because both
object files will contribute their static objects to the executable and
registerTest() will be called twice.
This reduces test runtime from 51 seconds to 38 seconds
@IanBriggs IanBriggs merged commit 9ed68de into devel Jul 11, 2019
@IanBriggs IanBriggs deleted the issue272-mpi-in-separate-process branch July 11, 2019 16:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c++ Involves touching c++ code documentation Involves touching documentation make Involves touching GNU Makefiles python Involves touching python code tests Involves touching tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

MPI to be run from child processes Tests wrapping main to be run in a separate process
2 participants