Ci update, bugfixes #68
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Travis fixes (sandialabs#55) Fix some travis/testing issues. Travis now pulls from ULFM master branch when it needs to rebuild ULFM. Travis has an environment variable enabling oversubscription during the tests, instead of having that on all platforms when running make test Tests that involve failure have their timeouts individually set to 1, so tests don't take 10+ seconds each w/ the default timeout of 10s Simplified travis scripts (no more .travis_helpers directory) * Revert "Travis fixes (sandialabs#55)" (sandialabs#56) Reverting un-reviewer PR, it was meant to be in my fork. This reverts commit a41fd3b. * Update README.md * Merge updates for HCLIB (sandialabs#57) * Add ability to query which processes failed * Add support for MPI_Test * Add support for testing pre-failure requests * Fix bug when ERR_PROC_FAILED/ERR_REVOKED discovered in MPI_Test * Fix MPI_Wait w/ cancelled requests * Add missing file to commit * Fix bug with MPI_STATUS_IGNORE * Fix another bug with MPI_Test * Add no-jump recovery option * Travis fixes (#2) Fix some travis/testing issues. Travis now pulls from ULFM master branch when it needs to rebuild ULFM. Travis has an environment variable enabling oversubscription during the tests, instead of having that on all platforms when running make test Tests that involve failure have their timeouts individually set to 1, so tests don't take 10+ seconds each w/ the default timeout of 10s Simplified travis scripts (no more .travis_helpers directory) * First pass at removing the request store New function, "Fenix_test_cancelled" for checking if pre-failure requests completed or were cancelled. One thing to try finding a solution for: If a failure was found during an MPI_Test, that request has already been removed from MPI internals and replaced w/ MPI_REQUEST_NULL. Fenix_test_cancelled will report that this req was completed * Implement custom errhandler This includes removing the option for comm_replace - users now must provide a comm pointer to fenix_init and cannot rely on fenix to automatically replace their input comm with the resilient comm. * Fenix comms are stack-allocated now, instead of malloced * Cleanup redundant set_errhandler calls * Fix data recovery bug * Add usage instructions to all examples/tests * Add support for MPI_Issend and MPI_Ssend (#3) Merge in Issend test Co-authored-by: mwhitlo@sandia.gov <mwhitlo@sandia.gov> Co-authored-by: sriraj <srirajpaul@gmail.com> Co-authored-by: Keita Teranishi <knteran@sandia.gov> Co-authored-by: mwhitlo@sandia.gov <mwhitlo@sandia.gov> Co-authored-by: sriraj <srirajpaul@gmail.com>
… epizon-project-master
Mostly related to communicator state management
Before, unfortunately placed errors could "overwrite" the reporting info of prior errors without that info ever making it to the user. Now, we guarantee that info at least makes it to the user's first recovery callback. IE, users will guaranteed see a role of FENIX_ROLE_INITIAL_RANK or FENIX_ROLE_RECOVERED_RANK for a process prior to seeing FENIX_ROLE_SURVIVOR_RANK.
MPI_Datatype is vendor-dependent and we aren't allowed to assume anything about it. Right now, ompi implements as a pointer and we segfault on recovery sometimes. Fix unran test, add timeout parameter Remove ompi version expected to fail from tests
…unimplemented feature, remove travis, cmake variable naming conventions
…_update Conflicts: CMakeLists.txt
Conflicts: README.md examples/01_hello_world/fenix/CMakeLists.txt examples/02_send_recv/fenix/CMakeLists.txt examples/05_subset_create/CMakeLists.txt examples/06_subset_createv/CMakeLists.txt src/fenix_data_recovery.c src/fenix_process_recovery.c test/failed_spares/CMakeLists.txt test/issend/CMakeLists.txt test/no_jump/CMakeLists.txt test/request_cancelled/CMakeLists.txt
|
Tests are run here, until this is merged to enable testing in this repo: https://github.com/Matthew-Whitlock/Fenix/actions/runs/4725316245/jobs/8383578049 Some tests "failed" due to timeout, but that's just a bug with ULFM where MPI_Finalize hangs occasionally. There's also an interesting segfault that I'm not able to reproduce during one of the tests. It's not something to handle in this PR though (potentially some issue related to ULFM). |
nmm0
requested changes
May 1, 2023
nmm0
approved these changes
Dec 12, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Improved several CMake semantics:
Merging in some bug fixes for various recovery modes (ideally a separate pr, but here we are)
Swap from Travis testing to Github actions