KokkosKernels: Performance Testing Prototype #38
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
KokkosKernels test, added forgotten include
1- Added compilation and test of sparse, graph, common unit tests. 2- Added compilation of perf_test executables. There are no tests associated with them.
This is in Impl directory, and previously defined in one of the blas headers that was removed. Tpetra multivector seems to be depending on this Impl namespace function. This is a temporary fix that brings back the functor to KokkosKernels::Impl namespace.
MicheldeMessieres
added a commit
to Tech-XCorp/kokkos-kernels
that referenced
this pull request
Aug 9, 2017
This update applies the performance archiver to the KokkosSparse_spmv.cpp test. In the current form the archiver closely follows the original implementation and archives to a yaml file the values which are printed. This is in progress and intended as a preliminary demo of how to apply the archiver to a real test. Turning on the archiver currently requires: Building yaml-cpp Setting KOKKOSKERNELS_ENABLE_TPLS="yaml" Setting YAML_LIBS AND YAML_PATH Then KokkosSparse_spmv.exe will archive to a file called KokkosSparse_spmv.yaml in the same folder and will enter tests results for different machine configurations based on the test setting. On the first run the yaml archive is created. Subsequent runs will compare values to the original entry. Note that right now the variance in times is high and will exceed the tolerance triggering the archive to indicate a non-passing event.
This should solve issue kokkos#32.
This is a first implementation of a performance archiver based on
Teuchos_XMLPerfTestArchive but using yaml instead of XML.
Summary:
On first run, the Performance archiver adds the test info to a yaml database.
On subsequent runs the test configuration is compared to prior results.
If the configuration does not match a new entry is added to the database.
If the configuration matches, the values are compared and failure events are
provided if they exceed the specified tolerances.
Unit Tests/Demos (enabled with TPL_ENABLE_yaml-cpp):
performance_demo - a lightweight example which uses mock values and
demonstrates usage of the Performance class. Eventually to be deleted.
performance_validate - a true unit test which runs through all the possible
functions of the archiver and verifies proper response. The test is designed
to mimic running one test repeatedly and validates that new input is properly
compared to old values in the database. This test may serve as a good
guide for learning which features are currently available with the archiver.
performance_demo_blas1 - takes the blas1 example and applies the
Performance archiver to it. Currently uses the Teuchos TimeMonitor outputs
as inputs for the archiver. This test will be deleted and the performance class
can be applied directly to the blas1 test once further design discussion has
taken place. This was intended as a minimal application to a real test.
This is work in progress and needs further discussion.
This is a minor update which adds an additional example line to clarify how machine configuration would be set. Eventually it's anticipated we'll do something like this for different modes such as Cuda, threads, etc. Also hooked up the actual comm size to make the demos a bit more real.
Removed the blas1 demo as realized it was probably not a good basis for the first demos since it's the only one with Teuchos. Removed the Teuchos links in the other demos to emphasize the system is Teuchos free. Next step will be to pick another test and apply the archiver to that, get it working with different Kokkos modes, etc.
…kos-kernels into kokkosPerformance
This update applies the performance archiver to the KokkosSparse_spmv.cpp test. In the current form the archiver closely follows the original implementation and archives to a yaml file the values which are printed. This is in progress and intended as a preliminary demo of how to apply the archiver to a real test. Turning on the archiver currently requires: Building yaml-cpp Setting KOKKOSKERNELS_ENABLE_TPLS="yaml" Setting YAML_LIBS AND YAML_PATH Then KokkosSparse_spmv.exe will archive to a file called KokkosSparse_spmv.yaml in the same folder and will enter tests results for different machine configurations based on the test setting. On the first run the yaml archive is created. Subsequent runs will compare values to the original entry. Note that right now the variance in times is high and will exceed the tolerance triggering the archive to indicate a non-passing event.
|
Hm the diff seems weirdly off, it shows me stuff which is already in develop ?? |
|
@crtrott Regret the mess. Was trying to push from a different server and got mangled. I will clean this up soon. |
Closed
|
OK closing this. Check out #44 , I took your branch merged it into develop and summited as a new one. It just felled off that there are supposed to be over 100k lines changed for this, and the fixed up pull request only has 1.3k lines change, which makes more sense. |
|
Still funny how this could happen. Somehow git got confused, because a simple merge from your branch worked perfectly fine, so its not the actual content .. |
|
@crtrott I'll work off the new branch. I had forgotten to set authorship originally and amended before pushing which perhaps caused some issue, though the diff seemed fine. Now it looks like it's supposed to. Thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@srajama1
This is a first implementation of a performance archiver based on
Teuchos_XMLPerfTestArchive but using yaml instead of XML.
Summary:
On first run, the Performance archiver adds the test info to a yaml database.
On subsequent runs the test configuration is compared to prior results.
If the configuration does not match a new entry is added to the database.
If the configuration matches, the values are compared and failure events are
provided if they exceed the specified tolerances.
Unit Tests/Demos (enabled with TPL_ENABLE_yaml-cpp):
performance_demo - a lightweight example which uses mock values and
demonstrates usage of the Performance class. Eventually to be deleted.
performance_validate - a true unit test which runs through all the possible
functions of the archiver and verifies proper response. The test is designed
to mimic running one test repeatedly and validates that new input is properly
compared to old values in the database. This test may serve as a good
guide for learning which features are currently available with the archiver.
performance_demo_blas1 - takes the blas1 example and applies the
Performance archiver to it. Currently uses the Teuchos TimeMonitor outputs
as inputs for the archiver. This test will be deleted and the performance class
can be applied directly to the blas1 test once further design discussion has
taken place. This was intended as a minimal application to a real test.
This is work in progress and needs further discussion.