Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Begin porting matmul test suite from iree-org/iree. #13

Merged
merged 11 commits into from
Aug 20, 2024

Conversation

ScottTodd
Copy link
Member

@ScottTodd ScottTodd commented Aug 16, 2024

Progress on #2.

This begins to port the tests from https://github.com/iree-org/iree/tree/main/tests/e2e/matmul. The changes here aim to be minimal, but a lot of cleanup is needed to make this sustainable.

  • A new CMake project named iree-test-suites-matmul is added. We could share this with other test suites that need CMake, or let each test suite create its own. I don't want to overgeneralize yet while so much is still up in the air.
  • The CMake project depends on the IREE runtime libraries and header files to build a iree-e2e-matmul-test binary. This binary handles preparing inputs, running a compiled program, and comparing outputs against a reference implementation. That may be split into multiple libraries or parts of it may move into Python or another test harness. TBD.
  • The CMake project then defines some test cases using iree_generated_e2e_runner_test. This may be split into an offline generation step and a test runner other than ctest. TBD.

TODO:

  • Fix namespace/symbol collisions in test_utils.h (fork further) and include it correctly
  • Fix namespace/symbol collisions in iree_e2e_generated_runner_test.cmake (fork further)
  • Add a GitHub Actions workflow running the tests
  • Structure the CMake project such that it can be used from iree-org/iree in place of the current tests
  • Fork or remove dependencies on the upstream CMake helpers used by matmul/iree_e2e_generated_runner_test.cmake?
  • Allow pointing at a local IREE, like https://github.com/ScottTodd/iree/tree/infra-e2e-test-suites which deletes the in-tree tests and supporting infra

Copy link

@erman-gurses erman-gurses left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Scott, LGTM!

@ScottTodd ScottTodd requested a review from marbre August 20, 2024 16:32
@ScottTodd ScottTodd merged commit a2cfb01 into iree-org:main Aug 20, 2024
1 check passed
@ScottTodd ScottTodd deleted the matmul-tests branch August 20, 2024 17:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants