Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor runtests.jl to use a single run_tests function #3168

Merged
merged 5 commits into from
Jan 4, 2023

Conversation

odow
Copy link
Member

@odow odow commented Jan 3, 2023

This PR continues the refactoring I have been doing of JuMP's tests.

  • Tests are now contained in a module which lives in test/test_xxx.jl file
  • Within a module, tests are functions beginning with test_
  • JuMPExtensions are tested by functions with the prefix test_extension_ and take two optional positional argumennts for the model type and variable type
  • Test individual files via the command line `julia runtests.jl [<file> [<file> ...]]`

This design is flexible, and separates the runninng of the tests from their definition. This should let us swap in other drivers instead of the one currently in runtests.jl.

For example, we might want to experiment with:

  • Running all test functions within a module in parallel
  • Running each test file in parallel
  • Providing a more flexible command line interface

If we get something nice, we could package it into Pukeko.jl for wider use in the Julia ecosystem.

Relatedly, this closes #1711, because it should enable external packages to run:

import JuMP
include(joinpath(dirname(dirname(pathof(JuMP))), "test", "Kokako.jl"))
const MODULES_TO_TEST = Kokako.include_modules_to_test(JuMP)
Kokako.run_tests(
    MODULES_TO_TEST,
    JuMPExtension.MyModel,
    JuMPExtension.MyVariableRef;
    test_prefix = "test_extension_",
)

@codecov
Copy link

codecov bot commented Jan 3, 2023

Codecov Report

Base: 97.70% // Head: 97.70% // No change to project coverage 👍

Coverage data is based on head (0b3fc3c) compared to base (0b3fc3c).
Patch has no changes to coverable lines.

❗ Current head 0b3fc3c differs from pull request most recent head 3fd9f04. Consider uploading reports for the commit 3fd9f04 to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #3168   +/-   ##
=======================================
  Coverage   97.70%   97.70%           
=======================================
  Files          33       33           
  Lines        4485     4485           
=======================================
  Hits         4382     4382           
  Misses        103      103           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@odow odow changed the title Refactor runtests.jl to use a single run_tests function WIP: Refactor runtests.jl to use a single run_tests function Jan 3, 2023
@odow odow force-pushed the od/test-refactor branch 5 times, most recently from ef49bb1 to 257fad6 Compare January 4, 2023 03:41
@odow odow changed the title WIP: Refactor runtests.jl to use a single run_tests function Refactor runtests.jl to use a single run_tests function Jan 4, 2023
@odow
Copy link
Member Author

odow commented Jan 4, 2023

Okay. I like where this is at. We have a decision on where the Kokako.jl file should live.

Pukeko.jl is a registered package that I was shamelessly inspired by.

Using the Base.Test macros makes things a lot simpler, which is one point in favor of this system. Perhaps I'll change the license to MIT for that file so that we can leave it here for now, and anyone is free to rip it off later. The functions are trivial.

This PR continues the refactoring I have been doing of JuMP's tests.

 * Tests are now contained in a module which lives in test/test_xxx.jl
   file
 * Within a module, tests are functions beginning with test_
 * JuMPExtensions are tested by functions with the prefix test_extension_
   and take two optional positional argumennts for the model type and
   variable type
 * Test individual files via the command line \`julia runtests.jl [\<file\> [\<file\> ...]]\`

This design is flexible, and separates the runninng of the tests from their
definition. This should let us swap in other drivers instead of the one
currently in runtests.jl.

For example, we might want to experiment with:

 * Running all test functions within a module in parallel
 * Running each test file in parallel
 * Providing a more flexible command line interface

If we get something nice, we could package it into Pukeko.jl for wider use
in the Julia ecosystem.
@blegat
Copy link
Member

blegat commented Jan 4, 2023

We can leave it in JuMP to play around it for a bit but I think it would be nice as an external package since we are using this pattern in many packages now

@odow odow merged commit ed3dc0f into master Jan 4, 2023
@odow odow deleted the od/test-refactor branch January 4, 2023 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Make JuMP tests callable from an external package
2 participants