Automatic validation

[blaisb]

Description

Although Lethe is covered by extensive testing through application_tests, sometimes these tests are not enough to measure changes in the models that do not break the code but yield poorer results. This PR introduces an additional way to validate Lethe. The idea is to leverage the examples of Lethe for which there are solutions to compare to and to use them to validate the sotware on complicated cases. The validation loop is meant to be ran manually and generates artifacts, logs and a PDF report that can be looked at. We aim at launching these validation cases manually every week or so and to increase the number of validation test to provide, asymptotically, additional validation and comparison to measure and track the performance of the code.

The structure is relatively simple. Examples that are used for the validation now have a validate.sh script within their folder. Furthermore, there is a list of cases i contrib/validation/validation_cases.txt

For every case, the path of the case is specified and the number of core to run it. Right now, running all of the 5 cases takes about 8 hours on my machine, but these test cases are extensive and provide a lot of good information. Once we acquire our own node, we can launch many more test cases and things are going to be much more robust. Adding a new case is very cheap and the fun part is that they also use the python script to post-process the results from the regular simulations and as such, they keep the examples in a very mature state.

Testing

I have tested it on my machine extensively while developing it. This is not an intrusive change, it does not alter the code.

Documentation

I had to use the argparse module pretty much everywhere in the python script to enable the dump of the data files. This has forced me to modify slightly some python scripts and improve them. I changed the documentation accordingly.

Code related list:

• All in-code documentation related to this PR is up to date (Doxygen format)
• Copyright headers are present and up to date
• Lethe documentation is up to date
• New feature has unit test(s) (preferred) or application test(s), and restart files are in the generator folder
• The branch is rebased onto master
• Changelog (CHANGELOG.md) is up to date
• Code is indented with indent-all and .prm files (examples and tests) with prm-indent

Pull request related list:

• Labels are applied
• There are at least 2 reviewers (or 1 if small feature) excluding the responsible for the merge
• If this PR closes an issue or is related to a project, it is linked in the "Projects" or "Development" section
• If the fix is temporary, an issue is opened
• The PR description is cleaned and ready for merge

[OresteMarquis]

Nice :) I really like this, it will be of so much help!

[OresteMarquis]

I finally managed to install magick with all the dependencies on my computer. Everything seems to work, but it is still running.

[OresteMarquis]

Everything worked on my computer! It is ready to merge in my opinion.