Home

Welcome to the tltesting wiki!

Ideas by Norbert

reuse, don't reprogram - reuse l3build (http://tug.org/TUGboat/tb35-3/tb111mitt-l3build.pdf, http://mirrors.ctan.org/macros/latex/contrib/l3build/l3build.pdf)
develop default tests for types of packages:
- latex packages are simply loaded into a otherwise empty document
- plain tex packages in the same way
- context (no idea)
provide a way to allow authors to submit more involved tests
possible generation of dependencies via -fls recorder (hard dependency for plain loading)
check what Karl did in ./Build/tests

must run on at least Mac OS, Windows and Linux, more platforms are appreciated
shall not require manual checks
shall be self contained
shall test a majority of user scenarios (letters, reports, books, etc.)
many small tests may be helpful, bigger test scenarios must not be omitted (I, for example, ran a 600 pages dissertation as part of my testing)
cannot test each and every scenario users may come across

use python (as I know it best)
make use of its unit test facilities
run a testcase by calling TeX engine n-times on specific file
check if the process ended properly
proper process ending does not mean that file is correct
Gather statistics on file: Size, number of pages, ...
Convert generated file to image and run imagemagick to collect statistical data.
Compare statistical data to known result. If threshold > level ==> testcase failed.