Cross implementation consistency tests

[Katzor]

As the result of a discussion at phax#148 create cross implementation consistency tests and contribute them to https://github.com/phax/ph-schematron.

A cross implementation consistency test validates a pair of (xml, schematron) files for example with schematrion-sch and schematron-pure and compares the results for equality.

schematrion-sch(xml, schematron) == schematron-pure(xml, schematron) for a certain definition of the equal operator == for comparing validation results.

[Katzor]

Todos

• Compare sch vs. pure vs. SchXslt
• Implement actual logic in src/main/java so it is usable by others
• A class that holds both results (SVRLs) with the behaviour to do comparisons

Questions

• Why is sequence of failed assertions different?
• Why is location attribute different with sch and pure?
• Why is text attribute different with sch and schxslt?

[Katzor]

ph-schematron-testfiles; external/test-sch/universalTests2.sch; external/test-xml/valid03.xml

The result of running this should be four assertion messages: U7, U8, U9, and U10 only.

schematron-pure: 8 assertion messages

1. U5: This assertion should never succeed because the rule should never fire.
2. U6: This report should never succeed because the rule should never fire.
3. U7: This assertion should always fail.
4. U8: This report should always succeed.
5. U9: This assertion should always fail.
6. U10: This report should always succeed.
7. U11: This assertion should never succeed because the rule should never fire.
8. U12: This report should never succeeed because the rule should never fire.