fsttest

Test your Foma FSTs!

Install

For macOS users with Homebrew:

$ brew install eddieantonio/eddieantonio/fsttest

For everybody else:

$ pip install fsttest

Usage

Write test_*.toml scripts in a folder called tests/, then run:

$ fsttest

Documentation

• How to specify the FST under test

Tutorial

In your FST project, create a folder called tests/:

$ mkdir tests

Within this folder create a file called test_{something}.toml where {something} is something specific to your FST. For example, test_phonology.toml:

$ touch tests/test_phonology.toml

Use the following template to create your test:

# tests/test_phonology.toml
[fst]
eval = "rewrite_rules.xfscript"
compose = ["TInsertion", "NiTDeletion", "Cleanup"]

[[tests]]
upper = "ni<ayaa<n"
expect = "dayaan"

[[tests]]
lower = "ki<tayaa<n"
expect = "kiayaan"

Then run the test!

$ fsttest
1/1 tests passed! ✨ 🍰 ✨

Line-by-line

Let's breakdown this file, line-by-line.

[fst]

This defines the FST under test — that is, the FST we want to use to transduce and test its output.

eval = "rewrite_rules.xfscript"

This says that our FST under test can be created by running rewrite_rules.xfscript in Foma, creating an FST. As an example, let's take the following rewrite_rules.xfscript:

# rewrite_rules.xfscript
define Vowel    a | e | i | o | u ;

define TInsertion [..] -> t || [n i | k i] "<" _ Vowel ;
define NiTDeletion n i "<" t -> d || _ Vowel ;
define Cleanup %< -> 0 ;

Yours will be different!

compose = ["TInsertion", "NiTDeletion"]

This line says that the FST under test is the result of composing the TInsertion regex with the NiTDeletion regex. That is, the FST puts its input into TInsertion and then passes the result to NiTDeletion. The result of passing it through both FSTs is the result we want to test.

[[tests]]

Next we define one or more test cases. Begin every test case with [[tests]]: note the two square brackets!

upper = "ni<ayaa<n"

This test case feeds the string ni<ayaa<n into the upper side of the FST. The upper side is conventionally the analysis side of the FST.

expect = "dayaan"

This says that we expect the lower side to be dayaa<n. That is, this test case says that, given the analysis ni<ayaa<n, the FST should produce dayaa<n among the possible surface forms.

[[tests]]

Next, we define another test case.

lower = "kitayaan"

In contrast to the previous test, we feed the input to the lower side of the FST. In other words, we want to do a lookup. Conventionally, this means we're providing a surface form, and asking the FST to return an analysis.

expect = "ki<ayaa<n"

This means we're expecting the analysis of ki<tayaa<n when we give the FST the wordform of kitayaan

License

Copyright © 2019, 2020 National Research Council Canada.

Licensed under the MIT license.