Add a processing pool for the modifiers

[jelmervdl]

Attempt to work around #33

• Modified the modifier interface to take in batches so they can benefit from batch processing. None of the modifiers has this implemented yet though.
• Run the modifier chain in multiple subprocesses in parallel (defaults to max(min(cpu_count,8),1), chunks of 16, which makes about sense with the default batch size of 100.
• Changing --batch-size or --chunk-size changes the output. Changing --workers does not.

Todo:

• Actually figure out how to implement a timing regression test in Github Actions

[jelmervdl]

There still seems to be one source of indeterministic behaviour somewhere , for me test_full_enzh and test_prefix_augment somewhat consistently seem to fail.

[jelmervdl]

Note to self: somehow the chunks in the pool seem to get mixed. In the end-to-end test exactly 13 lines got shuffled around even when running with --no-shuffle.

[XapaJIaMnu]

LGTM

[XapaJIaMnu]

Do we expect this to fail now? Or is the exception here added because exceptions no longer bubble up?

[jelmervdl]

Previously the exception was caught at the Trainer level when it applied the modifier line for line. The Trainer then logged it and skipped that line entirely.

Since we're now submitting batches to modifiers, I've changed these non-critical errors to do the same thing: just skip the line and log about it. Otherwise we'd skip the entire batch (or chunk that was assigned to a specific worker)

I've made sure that logging and raising exceptions still works inside the ModifierWorker processes.

[jelmervdl]

So long story short: this modification keeps the behaviour as it is now on the main branch.

[graemenail]

LGTM

(was going to ask a stupid question about determinism and if they're workers complete out of order, but then saw you covered that)