Detection of known CNS errors when failed run

[VGPReys]

• You have sticked to Python. Please talk to us before adding other programming languages to HADDOCK3
• Your PR is about CNS
• Your code is well documented: proper docstrings and explanatory comments for those tricky parts
• You structured the code into small functions as much as possible. You can use classes if there is a (state) purpose
• Your code follows our coding style
• You wrote tests for the new code
• tox tests pass. Run tox command inside the repository folder
• -test.cfg examples execute without errors. Inside examples/ run python run_tests.py -b
• PR does not add any dependencies, unless permission granted by the HADDOCK team
• PR does not break licensing
• Your PR is about writing documentation for already existing code 🔥
• Your PR is about writing tests for already existing code

---

This PR add a Known CNS error detection machinery triggered when tolerance threshold is not met (basically when the run will stop), which is often the case when the user will provide some weird input files.
In the module output directory, *.cnserr / *.cnserr.gz files are read and searched for known errors.

The new script src/haddock/gear/known_cns_errors.py holds the check functions and the list of known errors.
Can be simply updated by newly discovered error types by adding the string to search as key and the hint message to be returned to the user as values.

The CNSJob class has been updated to search for those errors at runtime, when STDOUT is still in memory.
If an error is detected, dump a .cnserr file that is later compressed, containing the error.
CNS modules have been modified to send the path to the potential xxxx.cnserr file, so we know where to write it.

Also has uni and integration tests to check that it is properly functional.

Closes #1012

[amjjbonvin]

This will add additional I/O / CPU requirements... May-be only worth to do it if an error is detected and not systematically for all out.gz files

[VGPReys]

Yes, already the case
Reading the *.out or .out.gz is only triggered when the tolerance threshold is not met and the run is about to fail.

[amjjbonvin]

Did you test in batch modo to check that the .err files generated by slurm are not interfering with this. Those should be cleaned by the machinery ideally (if not empty)

[VGPReys]

Did you test in batch modo to check that the .err files generated by slurm are not interfering with this. Those should be cleaned by the machinery ideally (if not empty)

Good point.
Did not try.
Should I modify the extension of CNS related .err files to .cnserr (or .cns.err) ?

[amjjbonvin]

Did you test in batch modo to check that the .err files generated by slurm are not interfering with this. Those should be cleaned by the machinery ideally (if not empty)

Good point. Did not try. Should I modify the extension of CNS related .err files to .cnserr (or .cns.err) ?

Maybe indeed .cnserr - then the .err files (if generated) can be cleaned.

[rvhonorato]

Much better! Just some comments