Validate input data

[fengelniederhammer]

When doing SILO preprocessing, validate the input data:

• Validate proper lineage tree and error appropriately #583
• Validate that primary keys in metadata are unique and error appropriately #584
• Validate data types of each metadata field #585
• Validate that Pango lineages have their alias defined and error appropriately #586
• Validate primary key exists in metadata, is unique and every metadata primary key exists in sequences #587
• Validate aligned sequences length #588
• Validate aligned sequences length #588
• Check for duplicate keys in fasta files #551

• configuration files

Add tests that check the behavior.

[JonasKellerer]

"primary key exists in metadata, is unique and every metadata primary key exists in sequences." exists already

[fengelniederhammer]

Mind #215 (comment) and improve log messages

[fengelniederhammer]

Try to reproduce #361 and fix it.

[Taepper]

Also: Unit tests that execute the preprocessing in both paths

[corneliusroemer]

Motivated by the bug hunt related to loculus-project/loculus#2667

Extra validation ideas:

• run a self test after import, running a bunch of queries/assertions
  • if downloading all sequences, get correct number of sequences
  • if downloading all sequences, don't get duplicate sequence ids
  • ...

[Taepper]

After #458 a lot of this should already be implemented.

We will add separate issues to verify and that check that proper error messages are returned accordingly.