alntmscore output is wrong

[ekiefl]

Expected Behavior

I expect alntmscore to be different from ttmscore.

Current Behavior

All alntmscore values are equal to ttmscore.

Steps to Reproduce (for bugs)

Here is a zipped directory of 75 structures in pdb format:

structures.zip

Unzip this directory. Then perform a search and convert the alignment.

rm -rf tmp
foldseek createdb ./structures/ targetDB
foldseek search targetDB targetDB alnDB tmp -a --exhaustive-search
foldseek convertalis targetDB targetDB alnDB output.txt --format-mode 4 --format-output query,target,qlen,tlen,alnlen,qtmscore,ttmscore,alntmscore,cigar,qseq,tseq

In a python environment with pandas, confirm that all alntmscore equal all ttmscore and sample 20 hits to be printed:

import pandas as pd

df = pd.read_csv("output.txt", sep="\t")

# This assert passes, meaning every ttmscore equals every alntmscore
assert (df.ttmscore == df.alntmscore).all()

print(
    df[["query", "target", "qlen", "tlen", "alnlen", "qtmscore", "ttmscore", "alntmscore"]]
    .sample(20)
    .tail(10)
    .to_markdown()
)

Output of script:

	query	target	qlen	tlen	alnlen	qtmscore	ttmscore	alntmscore
1187	B0RXV1	V7BU96	227	563	206	0.5245	0.2354	0.2354
5406	U5U2L0	V4KAC2	550	599	560	0.6095	0.5638	0.5638
3335	Q5F9Z5	Q9XZT6	206	250	234	0.6303	0.5362	0.5362
861	B0BUU8	Q20230	203	191	210	0.5333	0.5591	0.5591
842	B0BUU8	B0RXV1	203	227	199	0.8489	0.7633	0.7633
2810	K0F1X4	B1JTS0	178	206	211	0.6423	0.5692	0.5692
2927	L8IGY7	P48769	256	260	254	0.9265	0.9125	0.9125
4925	Q8R9S6	Q834T6	206	226	245	0.4625	0.4305	0.4305
3861	W5N438	B8F7G0	264	208	255	0.5412	0.6642	0.6642
960	B0K119	C5A558	203	190	218	0.5143	0.5417	0.5417

Foldseek Output (for bugs)

Full Foldseek standard out:

targetDB exists and will be overwritten
createdb ./structures/ targetDB

MMseqs Version:        	9.427df8a
Path to ProstT5
Chain name mode        	0
Write mapping file     	0
Mask b-factor threshold	0
Coord store mode       	2
Write lookup file      	1
Input format           	0
File Inclusion Regex   	.*
File Exclusion Regex   	^$
Threads                	14
Verbosity              	3

Output file: targetDB
[=================================================================] 100.00% 75 0s 11ms
Time for merging to targetDB_ss: 0h 0m 0s 1ms
Time for merging to targetDB_h: 0h 0m 0s 1ms
Time for merging to targetDB_ca: 0h 0m 0s 1ms
Time for merging to targetDB: 0h 0m 0s 1ms
Ignore 0 out of 75.
Too short: 0, incorrect: 0, not proteins: 0.
Time for processing: 0h 0m 0s 65ms
Create directory tmp
search targetDB targetDB alnDB tmp -a --exhaustive-search

MMseqs Version:              	9.427df8a
Seq. id. threshold           	0
Coverage threshold           	0
Coverage mode                	0
Max reject                   	2147483647
Max accept                   	2147483647
Add backtrace                	true
TMscore threshold            	0
TMalign hit order            	0
TMalign fast                 	1
Preload mode                 	0
Threads                      	14
Verbosity                    	3
LDDT threshold               	0
Sort by structure bit score  	1
Alignment type               	2
Exact TMscore                	0
Substitution matrix          	aa:3di.out,nucl:3di.out
Alignment mode               	3
Alignment mode               	0
E-value threshold            	10
Min alignment length         	0
Seq. id. mode                	0
Alternative alignments       	0
Max sequence length          	65535
Compositional bias           	1
Compositional bias           	1
Gap open cost                	aa:10,nucl:10
Gap extension cost           	aa:1,nucl:1
Compressed                   	0
Seed substitution matrix     	aa:3di.out,nucl:3di.out
Sensitivity                  	9.5
k-mer length                 	0
Target search mode           	0
k-score                      	seq:2147483647,prof:2147483647
Max results per query        	1000
Split database               	0
Split mode                   	2
Split memory limit           	0
Diagonal scoring             	true
Exact k-mer matching         	0
Mask residues                	0
Mask residues probability    	0.99995
Mask lower case residues     	1
Minimum diagonal score       	30
Selected taxa
Spaced k-mers                	1
Spaced k-mer pattern
Local temporary path
Exhaustive search mode       	true
Prefilter mode               	0
Search iterations            	1
Remove temporary files       	true
MPI runner
Force restart with latest tmp	false
Cluster search               	0

structurealign targetDB targetDB tmp/15707625884678452062/pref tmp/15707625884678452062/strualn --tmscore-threshold 0 --lddt-threshold 0 --sort-by-structure-bits 1 --alignment-type 2 --exact-tmscore 0 --sub-mat 'aa:3di.out,nucl:3di.out' -a 1 --alignment-mode 3 --alignment-output-mode 0 --wrapped-scoring 0 -e 10 --min-seq-id 0 --min-aln-len 0 --seq-id-mode 0 --alt-ali 0 -c 0 --cov-mode 0 --max-seq-len 65535 --comp-bias-corr 1 --comp-bias-corr-scale 0.5 --max-rejected 2147483647 --max-accept 2147483647 --add-self-matches 0 --db-load-mode 0 --pca substitution:1.100,context:1.400 --pcb substitution:4.100,context:5.800 --score-bias 0 --realign 0 --realign-score-bias -0.2 --realign-max-seqs 2147483647 --corr-score-weight 0 --gap-open aa:10,nucl:10 --gap-extend aa:1,nucl:1 --zdrop 40 --threads 14 --compressed 0 -v 3

[=================================================================] 100.00% 75 0s 223ms
Time for merging to strualn: 0h 0m 0s 1ms
Time for processing: 0h 0m 0s 282ms
mvdb tmp/15707625884678452062/strualn tmp/15707625884678452062/aln

Time for processing: 0h 0m 0s 2ms
mvdb tmp/15707625884678452062/aln alnDB -v 3

Time for processing: 0h 0m 0s 3ms
Removing temporary files
rmdb tmp/15707625884678452062/pref -v 3

Time for processing: 0h 0m 0s 1ms
output.txt exists and will be overwritten
convertalis targetDB targetDB alnDB output.txt --format-mode 4 --format-output query,target,qlen,tlen,alnlen,qtmscore,ttmscore,alntmscore,cigar,qseq,tseq

MMseqs Version:        	9.427df8a
Substitution matrix    	aa:3di.out,nucl:3di.out
Alignment format       	4
Format alignment output	query,target,qlen,tlen,alnlen,qtmscore,ttmscore,alntmscore,cigar,qseq,tseq
Gap open cost          	aa:10,nucl:10
Gap extension cost     	aa:1,nucl:1
Database output        	false
Preload mode           	0
Threads                	14
Compressed             	0
Verbosity              	3
Exact TMscore          	0

[=================================================================] 100.00% 75 0s 574ms
Time for merging to output.txt: 0h 0m 0s 3ms
Time for processing: 0h 0m 0s 715ms

Your Environment

Include as many relevant details about the environment you experienced the bug in.

• Git commit used (The string after "MMseqs Version:" when you execute foldseek without any parameters): 9.427df8a
• Which foldseek version was used (Statically-compiled, self-compiled, Conda, etc.): conda (build: pl5321h79102bb_0, also reproduced on pl5321h8ec77f5_1)
• For self-compiled and Homebrew: Compiler and Cmake versions used and their invocation: N/A
• Server specifications (especially CPU support for AVX2/SSE and amount of system memory): N/A
• Operating system and version: macOS 14.1

EDIT: The behavior is observed in the following versions (all installed via conda):

• 9.427df8a
• 8.ef4e960
• 7.04e0ec8
• 6.29e2557

The script fails in 5.53465f0 because qtmscore has not been implemented.

Related issues

#221

[ekiefl]

The behavior is also observed with easy-search:

rm -rf tmp
foldseek easy-search ./structures/ ./structures/ output.txt tmp -a --exhaustive-search --format-mode 4 --format-output query,target,qlen,tlen,alnlen,qtmscore,ttmscore,alntmscore,cigar,qseq,tseq

[austinhpatton]

I wanted to dig into this error a bit more, as I've found that in some cases we don't observe the equivalence between ttmscore and alntmscore. I suspected it had something to do with the structure of the --format-output specification. So I tried six combinations, changing the order in which alntmscore, qtmscore, and ttmscore were requested.

The commands used are shown below:

foldseek easy-search ./structures/ ./structures/ v1.m8 tmp_dir_1 --exhaustive-search 1 --alignment-type 2 --format-mode 4 --format-output query,target,qlen,tlen,alnlen,qtmscore,ttmscore,alntmscore
foldseek easy-search ./structures/ ./structures/ v2.m8 tmp_dir_2 --exhaustive-search 1 --alignment-type 2 --format-mode 4 --format-output query,target,qlen,alnlen,tlen,qtmscore,alntmscore,ttmscore
foldseek easy-search ./structures/ ./structures/ v3.m8 tmp_dir_2 --exhaustive-search 1 --alignment-type 2 --format-mode 4 --format-output query,target,tlen,alnlen,qlen,ttmscore,alntmscore,qtmscore
foldseek easy-search ./structures/ ./structures/ v4.m8 tmp_dir_3 --exhaustive-search 1 --alignment-type 2 --format-mode 4 --format-output query,target,tlen,qlen,alnlen,ttmscore,qtmscore,alntmscore
foldseek easy-search ./structures/ ./structures/ v5.m8 tmp_dir_4 --exhaustive-search 1 --alignment-type 2 --format-mode 4 --format-output query,target,alnlen,qlen,tlen,alntmscore,qtmscore,ttmscore
foldseek easy-search ./structures/ ./structures/ v6.m8 tmp_dir_5 --exhaustive-search 1 --alignment-type 2 --format-mode 4 --format-output query,target,alnlen,tlen,qlen,alntmscore,ttmscore,qtmscore

I modified @ekiefl's python script to check if alntmscore was always equal to either ttmscore or qtmscore given an output file provided from the commandline - this is shown below:

import pandas as pd
import argparse

parser = argparse.ArgumentParser()
parser.add_argument("input_file", type=str)
args = parser.parse_args()

df = pd.read_csv(args.input_file, sep="\t")

# Check if alntmscore equals ttmscore or qtmscore
if (df.ttmscore == df.alntmscore).all():
    print("All alntmscore values equal ttmscore.")
elif (df.qtmscore == df.alntmscore).all():
    print("All alntmscore values equal qtmscore.")
else:
    print("alntmscore values do not match ttmscore or qtmscore consistently.")

print(
    df[["query", "target", "qlen", "tlen", "alnlen", "qtmscore", "ttmscore", "alntmscore"]]
    .sample(20)
    .tail(10)
    .to_markdown()
)

And using this script, we find the following:

python foldseek_debug.py v1.m8
All alntmscore values equal ttmscore.
|      | query      | target     |   qlen |   tlen |   alnlen |   qtmscore |   ttmscore |   alntmscore |
|-----:|:-----------|:-----------|-------:|-------:|---------:|-----------:|-----------:|-------------:|
| 1694 | Q9PRC5.pdb | V4TDT2.pdb |    230 |    539 |      286 |     0.6175 |     0.2986 |       0.2986 |
|  317 | B0K7K7.pdb | B1AI06.pdb |    203 |    230 |      209 |     0.88   |     0.7825 |       0.7825 |
| 2455 | B1L821.pdb | U5U2L0.pdb |    197 |    550 |      182 |     0.5044 |     0.2132 |       0.2132 |
| 4026 | Q20230.pdb | W5PRH4.pdb |    191 |    277 |      245 |     0.6217 |     0.4638 |       0.4638 |
| 5581 | L1J5L1.pdb | Q54UT2.pdb |    254 |    285 |      256 |     0.6551 |     0.5885 |       0.5885 |
| 2561 | U7PJT1.pdb | W5N0Q0.pdb |    257 |    263 |      288 |     0.3744 |     0.368  |       0.368  |
| 1031 | Q8R9S6.pdb | I3MMU0.pdb |    206 |    277 |      276 |     0.3828 |     0.3006 |       0.3006 |
| 1800 | Q54UT2.pdb | Q54UT2.pdb |    285 |    285 |      285 |     1      |     1      |       1      |
| 1023 | Q8R9S6.pdb | L8IGY7.pdb |    206 |    256 |      252 |     0.4647 |     0.3913 |       0.3913 |
| 4843 | Q4DE18.pdb | K7GFB3.pdb |    351 |    242 |      240 |     0.4184 |     0.5662 |       0.5662 |

python foldseek_debug.py v2.m8
All alntmscore values equal qtmscore.
|      | query      | target     |   qlen |   tlen |   alnlen |   qtmscore |   ttmscore |   alntmscore |
|-----:|:-----------|:-----------|-------:|-------:|---------:|-----------:|-----------:|-------------:|
| 3434 | B1AIY1.pdb | B0U447.pdb |    224 |    217 |      260 |     0.3748 |     0.3864 |       0.3748 |
|  369 | W5N0Q0.pdb | P38493.pdb |    263 |    224 |      274 |     0.3627 |     0.4111 |       0.3627 |
| 1324 | P38493.pdb | B7MTM8.pdb |    224 |    213 |      263 |     0.39   |     0.4051 |       0.39   |
| 4291 | B0U447.pdb | B1L821.pdb |    217 |    197 |      196 |     0.7853 |     0.8619 |       0.7853 |
| 1887 | B0W8G8.pdb | W5N438.pdb |    246 |    264 |      258 |     0.8034 |     0.7519 |       0.8034 |
| 4467 | P48769.pdb | B7MTM8.pdb |    260 |    213 |      258 |     0.5212 |     0.6163 |       0.5212 |
| 5507 | P27707.pdb | W0R9N0.pdb |    260 |    221 |      246 |     0.676  |     0.7849 |       0.676  |
| 4474 | P48769.pdb | Q7VKH4.pdb |    260 |    214 |      265 |     0.5115 |     0.6031 |       0.5115 |
| 4617 | B1L821.pdb | Q20230.pdb |    197 |    191 |      211 |     0.4935 |     0.5054 |       0.4935 |
| 1502 | V4TDT2.pdb | V7BU96.pdb |    539 |    563 |      564 |     0.622  |     0.5973 |       0.622  |

python foldseek_debug.py v3.m8
All alntmscore values equal ttmscore.
|      | query      | target     |   qlen |   tlen |   alnlen |   qtmscore |   ttmscore |   alntmscore |
|-----:|:-----------|:-----------|-------:|-------:|---------:|-----------:|-----------:|-------------:|
| 1729 | B7MTM8.pdb | Q7VKH4.pdb |    213 |    214 |      209 |     0.9559 |     0.9515 |       0.9515 |
| 4020 | B1I165.pdb | L8IGY7.pdb |    221 |    256 |      249 |     0.5575 |     0.4923 |       0.4923 |
| 2513 | B0W8G8.pdb | B8F7G0.pdb |    246 |    208 |      235 |     0.5625 |     0.6493 |       0.6493 |
| 4939 | V4T3S7.pdb | Q1QEQ3.pdb |    470 |    230 |      205 |     0.2217 |     0.4064 |       0.4064 |
| 4871 | Q20230.pdb | V4KAC2.pdb |    191 |    599 |      182 |     0.511  |     0.1944 |       0.1944 |
| 3962 | W5JIH9.pdb | P63807.pdb |    275 |    219 |      265 |     0.3686 |     0.4393 |       0.4393 |
| 4892 | V4T3S7.pdb | I3MMU0.pdb |    470 |    277 |      278 |     0.3968 |     0.6615 |       0.6615 |
| 2351 | Q6FZE3.pdb | B1JI38.pdb |    192 |    212 |      231 |     0.4975 |     0.4619 |       0.4619 |
| 2572 | I3MMU0.pdb | B0W8G8.pdb |    277 |    246 |      256 |     0.701  |     0.7851 |       0.7851 |
| 4191 | Q9PRC5.pdb | Q8Y5W6.pdb |    230 |    224 |      257 |     0.4048 |     0.4136 |       0.4136 |

python foldseek_debug.py v4.m8
All alntmscore values equal qtmscore.
|      | query      | target     |   qlen |   tlen |   alnlen |   qtmscore |   ttmscore |   alntmscore |
|-----:|:-----------|:-----------|-------:|-------:|---------:|-----------:|-----------:|-------------:|
| 1507 | W5PR96.pdb | Q58EI2.pdb |    260 |    264 |      264 |     0.9114 |     0.898  |       0.9114 |
| 3144 | B6U6Y9.pdb | Q3AFE0.pdb |    489 |    289 |      181 |     0.1788 |     0.2795 |       0.1788 |
| 3583 | W5JIH9.pdb | Q20230.pdb |    275 |    191 |      224 |     0.4478 |     0.6059 |       0.4478 |
|  690 | O83373.pdb | B7MTM8.pdb |    208 |    213 |      217 |     0.7442 |     0.7285 |       0.7442 |
| 5293 | U7PMD6.pdb | B2UVL9.pdb |    307 |    191 |      279 |     0.3281 |     0.4828 |       0.3281 |
| 3994 | B4TFH5.pdb | Q9PRC5.pdb |    213 |    230 |      216 |     0.8171 |     0.7617 |       0.8171 |
| 4220 | V4TDT2.pdb | V6DIT2.pdb |    539 |    234 |      275 |     0.4053 |     0.9165 |       0.4053 |
| 2414 | V4T3S7.pdb | M3ZT88.pdb |    470 |    263 |      279 |     0.4116 |     0.7195 |       0.4116 |
| 2393 | Q6FZE3.pdb | B6U6Y9.pdb |    192 |    489 |      200 |     0.4129 |     0.1927 |       0.4129 |
| 2929 | W1NL77.pdb | I1HTV7.pdb |    586 |    517 |      521 |     0.5987 |     0.6735 |       0.5987 |

python foldseek_debug.py v5.m8
alntmscore values do not match ttmscore or qtmscore consistently.
|      | query      | target     |   qlen |   tlen |   alnlen |   qtmscore |   ttmscore |   alntmscore |
|-----:|:-----------|:-----------|-------:|-------:|---------:|-----------:|-----------:|-------------:|
|   22 | B0K7K7.pdb | O83373.pdb |    203 |    208 |      210 |     0.8133 |     0.7951 |       0.8133 |
| 1205 | M8CV53.pdb | U5U2L0.pdb |    526 |    550 |      520 |     0.6098 |     0.5853 |       0.6161 |
| 1249 | M8CV53.pdb | B0K7K7.pdb |    526 |    203 |      198 |     0.211  |     0.479  |       0.4891 |
| 2314 | B1JI38.pdb | Q3AFE0.pdb |    212 |    289 |      209 |     0.3783 |     0.3027 |       0.3827 |
| 4951 | I1HTV7.pdb | M8CV53.pdb |    517 |    526 |      468 |     0.6866 |     0.6754 |       0.7547 |
| 1175 | V6DIT2.pdb | B1J4Z5.pdb |    234 |    207 |      243 |     0.51   |     0.5661 |       0.5661 |
| 5024 | I1HTV7.pdb | Q8R9S6.pdb |    517 |    206 |      305 |     0.2084 |     0.4336 |       0.4336 |
| 3224 | P48769.pdb | B1AIY1.pdb |    260 |    224 |      292 |     0.3047 |     0.3421 |       0.3421 |
| 1199 | V6DIT2.pdb | B1J5H0.pdb |    234 |    228 |      272 |     0.4114 |     0.4199 |       0.4199 |
| 2686 | L8IGY7.pdb | C5A558.pdb |    256 |    190 |      235 |     0.4193 |     0.5343 |       0.5343 |

python foldseek_debug.py v6.m8
alntmscore values do not match ttmscore or qtmscore consistently.
|      | query      | target         |   qlen |   tlen |   alnlen |   qtmscore |   ttmscore |   alntmscore |
|-----:|:-----------|:---------------|-------:|-------:|---------:|-----------:|-----------:|-------------:|
| 3852 | B8F7G0.pdb | W5PR96.pdb     |    208 |    260 |      259 |     0.6604 |     0.545  |       0.6604 |
| 5475 | B1J4Z5.pdb | B1J4Z5.pdb     |    207 |    207 |      207 |     1      |     1      |       1      |
| 2670 | Q9XZT6.pdb | B1J4Z5.pdb     |    250 |    207 |      230 |     0.5031 |     0.5887 |       0.5887 |
| 3688 | I3MMU0.pdb | A0A818UTT8.pdb |    277 |    249 |      243 |     0.6877 |     0.7622 |       0.7804 |
| 3365 | Q4DE18.pdb | B1AIY1.pdb     |    351 |    224 |      267 |     0.2955 |     0.422  |       0.422  |
| 3928 | Q8Y5W6.pdb | I3MMU0.pdb     |    224 |    277 |      278 |     0.4254 |     0.3598 |       0.4254 |
| 4085 | W6ULK1.pdb | Q9UXG7.pdb     |    186 |    189 |      173 |     0.5381 |     0.5309 |       0.5719 |
| 3140 | B0K7K7.pdb | Q8R9S6.pdb     |    203 |    206 |      230 |     0.4541 |     0.4497 |       0.4541 |
| 4465 | Q834T6.pdb | P48769.pdb     |    226 |    260 |      265 |     0.4025 |     0.3607 |       0.4025 |
| 5571 | O83373.pdb | B1AI06.pdb     |    208 |    230 |      219 |     0.7591 |     0.6935 |       0.7591 |

In brief:

1. When alntmscore is not the first of the three TM-scores requested by --format-output, it always equals either ttmscore or qtmscore
   • Which score it is equal to switches dependent on the order in which columns are requested
2. When alntmscore is the first of the three TM-scores requested by --format-output, it does not consistently equal one or the other, but often (though not always) is equal to one of them, but not in a manner that is consistently predicted by qlen, tlen, or alnlen

[ekiefl]

Bumping this issue, @milot-mirdita and @martin-steinegger.

Below is an example of how detrimental this bug is.

I performed a one-versus-many calculation of TM-score with both Foldseek and TMalign. For each TMalign result, the "alntmscore" is calculated manually with the following prescription:

---

---

Then, the scores are ranked from high to low and the result is shown as the monotonically increasing trace (blue). Using the same ordering, the foldseek alntmscore results are shown as red dots using the following settings:

prefilter_mode=2
alignment_type=1
tmalign_fast=0
exact_tmscore=1

If instead I manually calculate alntmscore from the Foldseek values qlen, qtmscore, tlen and tmscore according to the above prescription, the results between TMalign and Foldseek converge:

As far as I can tell, everyone who uses the alntmscore output by Foldseek gets results akin to the first plot, as is demonstrated in the MRE I presented above.

[martin-steinegger]

Thank you for the analysis. How do you compute the alntmscore? I checked the code we do use std::min(static_cast(res.backtrace.size()), std::min(res.dbLen, res.qLen))) as normalization factor.

Normalizing by res.backtrace.size() might be better, maybe we should change this.

[martin-steinegger]

Okay I think I know what is going on. If you print out qtmscore,ttmscore,alntmscore then the alntmscore = ttmscore.
If you print ttmscore,qtmscore,alntmscore then alntmscore = qtmscore;
However, if you print alntmscore,qtmscore,ttmscore then it should work. This is a bug, which I will fix soon.

[ekiefl]

Thanks for the response.

How do you compute the alntmscore?

I provided some equations above, perhaps they didn't render. Or is there something more specific you are curious about?

In case of TMalign, our ranking is done by (qTM+tTM)/2. Might this explain why you see this kind of ranking?

Each comparison, whether calculated by TMalign or Foldseek, is ordered according to TMAlign's alntmscore. That's why the TMalign curve monotonically increases. So Foldseek's ranking is irrelevant given how the data has been presented.

Normalizing by res.backtrace.size() might be better, maybe we should change this.

Given that ttmscore is normalized by tlen, and qtmscore is normalized by qlen, I think I agree that alntmscore should be calculated by normalizing by alnlen.

---

However, a bug persists even if this were changed, as illustrated in this table output from the MRE. Allow me to explain.

Basically, in the table alntmscore always equates with ttmscore. Given that the normalization is min(alnlen, qlen, tlen), the only way in which this would be possible is if tlen is always smaller thanalnlen, however, the other columns show that isn't the case.

	query	target	qlen	tlen	alnlen	qtmscore	ttmscore	alntmscore
1187	B0RXV1	V7BU96	227	563	206	0.5245	0.2354	0.2354
5406	U5U2L0	V4KAC2	550	599	560	0.6095	0.5638	0.5638
3335	Q5F9Z5	Q9XZT6	206	250	234	0.6303	0.5362	0.5362
861	B0BUU8	Q20230	203	191	210	0.5333	0.5591	0.5591
842	B0BUU8	B0RXV1	203	227	199	0.8489	0.7633	0.7633
2810	K0F1X4	B1JTS0	178	206	211	0.6423	0.5692	0.5692
2927	L8IGY7	P48769	256	260	254	0.9265	0.9125	0.9125
4925	Q8R9S6	Q834T6	206	226	245	0.4625	0.4305	0.4305
3861	W5N438	B8F7G0	264	208	255	0.5412	0.6642	0.6642
960	B0K119	C5A558	203	190	218	0.5143	0.5417	0.5417

[martin-steinegger]

@ekiefl thank you so much. Please see my comment above. Could you please try to print out the tmscores in this order: alntmscore,qtmscore,ttmscore. Does this change anything?

[ekiefl]

Okay I think I know what is going on. If you print out qtmscore,ttmscore,alntmscore then the alntmscore = ttmscore.
If you print ttmscore,qtmscore,alntmscore then alntmscore = qtmscore;
However, if you print alntmscore,qtmscore,ttmscore then it should work. This is a bug, which I will fix soon.

Exactly. It has something to do with this variable:

foldseek/src/strucclustutils/structureconvertalis.cpp

Line 891 in bc212bc

tmres = tmaligner->computeTMscore(targetCaData, &targetCaData[res.dbLen], &targetCaData[res.dbLen+res.dbLen], res.dbLen,

Given that the normalization is currently min(alnlen, qlen, tlen), @austinhpatton's comment makes sense:

When alntmscore is the first of the three TM-scores requested by --format-output, it does not consistently equal one or the other, but often (though not always) is equal to one of them, but not in a manner that is consistently predicted by qlen, tlen, or alnlen

---

Could you try to print out the tmscores in this order: alntmscore,qtmscore,ttmscore. Does this change anything?

The specific example stems from a subset of a larger all-vs-all, so it's not easy to re-run the results. But, I have just confirmed the effect that the order alntmscore,qtmscore,ttmscore has on the MRE, which now produces this table:

	query	target	qlen	tlen	alnlen	qtmscore	ttmscore	alntmscore
3599	B1I165	U7PMD6	221	307	303	0.4044	0.315	0.4044
3772	B1JI38	Q9UXG7	212	189	209	0.7307	0.8108	0.8108
2365	I3MMU0	B4TFH5	277	213	254	0.4645	0.5776	0.5776
4929	Q20230	P0C1G0	191	228	230	0.6095	0.5279	0.6095
1568	Q9UXG7	Q8R9S6	189	206	217	0.4942	0.4614	0.4942
5620	Q7VKH4	U7PMD6	214	307	299	0.4353	0.3304	0.4353
760	W5N0Q0	V4CH82	263	229	262	0.7477	0.8545	0.8545
2397	I3MMU0	Q3AFE0	277	289	234	0.3628	0.3507	0.4174
5389	Q5F9Z5	B1AIY1	206	224	262	0.4457	0.4171	0.4457
1115	W6ULK1	P63807	186	219	221	0.2924	0.2573	0.2924

This table matches what one expects if alntmscore is normalized by the minimum of the lengths.

---

When you get around to fixing the bug, may I suggest reporting a rawtmscore that one can normalize how they see fit?

[martin-steinegger]

Thank you Evan. I pushed a fix. Could you retry it with the newest version?
It should fix the order issue and I changed the normalization to the backtrace size as well.

[ekiefl]

Great. I can test it out if you point me to some instructions for building from source.

[martin-steinegger]

We will update conda soon. Also the static compiled binaries should work.