Realign process stalls: Excessive BAM unpacking without analysis progression

[LauraNieboer]

First of all, I want to express my gratitude for your work on developing this algorithm. It has been very helpful for my research.

I've successfully run about 40 samples using Realign. However, I'm encountering some difficulties with the remaining half of my samples. The process seems to stall, making little progress. For example, one sample (totaling about 20 GB for the three BAM files used when running Realign) has read about 300 TB of data but keeps returning to the same place in the same file. It's using all the computational power to unpack BAM files and almost none on analysis.
I've attempted to address this by increasing the number of cores, but it doesn't seem to make a difference. My last attempt used 32 cores and 128 GB of RAM, with the maximum runtime available to me on the cluster being 7 days, which was not enough time.
I'm puzzled as to why about half of my samples were able to run successfully while the other half seems unable to finish. There are no major differences in the size of the input BAMs between the samples that ran successfully and those that are not able to run.
Is there a known reason for this problem? Are there any suggestions or workarounds you could provide to help process these remaining samples?

Thank you in advance for any assistance you can offer.

[iprada]

Hi Laura,

First of all: thanks for your kind words. I am happy to hear you find it helpful!

Regarding the issue you encounter, the problem is on Circle-Map side. I described the problem here #37 (comment)

Unfortunately, I am working on other things right now I do not have the time to fix it. But If I hear more about what you are trying to do I might be able to come up with some suggestions.

You can find my email here: https://di.ku.dk/english/staff/vip/researchers_ml/ feel free to write me and we take it from there.

best,

Iñigo