Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some problems during execution #61

Open
duan12345678 opened this issue May 10, 2024 · 1 comment
Open

Some problems during execution #61

duan12345678 opened this issue May 10, 2024 · 1 comment

Comments

@duan12345678
Copy link

Hi
To obtain the prequel input file for TOGA, I attempted the process based on the information provided, but encountered some difficulties. I'm unsure how to proceed and would appreciate any suggestions or guidance you may have. Thank you for your assistance.
Here are my command and the erro log:
./make_chains.py hg38 Ea /home/make_lastz_chains/genome/hg38.2bit
/home/make_lastz_chains/genome/Ea.2bit --pd /home/Ea -f --chaining_memory 16
I use --executor local instead of --executor slurm, is there any parameter to speed it up, it's too slow
current output :
1、
N E X T F L O W ~ version 21.10.6
Launching /home/make_lastz_chains/parallelization/execute_joblist.nf [intergalactic_khorana] - revision: 0483b29723
executor > local (5)
[5f/78dfbd] process > execute_jobs (2) [ 0%] 4 of 105150
2、
N E X T F L O W ~ version 21.10.6
Launching /home/make_lastz_chain/make_lastz_chains/parallelization/execute_joblist.nf [wise_austin] - revision: 0483b29723
executor > local (207)
[78/011ee5] process > execute_jobs (223) [ 9%] 196 of 2010
In addition, I noticed in the previous Q&As that there is a parameter "--executor_queuesize 8" that can make it multi-threaded, but it seems to no longer exist in the current version, whether it has been replaced by "--cluster_queue 8", My running speed is about 5% of the number of jobs finished a day, do you have any suggestions for this, thanks a lot.
best wishes

@MichaelHiller
Copy link
Collaborator

If I understand it right, jobs are running, just very slow.
There is no multithreading. Rather the data is split into chunks and each chunk can run in parallel, either on a compute cluster or a single node (where you can then use all its cores).

Regarding speed, pls check if you repeatModeled and masked both genomes (hg38 should be fine though).

@kirilenkobm Can you comment on the --executor_queuesize parameter?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants