Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error executing process > 'NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:BCFTOOLS_SORT #328

Closed
Rooksie opened this issue Aug 31, 2022 · 3 comments
Closed

Error executing process > 'NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:BCFTOOLS_SORT #328

Rooksie opened this issue Aug 31, 2022 · 3 comments
Labels
bug Something isn't working

Comments

@Rooksie
Copy link

Rooksie commented Aug 31, 2022

Description of the bug

I haven't seen a post, and I wondering if anyone else is having an issue with bcftools at the ivar stage. I am receiving this error today. I am able to run version 2.4.1 on the same files without the error and I am wondering if this is a bcftools image issue or possibly something else.

I previously tried removing the sample that errored, but on rerunning the error pops up again on a different sample, repeatedly.

Command used and terminal output

$nextflow run nf-core/viralrecon --input Clemson3.sample.sheet.csv --outdir . --platform illumina      -r 2.5      -c custom.pangolin.config      --protocol amplicon      --genome 'MN908947.3'      --primer_set artic      --primer_set_version 4.1      --skip_assembly      --min_mapped_reads 15000      -profile singularity
...
[Normal looking running]
...
[0b/6d799a] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:GUNZIP_GFF (GCA_009858895.3_ASM985889v3_genomic.200409.gff.gz)                 [100%] 1 of 1 ✔
[33/e5415a] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:CUSTOM_GETCHROMSIZES (SARS-CoV-2.reference.fasta)                              [100%] 1 of 1 ✔
[4e/129cec] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_KRAKEN2_DB (kraken2_human.tar.gz)                                        [100%] 1 of 1 ✔
[36/b391a9] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:COLLAPSE_PRIMERS (SARS-CoV-2.scheme.bed)                                       [100%] 1 of 1 ✔
[57/d7d179] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:BOWTIE2_BUILD (SARS-CoV-2.reference.fasta)                                     [100%] 1 of 1 ✔
[6c/df7778] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:UNTAR_NEXTCLADE_DB (nextclade_sars-cov-2_MN908947_2022-06-14T12_00_00Z.tar.gz) [100%] 1 of 1 ✔
[5d/2d0dc2] process > NFCORE_VIRALRECON:ILLUMINA:PREPARE_GENOME:SNPEFF_BUILD (SARS-CoV-2.reference.fasta)                                      [100%] 1 of 1 ✔
[3c/38533c] process > NFCORE_VIRALRECON:ILLUMINA:INPUT_CHECK:SAMPLESHEET_CHECK (Clemson3.sample.sheet.csv)                                     [100%] 1 of 1 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CAT_FASTQ                                                                                     -
[2d/b3dbba] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_RAW (Sample185)                                                  [100%] 185 of 185 ✔
[39/486a78] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTP (Sample185)                                                       [100%] 185 of 185 ✔
[0a/743425] process > NFCORE_VIRALRECON:ILLUMINA:FASTQC_FASTP:FASTQC_TRIM (Sample185)                                                 [100%] 185 of 185 ✔
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_TSV_FAIL_READS                                                                        -
[80/44fc3f] process > NFCORE_VIRALRECON:ILLUMINA:KRAKEN2_KRAKEN2 (Sample185)                                                          [100%] 185 of 185 ✔
[e5/db6a1f] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BOWTIE2_ALIGN (Sample185)                                              [100%] 185 of 185 ✔
[fe/4431bd] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:SAMTOOLS_SORT (Sample185)                            [100%] 185 of 185 ✔
[40/610d74] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:SAMTOOLS_INDEX (Sample185)                           [100%] 185 of 185 ✔
[d6/0ab86e] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS (sampleN)        [100%] 185 of 185 ✔
[ca/987f39] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT (sampleX)     [100%] 185 of 185 ✔
[20/36d29a] process > NFCORE_VIRALRECON:ILLUMINA:ALIGN_BOWTIE2:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTATS (Sample185)     [100%] 185 of 185 ✔
[5c/ac4a63] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_TSV_FAIL_MAPPED                                                                       [100%] 1 of 1 ✔
[c1/90e6d0] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:IVAR_TRIM (sampleX)                                               [100%] 184 of 184 ✔
[ef/7bd9b7] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:BAM_SORT_SAMTOOLS:SAMTOOLS_SORT (sampleX)                         [100%] 184 of 184 ✔
[8f/dcb9fb] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:BAM_SORT_SAMTOOLS:SAMTOOLS_INDEX (sampleX)                        [100%] 184 of 184 ✔
[ec/584b40] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_STATS (sampleN)     [100%] 184 of 184 ✔
[de/fd7385] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_FLAGSTAT (sampleX)  [100%] 184 of 184 ✔
[31/6d3265] process > NFCORE_VIRALRECON:ILLUMINA:PRIMER_TRIM_IVAR:BAM_SORT_SAMTOOLS:BAM_STATS_SAMTOOLS:SAMTOOLS_IDXSTATS (sampleX)  [100%] 184 of 184 ✔
[00/457ed8] process > NFCORE_VIRALRECON:ILLUMINA:PICARD_COLLECTMULTIPLEMETRICS (sampleN)                                            [ 98%] 182 of 184
[f6/feeca4] process > NFCORE_VIRALRECON:ILLUMINA:MOSDEPTH_GENOME (sampleN)                                                          [ 98%] 182 of 184
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PLOT_MOSDEPTH_REGIONS_GENOME                                                                  -
[f9/f1d828] process > NFCORE_VIRALRECON:ILLUMINA:MOSDEPTH_AMPLICON (sampleN)                                                        [ 98%] 182 of 184
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:PLOT_MOSDEPTH_REGIONS_AMPLICON                                                                -
[d0/1544fe] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:IVAR_VARIANTS (sampleN)                                              [ 98%] 182 of 184
[3d/24caa4] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:IVAR_VARIANTS_TO_VCF (sampleZ)                                       [100%] 180 of 180
[7f/69fa49] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:BCFTOOLS_SORT (sampleG)                                              [  1%] 3 of 180, failed: 3
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VCF_TABIX_STATS:TABIX_TABIX                                                     -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VCF_TABIX_STATS:BCFTOOLS_STATS                                                  -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VARIANTS_QC:SNPEFF_SNPSIFT:SNPEFF_ANN                                           -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VARIANTS_QC:SNPEFF_SNPSIFT:VCF_BGZIP_TABIX_STATS:TABIX_BGZIP                    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VARIANTS_QC:SNPEFF_SNPSIFT:VCF_BGZIP_TABIX_STATS:VCF_TABIX_STATS:TABIX_TABIX    -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VARIANTS_QC:SNPEFF_SNPSIFT:VCF_BGZIP_TABIX_STATS:VCF_TABIX_STATS:BCFTOOLS_STATS -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VARIANTS_QC:SNPEFF_SNPSIFT:SNPSIFT_EXTRACTFIELDS                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:VARIANTS_QC:ASCIIGENOME                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:BCFTOOLS_FILTER                                                            -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:TABIX_TABIX                                                                -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:MAKE_BED_MASK                                                              -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:BEDTOOLS_MERGE                                                             -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:BEDTOOLS_MASKFASTA                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:BCFTOOLS_CONSENSUS                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:RENAME_FASTA_HEADER                                                        -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:CONSENSUS_QC:QUAST                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:CONSENSUS_QC:PANGOLIN                                                      -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:CONSENSUS_QC:NEXTCLADE_RUN                                                 -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CONSENSUS_BCFTOOLS:CONSENSUS_QC:PLOT_BASE_DENSITY                                             -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC_TSV_NEXTCLADE                                                                         -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_LONG_TABLE:BCFTOOLS_QUERY                                                            -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:VARIANTS_LONG_TABLE:MAKE_VARIANTS_LONG_TABLE                                                  -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:CUSTOM_DUMPSOFTWAREVERSIONS                                                                   -
[-        ] process > NFCORE_VIRALRECON:ILLUMINA:MULTIQC                                                                                       -
-[nf-core/viralrecon] 184/185 samples passed Bowtie2 15000 mapped read threshold:
    4895927: sample2
    4869847: sample3
    1397118: sample4
    9120220: sample5
    5883515: sample6
    3218522: sample7
    ..see pipeline reports for full list
-
-[nf-core/viralrecon] 1 samples skipped since they failed Bowtie2 15000 mapped read threshold:
    154: sampl154
-[nf-core/viralrecon] Pipeline completed with errors-
Error executing process > 'NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:BCFTOOLS_SORT (sample1)'

Caused by:
  Process `NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:BCFTOOLS_SORT (sample1)` terminated with an error exit status (255)

Command executed:

  bcftools \
      sort \
      --output sample1.vcf.gz \
       \
      sample1.vcf

  cat <<-END_VERSIONS > versions.yml
  "NFCORE_VIRALRECON:ILLUMINA:VARIANTS_IVAR:BCFTOOLS_SORT":
      bcftools: $(bcftools --version 2>&1 | head -n1 | sed 's/^.*bcftools //; s/ .*$//')
  END_VERSIONS

Command exit status:
  255

Command output:
  (empty)

Command error:
  mkdtemp(/local_scratch/pbs.87345.pbs02/bcftools.UkuBYp) failed: No such file or directory

Work dir:
  /2.5/work/98/892b340f809e7e690618ff769fe75a

Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line

Relevant files

nextflow.log

$ cat custom.pangolin.config
process {
withName: PANGOLIN {
container = 'https://depot.galaxyproject.org/singularity/pangolin:4.1.2--pyhdfd78af_0'
}
}

System information

Nextflow version: 22.04.5
Hardware: HPC
Executor: PBS
Container engine: Singularity
OS : Rocky Linux 8.6
Version of nf-core/viralrecon: 2.5
@Rooksie Rooksie added the bug Something isn't working label Aug 31, 2022
@drpatelh
Copy link
Member

drpatelh commented Sep 2, 2022 •
edited
Loading

Hi @Rooksie ! Haven't seen this error before but looks like it is coming from here:

Command error:
mkdtemp(/local_scratch/pbs.87345.pbs02/bcftools.UkuBYp) failed: No such file or directory

For some reason, BCFTools is either unable to create or find that file on your Cluster. It could be because the temp space is full on your nodes?

@drpatelh
Copy link
Member

drpatelh commented Sep 2, 2022

You could try to create a custom.config with the contents below and pass to the pipeline via -c:

process {
    withName: '.*:.*:VARIANTS_IVAR:BCFTOOLS_SORT' {
        ext.args = '--temp-dir ./'
    }
}

This will attempt to create the temp directory in the work directory rather than in the standard TMP space.

@Rooksie
Copy link
Author

Rooksie commented Sep 5, 2022

Thank you @drpatelh, with that change the version 2.5 completed! I guess the picking of the default tmp directory for 2.5 must be different from 2.4.1 since that one never errored for me.

This may be a stupid question. if I want to combine my custom config file for the '--temp-dir' AND the updated Pangolin db, can I just list them one right after the other in the same config file? Or is there anything special I need to do? I.E.,
------------{custom.config}------------
process {
withName: '.:.:VARIANTS_IVAR:BCFTOOLS_SORT' {
ext.args = '--temp-dir ./'
}
}
process {
withName: PANGOLIN {
container = 'https://depot.galaxyproject.org/singularity/pangolin:4.1.2--pyhdfd78af_0'
}
}
------------{custom.config}------------

Will this work?

Thank you for giving me the solution.

@Rooksie Rooksie closed this as completed Nov 9, 2022
@Paururo Paururo mentioned this issue Jan 30, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Rooksie @drpatelh