Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add meta to bowtie build #3848

Merged
merged 37 commits into from
Jun 27, 2024
Merged

Add meta to bowtie build #3848

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
e41c0d6
Add meta to bowtie build
SPPearce Sep 20, 2023
40e871b
Merge branch 'master' into bowtie_build_meta
SPPearce Sep 21, 2023
fa5f4e4
Swap fastq ngscheckmate tests
SPPearce Sep 21, 2023
e30a6e0
Merge branch 'master' into bowtie_build_meta
SPPearce Oct 17, 2023
a0652a6
Merge branch 'master' into bowtie_build_meta
SPPearce Oct 19, 2023
3583d39
Merge branch 'master' into bowtie_build_meta
SPPearce May 3, 2024
04d6f0e
Swap to nf-test
SPPearce May 3, 2024
be8fc54
Remove spurious changes to bowtie2
SPPearce May 3, 2024
4aea567
Actually revert bowtie2
SPPearce May 3, 2024
95216f7
Add the rest of bowtie2
SPPearce May 3, 2024
af4cce4
Again
SPPearce May 3, 2024
7962029
Merge branch 'master' into bowtie_build_meta
SPPearce May 8, 2024
09bfe04
Merge branch 'master' into bowtie_build_meta
SPPearce May 13, 2024
59f9329
Merge branch 'master' into bowtie_build_meta
SPPearce May 22, 2024
5abb827
Fix merge conflict
SPPearce Jun 17, 2024
0dadefd
Revert bowtie2 changes
SPPearce Jun 17, 2024
1a14233
Swap to meta.id as prefix
SPPearce Jun 17, 2024
7481003
Swap bowtie align to nftest
SPPearce Jun 18, 2024
9041630
Update tests
SPPearce Jun 18, 2024
d1ff319
Merge branch 'master' into bowtie_build_meta
SPPearce Jun 18, 2024
f91050e
Use nft-bam
SPPearce Jun 18, 2024
16b496e
Update modules/nf-core/bowtie/build/tests/main.nf.test
SPPearce Jun 18, 2024
27a2513
Swap to testdata path
SPPearce Jun 18, 2024
8239f23
Update modules/nf-core/ngscheckmate/patterngenerator/main.nf
SPPearce Jun 18, 2024
789c3a6
Swap index finding to $
SPPearce Jun 18, 2024
d2f839b
Merge branch 'master' into bowtie_build_meta
SPPearce Jun 18, 2024
ecb2187
Merge branch 'master' into bowtie_build_meta
SPPearce Jun 19, 2024
c999e1d
Fix vafncm test
SPPearce Jun 25, 2024
ee6b12a
Update meta
SPPearce Jun 25, 2024
5380f26
Merge branch 'master' into bowtie_build_meta
SPPearce Jun 25, 2024
3507b85
Remove bam header md5sum
SPPearce Jun 25, 2024
3455f71
Merge branch 'master' into bowtie_build_meta
SPPearce Jun 25, 2024
fe1ca34
Update fastq test
SPPearce Jun 26, 2024
3d1759a
Merge branch 'master' into bowtie_build_meta
SPPearce Jun 26, 2024
81175d6
Update paired bam test
SPPearce Jun 26, 2024
5f3ed48
Delete modules/nf-core/bowtie/align/tests/nextflow.config
SPPearce Jun 27, 2024
88174bc
Merge branch 'master' into bowtie_build_meta
SPPearce Jun 27, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion modules/nf-core/bowtie/align/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ process BOWTIE_ALIGN {

input:
tuple val(meta), path(reads)
path index
tuple val(meta2), path(index)

output:
tuple val(meta), path('*.bam'), emit: bam
Expand Down
11 changes: 6 additions & 5 deletions modules/nf-core/bowtie/build/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
process BOWTIE_BUILD {
tag "$fasta"
tag "${meta.id}"
label 'process_high'

conda "${moduleDir}/environment.yml"
Expand All @@ -8,20 +8,21 @@ process BOWTIE_BUILD {
'biocontainers/bowtie:1.3.0--py38hed8969a_1' }"

input:
path fasta
tuple val(meta), path(fasta)

output:
path 'bowtie' , emit: index
path "versions.yml" , emit: versions
tuple val(meta), path('bowtie') , emit: index
path "versions.yml" , emit: versions

when:
task.ext.when == null || task.ext.when

script:
def args = task.ext.args ?: ''
def prefix = task.ext.prefix ?: "${fasta.baseName}"
SPPearce marked this conversation as resolved.
Show resolved Hide resolved
"""
mkdir bowtie
bowtie-build --threads $task.cpus $fasta bowtie/${fasta.baseName}
bowtie-build --threads $task.cpus $fasta bowtie/${prefix}
cat <<-END_VERSIONS > versions.yml
"${task.process}":
bowtie: \$(echo \$(bowtie --version 2>&1) | sed 's/^.*bowtie-align-s version //; s/ .*\$//')
Expand Down
12 changes: 11 additions & 1 deletion modules/nf-core/bowtie/build/meta.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,23 @@ tools:
arxiv: arXiv:1303.3997
licence: ["Artistic-2.0"]
input:
- meta:
type: map
description: |
Groovy Map containing information about the genome fasta
e.g. [ id:'test' ]
- fasta:
type: file
description: Input genome fasta file
output:
- meta:
type: map
description: |
Groovy Map containing nformation about the genome fasta
e.g. [ id:'test' ]
- index:
type: file
description: Bowtie genome index files
description: Folder containing bowtie genome index files
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
description: Folder containing bowtie genome index files
description: bowtie genome index files

If I see it correctly, you get all the ebwt files in index, not just the bowtie folder containing them

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are all inside the bowtie folder, so the downstream process receives the whole folder. Not sure why the nf-test doesn't show that on the md5sums

pattern: "*.ebwt"
- versions:
type: file
Expand Down
2 changes: 1 addition & 1 deletion modules/nf-core/ngscheckmate/patterngenerator/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ process NGSCHECKMATE_PATTERNGENERATOR {
input:
tuple val(meta), path(bed)
tuple val(meta2), path(fasta)
path(bowtie_index)
tuple val(meta3), path(bowtie_index)

output:
tuple val(meta), path("*.pt"), emit: pt
Expand Down
12 changes: 10 additions & 2 deletions tests/modules/nf-core/bowtie/align/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,11 @@ workflow test_bowtie_align_single_end {
file(params.test_data['sarscov2']['illumina']['test_1_fastq_gz'], checkIfExists: true)
]
]
fasta = file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)

fasta = [
[id: 'test'],
file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)
]

BOWTIE_BUILD ( fasta )
BOWTIE_ALIGN ( input, BOWTIE_BUILD.out.index )
Expand All @@ -26,7 +30,11 @@ workflow test_bowtie_align_paired_end {
file(params.test_data['sarscov2']['illumina']['test_2_fastq_gz'], checkIfExists: true)
]
]
fasta = file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)

fasta = [
[id: 'test'],
file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)
]

BOWTIE_BUILD ( fasta )
BOWTIE_ALIGN ( input, BOWTIE_BUILD.out.index )
Expand Down
6 changes: 5 additions & 1 deletion tests/modules/nf-core/bowtie/build_test/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ nextflow.enable.dsl = 2
include { BOWTIE_BUILD } from '../../../../../modules/nf-core/bowtie/build/main.nf'

workflow test_bowtie_build {
fasta = file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)

fasta = [
[id: 'test'],
file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)
]

BOWTIE_BUILD ( fasta )
}
8 changes: 2 additions & 6 deletions tests/modules/nf-core/ngscheckmate/fastq/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,6 @@ workflow test_ngscheckmate_fastq {
file(params.test_data['sarscov2']['genome']['genome_fasta_fai'], checkIfExists: true)
]

bowtie_fasta = file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)

// create a bed file representing the entire genome from the fasta index
GAWK_FAI(fasta_fai, [])

Expand All @@ -37,7 +35,7 @@ workflow test_ngscheckmate_fastq {
GAWK_BED(BEDTOOLS_MAKEWINDOWS.out.bed, [])

// generate the .pt file for ngscheckmate to use
BOWTIE_BUILD ( bowtie_fasta )
BOWTIE_BUILD ( fasta )
NGSCHECKMATE_PATTERNGENERATOR ( GAWK_BED.out.output, fasta, BOWTIE_BUILD.out.index )

NGSCHECKMATE_FASTQ ( input, NGSCHECKMATE_PATTERNGENERATOR.out.pt )
Expand All @@ -63,8 +61,6 @@ workflow test_ngscheckmate_fastq_paired {
file(params.test_data['sarscov2']['genome']['genome_fasta_fai'], checkIfExists: true)
]

bowtie_fasta = file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)

// create a bed file representing the entire genome from the fasta index
GAWK_FAI(fasta_fai, [])

Expand All @@ -75,7 +71,7 @@ workflow test_ngscheckmate_fastq_paired {
GAWK_BED(BEDTOOLS_MAKEWINDOWS.out.bed, [])

// generate the .pt file for ngscheckmate to use
BOWTIE_BUILD ( bowtie_fasta )
BOWTIE_BUILD ( fasta )
NGSCHECKMATE_PATTERNGENERATOR ( GAWK_BED.out.output, fasta, BOWTIE_BUILD.out.index )

NGSCHECKMATE_FASTQ ( input, NGSCHECKMATE_PATTERNGENERATOR.out.pt )
Expand Down
4 changes: 1 addition & 3 deletions tests/modules/nf-core/ngscheckmate/patterngenerator/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,9 +17,7 @@ workflow test_ngscheckmate_patterngenerator {
file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)
]

bowtie_fasta = file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)

BOWTIE_BUILD ( bowtie_fasta )
BOWTIE_BUILD ( fasta )

NGSCHECKMATE_PATTERNGENERATOR ( input, fasta, BOWTIE_BUILD.out.index )
}
5 changes: 1 addition & 4 deletions tests/modules/nf-core/ngscheckmate/vafncm/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -46,8 +46,6 @@ workflow test_ngscheckmate_vafncm {
file(params.test_data['sarscov2']['genome']['genome_fasta_fai'], checkIfExists: true)
]

bowtie_fasta = file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)

// create a bed file representing the entire genome from the fasta index
GAWK_FAI(fasta_fai, [])

Expand All @@ -58,11 +56,10 @@ workflow test_ngscheckmate_vafncm {
GAWK_BED(BEDTOOLS_MAKEWINDOWS.out.bed, [])

// generate the .pt file for ngscheckmate to use
BOWTIE_BUILD ( bowtie_fasta )
BOWTIE_BUILD ( fasta )
NGSCHECKMATE_PATTERNGENERATOR ( GAWK_BED.out.output, fasta, BOWTIE_BUILD.out.index )

NGSCHECKMATE_FASTQ ( channel.fromList(input), NGSCHECKMATE_PATTERNGENERATOR.out.pt )
ch_vafs = NGSCHECKMATE_FASTQ.out.vaf.map{it[1]}.collect().map{files -> [[id:"combined"], files]}
ch_vafs.view()
NGSCHECKMATE_VAFNCM(ch_vafs )
}
4 changes: 1 addition & 3 deletions tests/subworkflows/nf-core/fastq_ngscheckmate/main.nf
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,6 @@ workflow test_fastq_ngscheckmate {
file(params.test_data['sarscov2']['genome']['genome_fasta_fai'], checkIfExists: true)
]

bowtie_fasta = file(params.test_data['sarscov2']['genome']['genome_fasta'], checkIfExists: true)

// create a bed file representing the entire genome from the fasta index
GAWK_FAI(fasta_fai, [])

Expand All @@ -57,7 +55,7 @@ workflow test_fastq_ngscheckmate {
GAWK_BED(BEDTOOLS_MAKEWINDOWS.out.bed, [])

// generate the .pt file for ngscheckmate to use
BOWTIE_BUILD ( bowtie_fasta )
BOWTIE_BUILD ( fasta )
NGSCHECKMATE_PATTERNGENERATOR ( GAWK_BED.out.output, fasta, BOWTIE_BUILD.out.index )

FASTQ_NGSCHECKMATE ( channel.fromList(input), NGSCHECKMATE_PATTERNGENERATOR.out.pt )
Expand Down
Loading