bowtie2: remove code duplication

[matthdsm]

Remove code duplication from bowtie2 align

PR checklist

Closes #XXX

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
• If you've added a new tool - have you followed the module conventions in the contribution docs
• If necessary, include test data in your PR.
• Remove all TODO statements.
• Emit the versions.yml file.
• Follow the naming conventions.
• Follow the parameters requirements.
• Follow the input/output options guidelines.
• Add a resource label
• Use BioConda and BioContainers if possible to fulfil software requirements.
• Ensure that the test works with either Docker / Singularity. Conda CI tests can be quite flaky:
  • PROFILE=docker pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
  • PROFILE=singularity pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware
  • PROFILE=conda pytest --tag <MODULE> --symlink --keep-workflow-wd --git-aware

[drpatelh]

These have been all deleted because the BAM won't be present without the index files? Is there any harm in leaving them in as a sanity check?

[matthdsm]

This file was auto-generated by create-test-yaml, so I've no vested interest in the content

[matthdsm]

I also have no idea why the index files have ever been included in the output from bowtie2 align

[drpatelh]

Again, no idea what happened here 😏 Does /output/bowtie2/bowtie2/ exist when you run pytest?

[matthdsm]

.
├── 40
│   └── fde8dfbf23b62799ec1c8fe1edc520
│       ├── bowtie2 -> /tmp/pytest_workflow_um_6gy_g/bowtie2_align_test_bowtie2_align_single_end/work/43/d87f151078c05ee619a09f9adaae31/bowtie2
│       ├── test.bam
│       ├── test.bowtie2.log
│       ├── test_1.fastq.gz -> /tmp/pytest_workflow_um_6gy_g/bowtie2_align_test_bowtie2_align_single_end/work/stage/07/142d3883569478136b1e5e6536ffbd/test_1.fastq.gz
│       └── versions.yml
├── 43
│   └── d87f151078c05ee619a09f9adaae31
│       ├── bowtie2
│       │   ├── genome.1.bt2
│       │   ├── genome.2.bt2
│       │   ├── genome.3.bt2
│       │   ├── genome.4.bt2
│       │   ├── genome.rev.1.bt2
│       │   └── genome.rev.2.bt2
│       ├── genome.fasta -> /tmp/pytest_workflow_um_6gy_g/bowtie2_align_test_bowtie2_align_single_end/work/stage/7a/1e46d5cd1dad747bdec96651090752/genome.fasta
│       └── versions.yml
└── stage
    ├── 07
    │   └── 142d3883569478136b1e5e6536ffbd
    │       └── test_1.fastq.gz
    └── 7a
        └── 1e46d5cd1dad747bdec96651090752
            └── genome.fasta

[drpatelh]

Sorry, in the output directory not the work directory.

[matthdsm]

output/
└── bowtie2
    ├── bowtie2 -> /tmp/pytest_workflow_82vq1t4y/bowtie2_align_test_bowtie2_align_single_end/work/8b/b0d4b9040c8329f51428d5e92b6812/bowtie2
    │   ├── genome.1.bt2
    │   ├── genome.2.bt2
    │   ├── genome.3.bt2
    │   ├── genome.4.bt2
    │   ├── genome.rev.1.bt2
    │   └── genome.rev.2.bt2
    ├── test.bam -> /tmp/pytest_workflow_82vq1t4y/bowtie2_align_test_bowtie2_align_single_end/work/e7/61e8210898ae529cbed8912da0512a/test.bam
    ├── test.bowtie2.log -> /tmp/pytest_workflow_82vq1t4y/bowtie2_align_test_bowtie2_align_single_end/work/e7/61e8210898ae529cbed8912da0512a/test.bowtie2.log
    └── versions.yml -> /tmp/pytest_workflow_82vq1t4y/bowtie2_align_test_bowtie2_align_single_end/work/e7/61e8210898ae529cbed8912da0512a/versions.yml

[drpatelh]

Ok. So the indices are published so why are they being removed from the test.yml?🤔 This scares me a little just in case there are instances where create-test-yml is missing files it shouldn't. Worth checking again before the next release.

Would you mind creating another issue please with some commands to reproduce?

Thanks!

[JoseEspinosa]

Nice @matthdsm 😄
Just wanted to comment about the quoting since I saw you changed it completely and @drpatelh has already nicely reviewed the module.
The rule of thumb I used for the quoting in Nextflow (also taking into account examples on Nextflow docs) is that if you don't interpolate any variable inside a declaration, e.g. the pointers to the container, I then use single quotes, example in docs here.
Also, in the case of the declaration of the output channels if you don't include any variable in its declaration I also would suggest to use single quotes, see below:
tuple val(meta), path('*.bam') , emit: bam see here
while:
tuple val(meta), path("${x}.bam") , emit: bam see here
I saw there is an open PR on tools, probably will be better to discuss it there, will do

[matthdsm]

I went with the double quotes based of the comment by @ewels, who recommends double quotes throughout. I'm not againt single quotes, but I like uniformity. Looks cleaner IMO

[JoseEspinosa]

I went with the double quotes based of the comment by @ewels, who recommends double quotes throughout. I'm not againt single quotes, but I like uniformity. Looks cleaner IMO

👍 Fair enough, I just saw @ewels comment and add my point of view on the tools issue.

[drpatelh]

Yeah, I saw those changes too and chose to turn a blind eye. If we do settle on double quotes everywhere then we should update all modules at some point.

[JoseEspinosa]

Yeah, I saw those changes too and chose to turn a blind eye. If we do settle on double quotes everywhere then we should update all modules at some point.

Personally, besides my comments above, I don't like double quotes everywhere, I guess everyone has his own obsessions 🤣
Anyway, if in the end we go for double quotes everywhere, you are right we will need to update the modules, which is another reason not to go for it 😄

[drpatelh]

Another one for you @chris-cheshire. You added these entries to test.yml for large indices but we don't have the corresponding entries in main.nf to actually test them? This is why all of these lines have automatically been deleted by create-test-yml.

[chris-cheshire]

oh do we need to though? they use the same entry point, they just change a parameter

[drpatelh]

Yep, the workflow in main.nf needs to be explicitly duplicated for each test case. Maybe you can add these in a follow up PR after this is merged? Can ping me to review.

[chris-cheshire]

yeah my bad will do

[drpatelh]

Thanks @matthdsm ! 🚀

As mentioned there, be great if you can create an issue for this so we can look into it properly.

[matthdsm]

issue created! Thanks for the reviews!

nf-core/tools#1564