Fix problems with empty annotation intersection

[nictru]

This commit changes several things:

• Make sure pipeline terminates in an ordered way if no circRNAs are detected
• Rework the last few steps of the BSJ detection
• Rework the structure of the output directory
• Update the documentation

[github-actions]

nf-core lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 90172a0

+| ✅ 235 tests passed       |+
!| ❗   1 tests had warnings |!

❗ Test warnings:

• readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).

✅ Tests passed:

• files_exist - File found: .gitattributes
• files_exist - File found: .gitignore
• files_exist - File found: .nf-core.yml
• files_exist - File found: .editorconfig
• files_exist - File found: .prettierignore
• files_exist - File found: .prettierrc.yml
• files_exist - File found: CHANGELOG.md
• files_exist - File found: CITATIONS.md
• files_exist - File found: CODE_OF_CONDUCT.md
• files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
• files_exist - File found: nextflow_schema.json
• files_exist - File found: nextflow.config
• files_exist - File found: README.md
• files_exist - File found: .github/.dockstore.yml
• files_exist - File found: .github/CONTRIBUTING.md
• files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
• files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
• files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
• files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
• files_exist - File found: .github/workflows/branch.yml
• files_exist - File found: .github/workflows/ci.yml
• files_exist - File found: .github/workflows/linting_comment.yml
• files_exist - File found: .github/workflows/linting.yml
• files_exist - File found: assets/email_template.html
• files_exist - File found: assets/email_template.txt
• files_exist - File found: assets/sendmail_template.txt
• files_exist - File found: assets/nf-core-circrna_logo_light.png
• files_exist - File found: conf/modules.config
• files_exist - File found: conf/test.config
• files_exist - File found: conf/test_full.config
• files_exist - File found: docs/images/nf-core-circrna_logo_light.png
• files_exist - File found: docs/images/nf-core-circrna_logo_dark.png
• files_exist - File found: docs/output.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/usage.md
• files_exist - File found: main.nf
• files_exist - File found: assets/multiqc_config.yml
• files_exist - File found: conf/base.config
• files_exist - File found: conf/igenomes.config
• files_exist - File found: .github/workflows/awstest.yml
• files_exist - File found: .github/workflows/awsfulltest.yml
• files_exist - File found: modules.json
• files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
• files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
• files_exist - File not found check: .github/workflows/push_dockerhub.yml
• files_exist - File not found check: .markdownlint.yml
• files_exist - File not found check: .nf-core.yaml
• files_exist - File not found check: .yamllint.yml
• files_exist - File not found check: bin/markdown_to_html.r
• files_exist - File not found check: conf/aws.config
• files_exist - File not found check: docs/images/nf-core-circrna_logo.png
• files_exist - File not found check: lib/Checks.groovy
• files_exist - File not found check: lib/Completion.groovy
• files_exist - File not found check: lib/NfcoreTemplate.groovy
• files_exist - File not found check: lib/Utils.groovy
• files_exist - File not found check: lib/Workflow.groovy
• files_exist - File not found check: lib/WorkflowMain.groovy
• files_exist - File not found check: lib/WorkflowCircrna.groovy
• files_exist - File not found check: parameters.settings.json
• files_exist - File not found check: pipeline_template.yml
• files_exist - File not found check: Singularity
• files_exist - File not found check: lib/nfcore_external_java_deps.jar
• files_exist - File not found check: .travis.yml
• nextflow_config - Config variable found: manifest.name
• nextflow_config - Config variable found: manifest.nextflowVersion
• nextflow_config - Config variable found: manifest.description
• nextflow_config - Config variable found: manifest.version
• nextflow_config - Config variable found: manifest.homePage
• nextflow_config - Config variable found: timeline.enabled
• nextflow_config - Config variable found: trace.enabled
• nextflow_config - Config variable found: report.enabled
• nextflow_config - Config variable found: dag.enabled
• nextflow_config - Config variable found: process.cpus
• nextflow_config - Config variable found: process.memory
• nextflow_config - Config variable found: process.time
• nextflow_config - Config variable found: params.outdir
• nextflow_config - Config variable found: params.input
• nextflow_config - Config variable found: params.validationShowHiddenParams
• nextflow_config - Config variable found: params.validationSchemaIgnoreParams
• nextflow_config - Config variable found: manifest.mainScript
• nextflow_config - Config variable found: timeline.file
• nextflow_config - Config variable found: trace.file
• nextflow_config - Config variable found: report.file
• nextflow_config - Config variable found: dag.file
• nextflow_config - Config variable (correctly) not found: params.nf_required_version
• nextflow_config - Config variable (correctly) not found: params.container
• nextflow_config - Config variable (correctly) not found: params.singleEnd
• nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
• nextflow_config - Config variable (correctly) not found: params.name
• nextflow_config - Config variable (correctly) not found: params.enable_conda
• nextflow_config - Config timeline.enabled had correct value: true
• nextflow_config - Config report.enabled had correct value: true
• nextflow_config - Config trace.enabled had correct value: true
• nextflow_config - Config dag.enabled had correct value: true
• nextflow_config - Config manifest.name began with nf-core/
• nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
• nextflow_config - Config dag.file ended with .html
• nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
• nextflow_config - Config manifest.version ends in dev: dev
• nextflow_config - Config params.custom_config_version is set to master
• nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
• nextflow_config - Lines for loading custom profiles found
• nextflow_config - nextflow.config contains configuration profile test
• nextflow_config - Config default value correct: params.tools= circexplorer2
• nextflow_config - Config default value correct: params.bsj_reads= 1
• nextflow_config - Config default value correct: params.tool_filter= 1
• nextflow_config - Config default value correct: params.save_intermediates= false
• nextflow_config - Config default value correct: params.exon_boundary= 0
• nextflow_config - Config default value correct: params.sjdboverhang= 100
• nextflow_config - Config default value correct: params.chimJunctionOverhangMin= 10
• nextflow_config - Config default value correct: params.alignSJDBoverhangMin= 10
• nextflow_config - Config default value correct: params.limitSjdbInsertNsj= 1000000
• nextflow_config - Config default value correct: params.chimSegmentMin= 10
• nextflow_config - Config default value correct: params.seglen= 25
• nextflow_config - Config default value correct: params.min_intron= 20
• nextflow_config - Config default value correct: params.max_intron= 1000000
• nextflow_config - Config default value correct: params.min_map_len= 40
• nextflow_config - Config default value correct: params.min_fusion_distance= 200
• nextflow_config - Config default value correct: params.bootstrap_samples= 30
• nextflow_config - Config default value correct: params.save_unaligned= false
• nextflow_config - Config default value correct: params.save_reference= true
• nextflow_config - Config default value correct: params.hisat2_build_memory= 200.GB
• nextflow_config - Config default value correct: params.skip_trimming= false
• nextflow_config - Config default value correct: params.save_trimmed= false
• nextflow_config - Config default value correct: params.skip_fastqc= false
• nextflow_config - Config default value correct: params.min_trimmed_reads= 10000
• nextflow_config - Config default value correct: params.custom_config_version= master
• nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
• nextflow_config - Config default value correct: params.max_cpus= 50
• nextflow_config - Config default value correct: params.max_memory= 300.GB
• nextflow_config - Config default value correct: params.max_time= 240.h
• nextflow_config - Config default value correct: params.publish_dir_mode= copy
• nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
• nextflow_config - Config default value correct: params.validate_params= true
• files_unchanged - .gitattributes matches the template
• files_unchanged - .prettierrc.yml matches the template
• files_unchanged - CODE_OF_CONDUCT.md matches the template
• files_unchanged - LICENSE matches the template
• files_unchanged - .github/.dockstore.yml matches the template
• files_unchanged - .github/CONTRIBUTING.md matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
• files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
• files_unchanged - .github/workflows/branch.yml matches the template
• files_unchanged - .github/workflows/linting_comment.yml matches the template
• files_unchanged - .github/workflows/linting.yml matches the template
• files_unchanged - assets/email_template.html matches the template
• files_unchanged - assets/email_template.txt matches the template
• files_unchanged - assets/sendmail_template.txt matches the template
• files_unchanged - assets/nf-core-circrna_logo_light.png matches the template
• files_unchanged - docs/images/nf-core-circrna_logo_light.png matches the template
• files_unchanged - docs/images/nf-core-circrna_logo_dark.png matches the template
• files_unchanged - docs/README.md matches the template
• files_unchanged - .gitignore matches the template
• files_unchanged - .prettierignore matches the template
• actions_ci - '.github/workflows/ci.yml' is triggered on expected events
• actions_ci - '.github/workflows/ci.yml' checks minimum NF version
• actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
• actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
• actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
• readme - README Nextflow minimum version badge matched config. Badge: 23.04.0, Config: 23.04.0
• pipeline_todos - No TODO strings found
• pipeline_name_conventions - Name adheres to nf-core convention
• template_strings - Did not find any Jinja template strings (393 files)
• schema_lint - Schema lint passed
• schema_lint - Schema title + description lint passed
• schema_lint - Input mimetype lint passed: 'text/csv'
• schema_params - Schema matched params returned from nextflow config
• system_exit - No System.exit calls found
• actions_schema_validation - Workflow validation passed: release-announcements.yml
• actions_schema_validation - Workflow validation passed: awstest.yml
• actions_schema_validation - Workflow validation passed: awsfulltest.yml
• actions_schema_validation - Workflow validation passed: ci.yml
• actions_schema_validation - Workflow validation passed: linting_comment.yml
• actions_schema_validation - Workflow validation passed: fix-linting.yml
• actions_schema_validation - Workflow validation passed: clean-up.yml
• actions_schema_validation - Workflow validation passed: download_pipeline.yml
• actions_schema_validation - Workflow validation passed: linting.yml
• actions_schema_validation - Workflow validation passed: branch.yml
• merge_markers - No merge markers found in pipeline files
• modules_json - Only installed modules found in modules.json
• multiqc_config - assets/multiqc_config.yml found and not ignored.
• multiqc_config - assets/multiqc_config.yml contains report_section_order
• multiqc_config - assets/multiqc_config.yml contains export_plots
• multiqc_config - assets/multiqc_config.yml contains report_comment
• multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
• multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
• multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
• modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
• base_config - conf/base.config found and not ignored.
• modules_config - conf/modules.config found and not ignored.
• modules_config - CUSTOM_DUMPSOFTWAREVERSIONS found in conf/modules.config and Nextflow scripts.
• modules_config - CAT_FASTQ found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_FAIDX found in conf/modules.config and Nextflow scripts.
• modules_config - CLEAN_FASTA found in conf/modules.config and Nextflow scripts.
• modules_config - GTFFILTER found in conf/modules.config and Nextflow scripts.
• modules_config - SEQKIT_SPLIT found in conf/modules.config and Nextflow scripts.
• modules_config - BOWTIE_BUILD found in conf/modules.config and Nextflow scripts.
• modules_config - BOWTIE2_BUILD found in conf/modules.config and Nextflow scripts.
• modules_config - BWA_INDEX found in conf/modules.config and Nextflow scripts.
• modules_config - HISAT2_EXTRACTSPLICESITES found in conf/modules.config and Nextflow scripts.
• modules_config - HISAT2_BUILD found in conf/modules.config and Nextflow scripts.
• modules_config - STAR_GENOMEGENERATE found in conf/modules.config and Nextflow scripts.
• modules_config - FILTER_BSJS found in conf/modules.config and Nextflow scripts.
• modules_config - MASK_SCORES found in conf/modules.config and Nextflow scripts.
• modules_config - CONCAT_TOOLS_PER_SAMPLE found in conf/modules.config and Nextflow scripts.
• modules_config - COUNT_TOOLS found in conf/modules.config and Nextflow scripts.
• modules_config - FILTER_MIN_TOOLS found in conf/modules.config and Nextflow scripts.
• modules_config - CONCAT_SAMPLES found in conf/modules.config and Nextflow scripts.
• modules_config - UPSET_SAMPLES found in conf/modules.config and Nextflow scripts.
• modules_config - UPSET_ALL found in conf/modules.config and Nextflow scripts.
• modules_config - FASTA_COMBINED found in conf/modules.config and Nextflow scripts.
• modules_config - FASTA_PER_SAMPLE found in conf/modules.config and Nextflow scripts.
• modules_config - FASTA_PER_SAMPLE_TOOL found in conf/modules.config and Nextflow scripts.
• modules_config - ADD_BACKSPLICE found in conf/modules.config and Nextflow scripts.
• modules_config - COMBINE_TRANSCRIPTOME_GTFS found in conf/modules.config and Nextflow scripts.
• modules_config - EXCLUDE_OVERLONG_TRANSCRIPTS found in conf/modules.config and Nextflow scripts.
• modules_config - TRANSCRIPTOME found in conf/modules.config and Nextflow scripts.
• modules_config - MARK_CIRCULAR found in conf/modules.config and Nextflow scripts.
• modules_config - PSIRC_INDEX found in conf/modules.config and Nextflow scripts.
• modules_config - PSIRC_QUANT found in conf/modules.config and Nextflow scripts.
• modules_config - CUSTOM_TX2GENE found in conf/modules.config and Nextflow scripts.
• modules_config - TXIMETA_TXIMETA found in conf/modules.config and Nextflow scripts.
• modules_config - TXIMETA_TXIMPORT found in conf/modules.config and Nextflow scripts.
• modules_config - MERGE_EXPERIMENTS found in conf/modules.config and Nextflow scripts.
• modules_config - TARGETSCAN_DATABASE found in conf/modules.config and Nextflow scripts.
• modules_config - TARGETSCAN found in conf/modules.config and Nextflow scripts.
• modules_config - MIRANDA found in conf/modules.config and Nextflow scripts.
• modules_config - MIRNA_TARGETS found in conf/modules.config and Nextflow scripts.
• modules_config - CIRCTEST_PREPARE found in conf/modules.config and Nextflow scripts.
• modules_config - CIRCTEST_CIRCTEST found in conf/modules.config and Nextflow scripts.
• nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
• nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 2.14.1

Run details

• nf-core/tools version 2.14.1
• Run at 2024-08-06 12:40:25

[ZabalaAitor]

I ran 4 samples (2 samples with circRNA detection and 2 without detection).

Previous results:
HC01: 2 circles
HC02: 1 circle
HC04: 0 circles
RR05: 0 circles

New results:
HC01: 3 circles
HC02: 2 circles
HC04: 1 circle
RR05: 0 circles

Additionally, I encountered the following error:

ERROR ~ Error executing process > 'NFCORE_CIRCRNA:CIRCRNA:CIRCRNA_DISCOVERY:ANNOTATION:ANNOTATE (RR05:circrna_finder)'

Caused by:
  Process `NFCORE_CIRCRNA:CIRCRNA:CIRCRNA_DISCOVERY:ANNOTATION:ANNOTATE (RR05:circrna_finder)` terminated with an error exit status (1)

I have attached the output file (slurm-505469.txt) which details the error (the specified working folder has the RR05.intersect.bed file empty).

[nictru]

@mashehu you really see everything, very impressive