Feat/init ci nf test #580

sateeshperi · 2023-05-09T00:53:03Z

Summary

✨ Add NF-TEST pipeline end-to-end tests for existing CI tests
NF-TEST pipeline Assertions - ✨ Featuring NF-TEST Snapshots, new File().exists() and !new File().exists() checks
🛠️ tests/pipeline/UTILS.groovy for local groovy functions used in tests
🤖 GitHub Actions CI - pull_request to dev tests with NXF_VER latest-everything
🤖 GitHub Actions CI - pull_request to master tests with NXF_VER 22.10.1 & latest-everything

github-actions · 2023-05-09T00:54:48Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 91f1b4e

+| ✅ 154 tests passed       |+
#| ❔   3 tests were ignored |#
!| ❗   2 tests had warnings |!

❗ Test warnings:

readme - README did not have a Nextflow minimum version badge.
schema_lint - Parameter input is not defined in the correct subschema (input_output_options)

❔ Tests ignored:

files_exist - File is ignored: conf/igenomes.config
files_unchanged - File ignored due to lint config: .gitattributes
actions_ci - actions_ci

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .editorconfig
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-ampliseq_logo_light.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-ampliseq_logo_light.png
files_exist - File found: docs/images/nf-core-ampliseq_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreSchema.groovy
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: lib/WorkflowAmpliseq.groovy
files_exist - File found: modules.json
files_exist - File found: pyproject.toml
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: docs/images/nf-core-ampliseq_logo.png
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.show_hidden_params
nextflow_config - Config variable found: params.schema_ignore_params
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: '2.6.0dev'
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-ampliseq_logo_light.png matches the template
files_unchanged - docs/images/nf-core-ampliseq_logo_light.png matches the template
files_unchanged - docs/images/nf-core-ampliseq_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - lib/NfcoreSchema.groovy matches the template
files_unchanged - lib/NfcoreTemplate.groovy matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
files_unchanged - pyproject.toml matches the template
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Zenodo placeholder was replaced with DOI.
pipeline_todos - No TODO strings found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (234 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: fix-linting.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: branch.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - 'assets/multiqc_config.yml' follows the ordering scheme of the minimally required plugins.
multiqc_config - 'assets/multiqc_config.yml' contains a matching 'report_comment'.
multiqc_config - 'assets/multiqc_config.yml' contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'

Run details

nf-core/tools version 2.8
Run at 2023-06-21 15:38:12

tests/pipeline/misc/custom_reftax.nf.test

edmundmiller

Swap profiles for config paths. Maybe make an issue about migrating to tags?

.github/workflows/ci.yml

tests/pipeline/misc/custom_reftax.nf.test

tests/pipeline/misc/doubleprimers.nf.test

.github/workflows/ci.yml

d4straub

I cannot really comment on most of it.

Did I understand correctly, that:

*.nf.test.snap hold md5sums that are checked for via *.nf.test with snapshot(<file>).match(<key>)?
in *.nf.test new File(<file>).exists() "only" checks for existence but not checksums

This seems like a good addition.

edit; one more thing:
why only using "latest-everything" configuration and not the minimum nf version?

.github/workflows/ci.yml

grst

LGTM! I like how you combine tags and profiles as this allows to run tests with and without nf-test.

.github/workflows/ci.yml

edmundmiller

Looks good to me! I feel like the CI is getting complicated, though.

.github/workflows/ci.yml

sateeshperi · 2023-06-20T21:57:29Z

Looks good to me! I feel like the CI is getting complicated, though.

😅 on the contrary I feel its streamlining much more now as we can kinda standardize the tests based on the profiles for all pipelines with multi profiles.

tests/pipeline/test.nf.test.snap

.github/workflows/ci.yml

d4straub

LGTM

sateeshperi self-assigned this May 10, 2023

sateeshperi added the enhancement New feature or request label May 10, 2023

sateeshperi requested a review from edmundmiller May 10, 2023 11:56

sateeshperi force-pushed the feat/init-ci-nf-test branch from a3e3c25 to f18878a Compare May 26, 2023 00:09

This comment was marked as outdated.

Sign in to view

sateeshperi requested review from edmundmiller and removed request for edmundmiller May 26, 2023 00:34

This comment was marked as outdated.

Sign in to view

sateeshperi requested a review from ewels May 26, 2023 01:27

sateeshperi commented May 26, 2023

View reviewed changes

tests/pipeline/misc/custom_reftax.nf.test Outdated Show resolved Hide resolved

This comment was marked as outdated.

Sign in to view

edmundmiller requested changes May 31, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

tests/pipeline/misc/custom_reftax.nf.test Outdated Show resolved Hide resolved

tests/pipeline/misc/doubleprimers.nf.test Outdated Show resolved Hide resolved

sateeshperi requested a review from edmundmiller June 5, 2023 01:45

grst reviewed Jun 14, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

sateeshperi force-pushed the feat/init-ci-nf-test branch from 364d086 to 1a4605c Compare June 19, 2023 22:51

sateeshperi commented Jun 19, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

sateeshperi force-pushed the feat/init-ci-nf-test branch 3 times, most recently from a25b327 to 1a662e5 Compare June 20, 2023 01:09

sateeshperi added 7 commits June 19, 2023 21:27

init-nf-tests-snapshots

3b94a03

update-ci-for-nf-tests

3a4559b

update-ci-for-nf-tests

bcfbd8d

update-dada2-snapshots

a853e1f

update-snapshots

eb09a40

update-pe-pacbio-snapshots

97e116d

update-snapshots-rm-nxf-ver

7802396

sateeshperi added 10 commits June 19, 2023 21:27

fix-cache-ref

0912655

fix-samplesheets-remote-paths

092b8cb

fix-samplesheets-remote-paths-sed

03c2dff

rm docker hardcode for test profiles

5577bfa

fix tag missing error

498aa9c

update test UTILS and test multi profile job matrix

173dc9d

test multi profile job matrix

aec9f6e

fix novaseq snap

7d4ee50

ci fail fast to false

3b591cf

update snapshots after rebase

4b093cd

sateeshperi force-pushed the feat/init-ci-nf-test branch from 1a662e5 to 4b093cd Compare June 20, 2023 02:08

sateeshperi requested review from d4straub, erikrikarddaniel and grst and removed request for ewels June 20, 2023 12:38

d4straub reviewed Jun 20, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

sateeshperi added 2 commits June 20, 2023 13:58

re-organize tests and use tags in ci job matrix

49579a9

lint ignore .gitattributes

179bf77

grst approved these changes Jun 20, 2023

View reviewed changes

sateeshperi added this to the 2.6.0 milestone Jun 20, 2023

edmundmiller reviewed Jun 20, 2023

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

edmundmiller approved these changes Jun 20, 2023

View reviewed changes

grst reviewed Jun 20, 2023

View reviewed changes

.github/workflows/ci.yml Outdated Show resolved Hide resolved

d4straub reviewed Jun 21, 2023

View reviewed changes

tests/pipeline/test.nf.test.snap Show resolved Hide resolved

CI - replace test data caching with git clone fetch-depth 1

91f1b4e

maxulysse reviewed Jun 21, 2023

View reviewed changes

.github/workflows/ci.yml Show resolved Hide resolved

d4straub approved these changes Jun 22, 2023

View reviewed changes

sateeshperi merged commit c675e10 into dev Jun 22, 2023

sateeshperi deleted the feat/init-ci-nf-test branch June 22, 2023 12:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feat/init ci nf test #580

Feat/init ci nf test #580

sateeshperi commented May 9, 2023 •

edited

Loading

github-actions bot commented May 9, 2023 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

edmundmiller left a comment

d4straub left a comment •

edited

Loading

grst left a comment

edmundmiller left a comment

sateeshperi commented Jun 20, 2023 •

edited

Loading

d4straub left a comment

Feat/init ci nf test #580

Feat/init ci nf test #580

Conversation

sateeshperi commented May 9, 2023 • edited Loading

github-actions bot commented May 9, 2023 • edited Loading

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as outdated.

edmundmiller left a comment

Choose a reason for hiding this comment

d4straub left a comment • edited Loading

Choose a reason for hiding this comment

grst left a comment

Choose a reason for hiding this comment

edmundmiller left a comment

Choose a reason for hiding this comment

sateeshperi commented Jun 20, 2023 • edited Loading

d4straub left a comment

Choose a reason for hiding this comment

sateeshperi commented May 9, 2023 •

edited

Loading

github-actions bot commented May 9, 2023 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️

d4straub left a comment •

edited

Loading

sateeshperi commented Jun 20, 2023 •

edited

Loading