Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add pre-commit docs for modules #1589

Merged
merged 2 commits into from
Feb 23, 2023
Merged

add pre-commit docs for modules #1589

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
269dde9
add pre-commit docs for modules
mashehu Feb 23, 2023
8271a32
[automated] Fix linting with Prettier
nf-core-bot Feb 23, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 58 additions & 52 deletions markdown/developers/modules.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,17 @@ If the module doesn't exist on `nf-core/modules`:

We have implemented a number of commands in the `nf-core/tools` package to make it incredibly easy for you to create and contribute your own modules to nf-core/modules.

1. Install the latest version of [`nf-core/tools`](https://github.com/nf-core/tools#installation) (`>=2.3`)
1. Install the latest version of [`nf-core/tools`](https://github.com/nf-core/tools#installation) (`>=2.7`)
2. Install [`Nextflow`](https://www.nextflow.io/docs/latest/getstarted.html#installation) (`>=21.04.0`)
3. Install any of [`Docker`](https://docs.docker.com/engine/installation/), [`Singularity`](https://www.sylabs.io/guides/3.0/user-guide/) or [`Conda`](https://conda.io/miniconda.html)
4. [Fork and clone the nf-core/modules repo locally](#uploading-to-nf-coremodules)
5. Set up git on your computer by adding a new git remote of the main nf-core git repo called `upstream`
4. Setup up [pre-commit](https://pre-commit.com/) (comes packaged with [`nf-core/tools`](https://github.com/nf-core/tools#installation), watch the [pre-commit bytesize talk](https://www.youtube.com/watch?v=08d6zv6zvdM&t=215) if you want to know more about it) to ensure that your code is linted and formatted correctly before you commit it to the repository

```bash
pre-commit install
```

5. [Fork and clone the nf-core/modules repo locally](#uploading-to-nf-coremodules)
6. Set up git on your computer by adding a new git remote of the main nf-core git repo called `upstream`

```bash
git remote add upstream https://github.com/nf-core/modules.git
Expand All @@ -61,7 +67,7 @@ We have implemented a number of commands in the `nf-core/tools` package to make
git checkout -b fastqc
```

6. Create a module using the [nf-core DSL2 module template](https://github.com/nf-core/tools/blob/master/nf_core/module-template/modules/main.nf):
7. Create a module using the [nf-core DSL2 module template](https://github.com/nf-core/tools/blob/master/nf_core/module-template/modules/main.nf):

```console
$ nf-core modules create fastqc --author @joebloggs --label process_low --meta
Expand Down Expand Up @@ -95,7 +101,7 @@ We have implemented a number of commands in the `nf-core/tools` package to make

2. [`./modules/fastqc/meta.yml`](https://github.com/nf-core/modules/blob/master/modules/fastqc/meta.yml)

This file will be used to store general information about the module and author details - the majority of which will already be auto-filled. However, you will need to add a brief description of the files defined in the `input` and `output` section of the main script since these will be unique to each module.
This file will be used to store general information about the module and author details - the majority of which will already be auto-filled. However, you will need to add a brief description of the files defined in the `input` and `output` section of the main script since these will be unique to each module. We check it's formatting and validity based on a [JSON schema](https://github.com/nf-core/modules/blob/master/.yaml-schema.json) during linting (and in the pre-commit hook).

3. [`./tests/modules/fastqc/main.nf`](https://github.com/nf-core/modules/blob/master/tests/modules/fastqc/main.nf)

Expand All @@ -122,7 +128,7 @@ We have implemented a number of commands in the `nf-core/tools` package to make

`md5sum` checks are the preferable choice of test to determine file changes, however, this may not be possible for all outputs generated by some tools e.g. if they include time stamps or command-related headers. Please do your best to avoid just checking for the file being present e.g. it may still be possible to check that the file contains the appropriate text snippets.

7. Create a yaml file containing information required for module unit testing
8. Create a yaml file containing information required for module unit testing

```console
$ nf-core modules create-test-yml
Expand Down Expand Up @@ -155,52 +161,52 @@ We have implemented a number of commands in the `nf-core/tools` package to make

> NB: See docs for [running tests manually](#running-tests-manually) if you would like to run the tests manually.

8. Check that the new module you've added follows the [new module guidelines](#new-module-guidelines-and-pr-review-checklist)

9. Lint the module locally to check that it adheres to nf-core guidelines before submission

```console
$ nf-core modules lint fastqc --dir .

,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'

nf-core/tools version 2.3dev0 - https://nf-co.re

INFO Linting modules repo: . __init__.py:15
INFO Linting module: fastqc __init__.py:163

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [!] 3 Test Warnings │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────┬──────────────────────────────────────────────────────────────┬──────────────────────────────────╮
│ Module name │ Test message │ File path │
├──────────────┼──────────────────────────────────────────────────────────────┼──────────────────────────────────┤
│ fastqc │ TODO string in meta.yml: #Add a description of the module... │ modules/nf-core/modules/fastqc/ │
│ fastqc │ TODO string in meta.yml: #Add a description and other det... │ modules/nf-core/modules/fastqc/ │
│ fastqc │ TODO string in meta.yml: #Add a description of all of the... │ modules/nf-core/modules/fastqc/ │
╰──────────────┴──────────────────────────────────────────────────────────────┴──────────────────────────────────╯
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [!] 1 Test Failed │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────┬──────────────────────────────────────────────────────────────┬──────────────────────────────────╮
│ Module name │ Test message │ File path │
├──────────────┼──────────────────────────────────────────────────────────────┼──────────────────────────────────┤
│ fastqc │ 'meta' map not emitted in output channel(s) │ modules/nf-core/modules/fastqc/ │
╰──────────────┴──────────────────────────────────────────────────────────────┴──────────────────────────────────╯
╭──────────────────────╮
│ LINT RESULTS SUMMARY │
├──────────────────────┤
│ [✔] 38 Tests Passed │
│ [!] 3 Test Warning │
│ [✗] 1 Test Failed │
╰──────────────────────╯
```

10. Once ready, the code can be pushed and a pull request (PR) created
9. Check that the new module you've added follows the [new module guidelines](#new-module-guidelines-and-pr-review-checklist)

10. Lint the module locally to check that it adheres to nf-core guidelines before submission

```console
$ nf-core modules lint fastqc --dir .

,--./,-.
___ __ __ __ ___ /,-._.--~\
|\ | |__ __ / ` / \ |__) |__ } {
| \| | \__, \__/ | \ |___ \`-._,-`-,
`._,._,'

nf-core/tools version 2.3dev0 - https://nf-co.re

INFO Linting modules repo: . __init__.py:15
INFO Linting module: fastqc __init__.py:163

╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [!] 3 Test Warnings │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────┬──────────────────────────────────────────────────────────────┬──────────────────────────────────╮
│ Module name │ Test message │ File path │
├──────────────┼──────────────────────────────────────────────────────────────┼──────────────────────────────────┤
│ fastqc │ TODO string in meta.yml: #Add a description of the module... │ modules/nf-core/modules/fastqc/ │
│ fastqc │ TODO string in meta.yml: #Add a description and other det... │ modules/nf-core/modules/fastqc/ │
│ fastqc │ TODO string in meta.yml: #Add a description of all of the... │ modules/nf-core/modules/fastqc/ │
╰──────────────┴──────────────────────────────────────────────────────────────┴──────────────────────────────────╯
╭────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ [!] 1 Test Failed │
╰────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭──────────────┬──────────────────────────────────────────────────────────────┬──────────────────────────────────╮
│ Module name │ Test message │ File path │
├──────────────┼──────────────────────────────────────────────────────────────┼──────────────────────────────────┤
│ fastqc │ 'meta' map not emitted in output channel(s) │ modules/nf-core/modules/fastqc/ │
╰──────────────┴──────────────────────────────────────────────────────────────┴──────────────────────────────────╯
╭──────────────────────╮
│ LINT RESULTS SUMMARY │
├──────────────────────┤
│ [✔] 38 Tests Passed │
│ [!] 3 Test Warning │
│ [✗] 1 Test Failed │
╰──────────────────────╯
```

11. Once ready, the code can be pushed and a pull request (PR) created

On a regular basis you can pull upstream changes into this branch and it is recommended to do so before pushing and creating a pull request - see below. Rather than merging changes directly from upstream the rebase strategy is recommended so that your changes are applied on top of the latest master branch from the nf-core repo. This can be performed as follows

Expand Down