Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fully implement annotation #35

Merged
merged 41 commits into from
May 25, 2023
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
6b863ec
Template update for nf-core/tools version 2.6
nvnieuwk Oct 7, 2022
3d7d169
Template update for nf-core/tools version 2.7.2
nvnieuwk Jan 26, 2023
bacea54
first steps
nvnieuwk Apr 14, 2023
4c6916f
fully implement the annotation
nvnieuwk Apr 17, 2023
363abf3
update test
nvnieuwk Apr 17, 2023
fe17d76
offer a lamb to the linting gods
nvnieuwk Apr 17, 2023
b551476
bug fix
nvnieuwk Apr 17, 2023
74df144
offer a virgin to the linting gods (hope this is enough)
nvnieuwk Apr 17, 2023
1aaa70f
Add dynamic VCFanno config creation
nvnieuwk Apr 19, 2023
c6ca962
go on a pilgrimage to the mountain of the linting gods and make a blo…
nvnieuwk Apr 19, 2023
81f2917
fix and update annotate test to stub
nvnieuwk Apr 19, 2023
8281284
update vep container on stub runs
nvnieuwk Apr 19, 2023
f2d632f
add postannotations support to the configs
nvnieuwk Apr 19, 2023
37aafe9
simplified the vcfanno config code and made it smarter
nvnieuwk Apr 19, 2023
f1342e5
added new fields from the annotsv vcf to the default configs
nvnieuwk Apr 19, 2023
3d25cc6
added a param for the default annotsv file name
nvnieuwk Apr 19, 2023
b3d3a2c
fix small bug with param
nvnieuwk Apr 19, 2023
1a67261
fix small bug with param
nvnieuwk Apr 19, 2023
96a3dee
Add all default configs
nvnieuwk Apr 19, 2023
33123c5
sacrificing my firstborn child to the linting gods
nvnieuwk Apr 19, 2023
b2a9c91
Moved the unused configs to a separate folder
nvnieuwk Apr 19, 2023
6c5f4e1
update configs
nvnieuwk Apr 20, 2023
ab2756f
update vep config
nvnieuwk Apr 20, 2023
47a12fc
add option to annotate on small variants with AnnotSV (real test file…
nvnieuwk Apr 20, 2023
732dc8b
Remove unused VEP parameters
nvnieuwk Apr 20, 2023
10b11f7
fix issues
nvnieuwk Apr 20, 2023
799e4f9
add tests for small variants vcfs
nvnieuwk Apr 20, 2023
3afdab1
add an option to also output the annotsv outputs
nvnieuwk Apr 21, 2023
0cd4c3d
Merge wrongly placed commits (#40)
nvnieuwk May 8, 2023
db84300
Template update for nf-core/tools version 2.8
nvnieuwk May 11, 2023
8cc579e
Merge branch 'TEMPLATE' into template-v2.8
nvnieuwk May 11, 2023
6638f54
black linting
nvnieuwk May 11, 2023
7f8d565
update modules
nvnieuwk May 11, 2023
a0bde4d
update modules
nvnieuwk May 11, 2023
e9bb263
fix local modules
nvnieuwk May 11, 2023
0e40f59
bump
nvnieuwk May 11, 2023
48bed31
eclint
nvnieuwk May 11, 2023
dbb159b
Merge pull request #42 from CenterForMedicalGeneticsGhent/template-v2.8
nvnieuwk May 11, 2023
9d69532
update genome in docs
nvnieuwk May 25, 2023
4b626f8
update vcfanno configs retina and rpe
nvnieuwk May 25, 2023
f023b52
fix rpe bed file in config
nvnieuwk May 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ trim_trailing_whitespace = true
indent_size = 4
indent_style = space

[*.{md,yml,yaml,html,css,scss,js,cff}]
[*.{md,yml,yaml,html,css,scss,js}]
indent_size = 2

# These files are edited and tested upstream in nf-core/modules
Expand Down
5 changes: 4 additions & 1 deletion .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,10 @@ body:

* Hardware _(eg. HPC, Desktop, Cloud)_
* Executor _(eg. slurm, local, awsbatch)_
* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter or Charliecloud)_

* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud,
or Apptainer)_

* OS _(eg. CentOS Linux, macOS, Linux Mint)_

* Version of CenterForMedicalGeneticsGhent/nf-cmgg-structural _(eg. 1.1, 1.5,
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/branch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
- name: Check PRs
if: github.repository == 'CenterForMedicalGeneticsGhent/nf-cmgg-structural'
run: |
{ [[ ${{github.event.pull_request.head.repo.full_name }} == CenterForMedicalGeneticsGhent/nf-cmgg-structural ]] && [[ $GITHUB_HEAD_REF = "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]
{ [[ ${{github.event.pull_request.head.repo.full_name }} == CenterForMedicalGeneticsGhent/nf-cmgg-structural ]] && [[ $GITHUB_HEAD_REF == "dev" ]]; } || [[ $GITHUB_HEAD_REF == "patch" ]]

# If the above check failed, post a comment on the PR explaining the failure
# NOTE - this doesn't currently work if the PR is coming from a fork, due to limitations in GitHub actions secrets
Expand Down
24 changes: 24 additions & 0 deletions .github/workflows/clean-up.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: "Close user-tagged issues and PRs"
on:
schedule:
- cron: "0 0 * * 0" # Once a week

jobs:
clean-up:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v7
with:
stale-issue-message: "This issue has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment otherwise this issue will be closed in 20 days."
stale-pr-message: "This PR has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment if it is still useful."
close-issue-message: "This issue was closed because it has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor and then staled for 20 days with no activity."
days-before-stale: 30
days-before-close: 20
days-before-pr-close: -1
any-of-labels: "awaiting-changes,awaiting-feedback"
exempt-issue-labels: "WIP"
exempt-pr-labels: "WIP"
repo-token: "${{ secrets.GITHUB_TOKEN }}"
32 changes: 31 additions & 1 deletion .github/workflows/linting.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,36 @@ jobs:
- name: Run Prettier --check
run: prettier --check ${GITHUB_WORKSPACE}

PythonBlack:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3

- name: Check code lints with Black
uses: psf/black@stable

# If the above check failed, post a comment on the PR explaining the failure
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
with:
message: |
## Python linting (`black`) is failing

To keep the code consistent with lots of contributors, we run automated code consistency checks.
To fix this CI test, please run:

* Install [`black`](https://black.readthedocs.io/en/stable/): `pip install black`
* Fix formatting errors in your pipeline: `black .`

Once you push these changes the test should pass, and you can hide this comment :+1:

We highly recommend setting up Black in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!

Thanks again for your contribution!
repo-token: ${{ secrets.GITHUB_TOKEN }}
allow-repeats: false

nf-core:
runs-on: ubuntu-latest
steps:
Expand All @@ -48,7 +78,7 @@ jobs:

- uses: actions/setup-python@v4
with:
python-version: "3.7"
python-version: "3.8"
architecture: "x64"

- name: Install dependencies
Expand Down
3 changes: 3 additions & 0 deletions .nf-core.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ lint:
- assets/nf-core-nf-cmgg-structural_logo_light.png
- docs/images/nf-core-nf-cmgg-structural_logo_light.png
- docs/images/nf-core-nf-cmgg-structural_logo_dark.png
- assets/nf-core-nf-cmgg-structural_logo_light.png
- docs/images/nf-core-nf-cmgg-structural_logo_light.png
- docs/images/nf-core-nf-cmgg-structural_logo_dark.png
- .github/ISSUE_TEMPLATE/config.yml
- .github/workflows/awstest.yml
- .github/workflows/awsfulltest.yml
Expand Down
5 changes: 5 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
repos:
- repo: https://github.com/pre-commit/mirrors-prettier
rev: "v2.7.1"
hooks:
- id: prettier
38 changes: 16 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,34 +12,28 @@

The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html) implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from [nf-core/modules](https://github.com/nf-core/modules) in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community!

## Pipeline summary

![metro map](docs/images/metro_map.png)

## Quick Start

1. Install [`Nextflow`](https://www.nextflow.io/docs/latest/getstarted.html#installation) (`>=22.10.5`)

2. Install any of [`Docker`](https://docs.docker.com/engine/installation/), [`Singularity`](https://www.sylabs.io/guides/3.0/user-guide/) (you can follow [this tutorial](https://singularity-tutorial.github.io/01-installation/)), [`Podman`](https://podman.io/), [`Shifter`](https://nersc.gitlab.io/development/shifter/how-to-use/) or [`Charliecloud`](https://hpc.github.io/charliecloud/) for full pipeline reproducibility _(you can use [`Conda`](https://conda.io/miniconda.html) both to install Nextflow itself and also to manage software within pipelines. Please only use it within pipelines as a last resort; see [docs](https://nf-co.re/usage/configuration#basic-configuration-profiles))_.

3. Download the pipeline and test it on a minimal dataset with a single command:

```bash
nextflow run CenterForMedicalGeneticsGhent/nf-cmgg-structural -profile test,YOURPROFILE --outdir <OUTDIR>
```
## Usage

Note that some form of configuration will be needed so that Nextflow knows how to fetch the required software. This is usually done in the form of a config profile (`YOURPROFILE` in the example command above). You can chain multiple config profiles in a comma-separated string.
> **Note**
> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how
> to set-up Nextflow. Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline)
> with `-profile test` before running the workflow on actual data.

> - The pipeline comes with config profiles called `docker`, `singularity`, `podman`, `shifter`, `charliecloud` and `conda` which instruct the pipeline to use the named tool for software management. For example, `-profile test,docker`.
> - Please check [nf-core/configs](https://github.com/nf-core/configs#documentation) to see if a custom config file to run nf-core pipelines already exists for your Institute. If so, you can simply use `-profile <institute>` in your command. This will enable either `docker` or `singularity` and set the appropriate execution settings for your local compute environment.
> - If you are using `singularity`, please use the [`nf-core download`](https://nf-co.re/tools/#downloading-pipelines-for-offline-use) command to download images first, before running the pipeline. Setting the [`NXF_SINGULARITY_CACHEDIR` or `singularity.cacheDir`](https://www.nextflow.io/docs/latest/singularity.html?#singularity-docker-hub) Nextflow options enables you to store and re-use the images from a central location for future pipeline runs.
> - If you are using `conda`, it is highly recommended to use the [`NXF_CONDA_CACHEDIR` or `conda.cacheDir`](https://www.nextflow.io/docs/latest/conda.html) settings to store the environments in a central location for future pipeline runs.
Now, you can run the pipeline using:

4. Start running your own analysis!
```bash
nextflow run CenterForMedicalGeneticsGhent/nf-cmgg-structural \
-profile <docker/singularity/.../institute> \
--input samplesheet.csv \
--outdir <OUTDIR>
```

```bash
nextflow run CenterForMedicalGeneticsGhent/nf-cmgg-structural --input samplesheet.csv --outdir <OUTDIR> --genome GRCh38 -profile <docker/singularity/podman/shifter/charliecloud/conda/institute>
```
> **Warning:**
> Please provide pipeline parameters via the CLI or Nextflow `-params-file` option. Custom config files including those
> provided by the `-c` Nextflow option can be used to provide any configuration _**except for parameters**_;
> see [docs](https://nf-co.re/usage/configuration#custom-configuration-files).

## Documentation

Expand Down
Loading