Skip to content

Commit

Permalink
Merge pull request #1453 from nf-core/dev
Browse files Browse the repository at this point in the history
Dev > Master for v2.3 release
  • Loading branch information
sateeshperi authored Mar 15, 2022
2 parents e563811 + a744c35 commit b1ce471
Show file tree
Hide file tree
Showing 68 changed files with 1,762 additions and 608 deletions.
45 changes: 45 additions & 0 deletions .github/workflows/build_gitpod.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: nf-core gitpod build and push
# Builds the docker image for gitpod and pushes it to DockerHub

on:
push:
branches:
- 'master'
- 'dev'
paths:
- 'nf_core/gitpod/gitpod.Dockerfile'
- '.github/workflows/build_gitpod.yml'

jobs:
push_to_registry:
name: Push Docker image to Docker Hub
runs-on: ubuntu-latest
# Only run for the nf-core repo, for releases and merged PRs
if: ${{ github.repository == 'nf-core/tools' }}
steps:
- name: Check out the repo
uses: actions/checkout@v2

- name: Log in to Docker Hub
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_PASS }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v3
with:
images: nfcore/gitpod
tags: |
type=semver,pattern={{version}}
type=raw,value=latest
- name: Build and push Docker image
uses: docker/build-push-action@v2
with:
context: .
file: nf_core/gitpod/gitpod.Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
2 changes: 1 addition & 1 deletion .github/workflows/create-test-wf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ jobs:
- name: Run nf-core/tools
run: |
nf-core --log-file log.txt create -n testpipeline -d "This pipeline is for testing" -a "Testing McTestface"
nextflow run nf-core-testpipeline -profile test,docker
nextflow run nf-core-testpipeline -profile test,docker --outdir ./results
- name: Upload log file artifact
if: ${{ always() }}
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/markdown-lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,7 @@ jobs:
steps:
- uses: actions/checkout@v2

- uses: actions/setup-node@v1
with:
node-version: "10"
- uses: actions/setup-node@v2

- name: Install markdownlint
run: npm install -g markdownlint-cli
Expand Down
1 change: 0 additions & 1 deletion .github/workflows/push_dockerhub_dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,4 +27,3 @@ jobs:
run: |
echo "$DOCKERHUB_PASS" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
docker push nfcore/tools:dev
6 changes: 1 addition & 5 deletions .github/workflows/pytest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,7 @@ name: Python tests
# Only run if we changed a Python file
on:
push:
paths:
- "**.py"
pull_request:
paths:
- "**.py"

# Uncomment if we need an edge release of Nextflow again
# env: NXF_EDGE: 1
Expand Down Expand Up @@ -41,7 +37,7 @@ jobs:
sudo mv nextflow /usr/local/bin/
- name: Test with pytest
run: python3 -m pytest --color=yes --cov-report=xml --cov-config=.github/.coveragerc --cov=nf_core
run: python3 -m pytest tests/ --color=yes --cov-report=xml --cov-config=.github/.coveragerc --cov=nf_core

- uses: codecov/codecov-action@v1
name: Upload code coverage report
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/tools-api-docs-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ jobs:
runs-on: ubuntu-18.04
strategy:
matrix:
dir:
- latest
- ${{ github.event.release.tag_name }}
dir:
- latest
- ${{ github.event.release.tag_name }}
steps:
- name: Check out source-code repository
uses: actions/checkout@v2
Expand Down
39 changes: 39 additions & 0 deletions .github/workflows/yamllint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Lint YAML
on: [push, pull_request]
jobs:
YAML:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@master
- name: 'Yamllint'
uses: karancode/yamllint-github-action@master
with:
yamllint_file_or_dir: '.'
yamllint_config_filepath: '.yamllint.yml'

# If the above check failed, post a comment on the PR explaining the failure
- name: Post PR comment
if: failure()
uses: mshick/add-pr-comment@v1
with:
message: |
## YAML linting is failing
To keep the code consistent with lots of contributors, we run automated code consistency checks.
To fix this CI test, please run:
* Install `yamllint`
* Install `yamllint` following [this](https://yamllint.readthedocs.io/en/stable/quickstart.html#installing-yamllint)
instructions or alternative install it in your [conda environment](https://anaconda.org/conda-forge/yamllint)
* Fix the markdown errors
* Run the test locally: `yamllint $(find . -type f -name "*.yml" -o -name "*.yaml") -c ./.yamllint.yml`
* Fix any reported errors in your YAML files
Once you push these changes the test should pass, and you can hide this comment :+1:
We highly recommend setting up yaml-lint in your code editor so that this formatting is done automatically on save. Ask about it on Slack for help!
Thanks again for your contribution!
repo-token: ${{ secrets.GITHUB_TOKEN }}
allow-repeats: false
17 changes: 17 additions & 0 deletions .gitpod.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
image: nfcore/gitpod:latest
tasks:
- name: install current state of nf-core/tools
command: python -m pip install --upgrade -r requirements-dev.txt -e .
vscode:
extensions: # based on nf-core.nf-core-extensionpack
- codezombiech.gitignore # Language support for .gitignore files
# - cssho.vscode-svgviewer # SVG viewer
- davidanson.vscode-markdownlint # Markdown/CommonMark linting and style checking for Visual Studio Code
- eamodio.gitlens # Quickly glimpse into whom, why, and when a line or code block was changed
- EditorConfig.EditorConfig # override user/workspace settings with settings found in .editorconfig files
- Gruntfuggly.todo-tree # Display TODO and FIXME in a tree view in the activity bar
- mechatroner.rainbow-csv # Highlight columns in csv files in different colors
# - nextflow.nextflow # Nextflow syntax highlighting
- oderwat.indent-rainbow # Highlight indentation level
- streetsidesoftware.code-spell-checker # Spelling checker for source code
- DavidAnson.vscode-markdownlint # Linter for markdown files
7 changes: 7 additions & 0 deletions .yamllint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
extends: default
ignore: |
nf_core/*-template/
rules:
document-start: disable
line-length: disable
truthy: disable
60 changes: 59 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,64 @@
# nf-core/tools: Changelog

## [v2.3 - Mercury Vulture](https://github.com/nf-core/tools/releases/tag/2.3) - [2022-03-15]

### Template

* Removed mention of `--singularity_pull_docker_container` in pipeline `README.md`
* Replaced equals with ~ in nf-core headers, to stop false positive unresolved conflict errors when committing with VSCode.
* Add retry strategy for AWS megatests after releasing [nf-core/tower-action v2.2](https://github.com/nf-core/tower-action/releases/tag/v2.2)
* Added `.nf-core.yml` file with `repository_type: pipeline` for modules commands
* Update igenomes path to the `BWAIndex` to fetch the whole `version0.6.0` folder instead of only the `genome.fa` file
* Remove pinned Node version in the GitHub Actions workflows, to fix errors with `markdownlint`
* Bumped `nf-core/tower-action` to `v3` and removed `pipeline` and `revision` from the AWS workflows, which were not needed
* Add yamllint GitHub Action.
* Add `.yamllint.yml` to avoid line length and document start errors ([#1407](https://github.com/nf-core/tools/issues/1407))
* Add `--publish_dir_mode` back into the pipeline template ([nf-core/rnaseq#752](https://github.com/nf-core/rnaseq/issues/752#issuecomment-1039451607))
* Add optional loading of of pipeline-specific institutional configs to `nextflow.config`
* Make `--outdir` a mandatory parameter ([nf-core/tools#1415](https://github.com/nf-core/tools/issues/1415))

## General

* Updated `nf-core download` to work with latest DSL2 syntax for containers ([#1379](https://github.com/nf-core/tools/issues/1379))
* Made `nf-core modules create` detect repository type with explicit `.nf-core.yml` instead of random readme stuff ([#1391](https://github.com/nf-core/tools/pull/1391))
* Added a Gitpod environment and Dockerfile ([#1384](https://github.com/nf-core/tools/pull/1384))
* Adds conda, Nextflow, nf-core, pytest-workflow, mamba, and pip to base Gitpod Docker image.
* Adds GH action to build and push Gitpod Docker image.
* Adds Gitpod environment to template.
* Adds Gitpod environment to tools with auto build of nf-core tool.
* Shiny new command-line help formatting ([#1403](https://github.com/nf-core/tools/pull/1403))
* Call the command line help with `-h` as well as `--help` (was formerly just the latter) ([#1404](https://github.com/nf-core/tools/pull/1404))
* Add `.yamllint.yml` config file to avoid line length and document start errors in the tools repo itself.
* Switch to `yamllint-github-action`to be able to configure yaml lint exceptions ([#1404](https://github.com/nf-core/tools/issues/1413))
* Prevent module linting KeyError edge case ([#1321](https://github.com/nf-core/tools/issues/1321))
* Bump-versions: Don't trim the trailing newline on files, causes editorconfig linting to fail ([#1265](https://github.com/nf-core/tools/issues/1265))
* Handle exception in `nf-core list` when a broken git repo is found ([#1273](https://github.com/nf-core/tools/issues/1273))
* Updated URL for pipeline lint test docs ([#1348](https://github.com/nf-core/tools/issues/1348))
* Updated `nf-core create` to tolerate failures and retry when fetching pipeline logos from the website ([#1369](https://github.com/nf-core/tools/issues/1369))

### Modules

* New command `nf-core modules info` that prints nice documentation about a module to the terminal :sparkles: ([#1427](https://github.com/nf-core/tools/issues/1427))
* Linting a pipeline now fails instead of warning if a local copy of a module does not match the remote ([#1313](https://github.com/nf-core/tools/issues/1313))
* Fixed linting bugs where warning was incorrectly generated for:
* `Module does not emit software version`
* `Container versions do not match`
* `input:` / `output:` not being specified in module
* Allow for containers from other biocontainers resource as defined [here](https://github.com/nf-core/modules/blob/cde237e7cec07798e5754b72aeca44efe89fc6db/modules/cat/fastq/main.nf#L7-L8)
* Fixed traceback when using `stageAs` syntax as defined [here](https://github.com/nf-core/modules/blob/cde237e7cec07798e5754b72aeca44efe89fc6db/modules/cat/fastq/main.nf#L11)
* Added `nf-core schema docs` command to output pipline parameter documentation in Markdown format for inclusion in GitHub and other documentation systems ([#741](https://github.com/nf-core/tools/issues/741))
* Allow conditional process execution from the configuration file ([#1393](https://github.com/nf-core/tools/pull/1393))
* Add linting for when condition([#1397](https://github.com/nf-core/tools/pull/1397))
* Added modules ignored table to `nf-core modules bump-versions`. ([#1234](https://github.com/nf-core/tools/issues/1234))
* Added `--conda-package-version` flag for specifying version of conda package in `nf-core modules create`. ([#1238](https://github.com/nf-core/tools/issues/1238))
* Add option of writing diffs to file in `nf-core modules update` using either interactive prompts or the new `--diff-file` flag.
* Fixed edge case where module names that were substrings of other modules caused both to be installed ([#1380](https://github.com/nf-core/tools/issues/1380))
* Tweak handling of empty files when generating the test YAML ([#1376](https://github.com/nf-core/tools/issues/1376))
* Fail linting if a md5sum for an empty file is found (instead of a warning)
* Don't skip the md5 when generating a test file if an empty file is found (so that linting fails and can be manually checked)
* Linting checks test files for `TODO` statements as well as the main module code ([#1271](https://github.com/nf-core/tools/issues/1271))
* Handle error if `manifest` isn't set in `nextflow.config` ([#1418](https://github.com/nf-core/tools/issues/1418))

## [v2.2 - Lead Liger](https://github.com/nf-core/tools/releases/tag/2.2) - [2021-12-14]

### Template
Expand Down Expand Up @@ -51,7 +110,6 @@
* Check if README is from modules repo
* Update module template to DSL2 v2.0 (remove `functions.nf` from modules template and updating `main.nf` ([#1289](https://github.com/nf-core/tools/pull/))
* Substitute get process/module name custom functions in module `main.nf` using template replacement ([#1284](https://github.com/nf-core/tools/issues/1284))
* Linting now fails instead of warning if a local copy of a module does not match the remote ([#1313](https://github.com/nf-core/tools/issues/1313))
* Check test YML file for md5sums corresponding to empty files ([#1302](https://github.com/nf-core/tools/issues/1302))
* Exit with an error if empty files are found when generating the test YAML file ([#1302](https://github.com/nf-core/tools/issues/1302))

Expand Down
Loading

0 comments on commit b1ce471

Please sign in to comment.