Merge branch 'dev' into modules-remove-patch

.github/workflows/clean-up.yml

@@ -0,0 +1,24 @@
+name: "Close user-tagged issues and PRs"
+on:
+  schedule:
+    - cron: "0 0 * * 0" # Once a week
+
+jobs:
+  clean-up:
+    runs-on: ubuntu-latest
+    permissions:
+      issues: write
+      pull-requests: write
+    steps:
+      - uses: actions/stale@v7
+        with:
+          stale-issue-message: "This issue has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment otherwise this issue will be closed in 20 days."
+          stale-pr-message: "This PR has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment if it is still useful."
+          close-issue-message: "This issue was closed because it has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor and then staled for 20 days with no activity."
+          days-before-stale: 30
+          days-before-close: 20
+          days-before-pr-close: -1
+          any-of-labels: "awaiting-changes,awaiting-feedback"
+          exempt-issue-labels: "WIP"
+          exempt-pr-labels: "WIP"
+          repo-token: ${{ secrets.GITHUB_TOKEN }}

.github/workflows/create-lint-wf.yml

@@ -108,7 +108,7 @@ jobs:
         run: nf-core --log-file log.txt modules install fastqc --dir nf-core-testpipeline/ --force
 
       - name: nf-core modules install gitlab
-        run: nf-core --log-file log.txt modules --git-remote https://gitlab.com/nf-core/modules-test.git --branch main install fastqc --force --dir nf-core-testpipeline/
+        run: nf-core --log-file log.txt modules --git-remote https://gitlab.com/nf-core/modules-test.git --branch branch-tester install fastp --dir nf-core-testpipeline/
 
       - name: nf-core modules list local
         run: nf-core --log-file log.txt modules list local --dir nf-core-testpipeline/

CHANGELOG.md

@@ -12,6 +12,9 @@
 - Consistent syntax for branch checks in PRs ([#2202](https://github.com/nf-core/tools/issues/2202))
 - Fixed minor Jinja2 templating bug that caused the PR template to miss a newline
 - Updated AWS tests to use newly moved `seqeralabs/action-tower-launch` instead of `nf-core/tower-action`
+- Remove `.cff` files from `.editorconfig` [(#2145)[https://github.com/nf-core/tools/pull/2145]]
+- Simplify pipeline README ([#2186](https://github.com/nf-core/tools/issues/2186))
+- Added support for the apptainer container engine via `-profile apptainer`. ([#2244](https://github.com/nf-core/tools/issues/2244)) [Contributed by @jfy133]
 
 ### Linting
 
@@ -22,6 +25,8 @@
 ### Modules
 
 - Add an `--empty-template` option to create a module without TODO statements or examples ([#2175](https://github.com/nf-core/tools/pull/2175) & [#2177](https://github.com/nf-core/tools/pull/2177))
+- Removed the `nf-core modules mulled` command and all its code dependencies ([2199](https://github.com/nf-core/tools/pull/2199)).
+- Take into accout the provided `--git_remote` URL when linting all modules ([2243](https://github.com/nf-core/tools/pull/2243)).
 
 ### Subworkflows
 
@@ -37,6 +42,8 @@
 - Add the Nextflow version to Gitpod container matching the minimal Nextflow version for nf-core (according to `nextflow.config`) ([#2196](https://github.com/nf-core/tools/pull/2196))
 - Use `nfcore/gitpod:dev` container in the dev branch ([#2196](https://github.com/nf-core/tools/pull/2196))
 - Replace requests_mock with responses in test mocks ([#2165](https://github.com/nf-core/tools/pull/2165)).
+- Add warning when installing a module from an `org_path` that exists in multiple remotes in `modules.json` ([#2228](https://github.com/nf-core/tools/pull/2228) [#2239](https://github.com/nf-core/tools/pull/2239)).
+- Add the possibility to translate refgenie asset aliases to the ones used in a pipeline with an alias_translations.yaml file ([#2242](https://github.com/nf-core/tools/pull/2242)).
 
 ## [v2.7.2 - Mercury Eagle Patch](https://github.com/nf-core/tools/releases/tag/2.7.2) - [2022-12-19]
 

README.md

@@ -42,7 +42,6 @@ A python package with helper tools for the nf-core community.
   - [`modules lint` - Check a module against nf-core guidelines](#check-a-module-against-nf-core-guidelines)
   - [`modules test` - Run the tests for a module](#run-the-tests-for-a-module-using-pytest)
   - [`modules bump-versions` - Bump software versions of modules](#bump-bioconda-and-container-versions-of-modules-in)
-  - [`modules mulled` - Generate the name for a multi-tool container image](#generate-the-name-for-a-multi-tool-container-image)
 
 - [`nf-core subworkflows` - commands for dealing with subworkflows](#subworkflows)
   - [`subworkflows list` - List available subworkflows](#list-subworkflows)
@@ -972,17 +971,6 @@ bump-versions:
   star/align: "2.6.1d"
 ```
 
-### Generate the name for a multi-tool container image
-
-When you want to use an image of a multi-tool container and you know the specific dependencies and their versions of that container, for example, by looking them up in the [BioContainers hash.tsv](https://github.com/BioContainers/multi-package-containers/blob/master/combinations/hash.tsv), you can use the `nf-core modules mulled` helper tool. This tool generates the name of a BioContainers mulled image.
-
-<!-- RICH-CODEX
-working_dir: tmp/modules
-after_command: cd ../../ && rm -rf tmp
--->
-
-![`nf-core modules mulled pysam==0.16.0.1 biopython==1.78`](docs/images/nf-core-modules-mulled.svg)
-
 ## Subworkflows
 
 After the launch of nf-core modules, we can provide now also nf-core subworkflows to fully utilize the power of DSL2 modularization.

nf_core/__main__.py

@@ -52,7 +52,7 @@
         },
         {
             "name": "Developing new modules",
-            "commands": ["create", "create-test-yml", "lint", "bump-versions", "mulled", "test"],
+            "commands": ["create", "create-test-yml", "lint", "bump-versions", "test"],
         },
     ],
     "nf-core subworkflows": [
@@ -879,49 +879,6 @@ def bump_versions(ctx, tool, dir, all, show_all):
         sys.exit(1)
 
 
-# nf-core modules mulled
-@modules.command()
-@click.argument("specifications", required=True, nargs=-1, metavar="<tool==version> <...>")
-@click.option(
-    "--build-number",
-    type=int,
-    default=0,
-    show_default=True,
-    metavar="<number>",
-    help="The build number for this image. This is an incremental value that starts at zero.",
-)
-def mulled(specifications, build_number):
-    """
-    Generate the name of a BioContainers mulled image version 2.
-
-    When you know the specific dependencies and their versions of a multi-tool container image and you need the name of
-    that image, this command can generate it for you.
-
-    """
-    from nf_core.modules.mulled import MulledImageNameGenerator
-
-    try:
-        image_name = MulledImageNameGenerator.generate_image_name(
-            MulledImageNameGenerator.parse_targets(specifications), build_number=build_number
-        )
-    except ValueError as e:
-        log.error(e)
-        sys.exit(1)
-    if not MulledImageNameGenerator.image_exists(image_name):
-        log.error("The generated multi-tool container image name does not seem to exist yet.")
-        log.info(
-            "Please double check that your provided combination of tools and versions exists in the file: "
-            "[link=https://github.com/BioContainers/multi-package-containers/blob/master/combinations/hash.tsv]BioContainers/multi-package-containers 'combinations/hash.tsv'[/link]"
-        )
-        log.info(
-            "If it does not, please add your desired combination as detailed at: "
-            "https://github.com/BioContainers/multi-package-containers"
-        )
-        sys.exit(1)
-    log.info("Mulled container hash:")
-    stdout.print(image_name)
-
-
 # nf-core modules test
 @modules.command("test")
 @click.pass_context

nf_core/components/install.py

@@ -58,6 +58,12 @@ def install(self, component, silent=False):
         if not silent:
             modules_json.check_up_to_date()
 
+        # Verify that the remote repo's org_path does not match the org_path of any alternate repo among the installed modules
+        if self.check_alternate_remotes(modules_json):
+            err_msg = f"You are trying to install {self.component_type} from different repositories with the same organization name '{self.modules_repo.repo_path}' (set in the `.nf-core.yml` file in the `org_path` field).\nThis is not supported, and will likely cause problems. org_path should be set to the github account/organization name."
+            log.error(err_msg)
+            return False
+
         # Verify SHA
         if not self.modules_repo.verify_sha(self.prompt, self.sha):
             return False
@@ -264,3 +270,20 @@ def clean_modules_json(self, component, modules_repo, modules_json):
                             self.component_type, component, repo_to_remove, modules_repo.repo_path
                         )
                         return component_values["installed_by"]
+
+    def check_alternate_remotes(self, modules_json):
+        """
+        Check whether there are previously installed components with the same org_path but different remote urls
+        Log error if multiple remotes exist.
+
+        Return:
+            True: if problematic components are found
+            False: if problematic components are not found
+        """
+        modules_json.load()
+        for repo_url, repo_content in modules_json.modules_json.get("repos", dict()).items():
+            for component_type in repo_content:
+                for dir in repo_content.get(component_type, dict()).keys():
+                    if dir == self.modules_repo.repo_path and repo_url != self.modules_repo.remote_url:
+                        return True
+        return False

nf_core/lint/readme.py

@@ -62,24 +62,4 @@ def readme(self):
         else:
             warned.append("README did not have a Nextflow minimum version badge.")
 
-    # Check that the minimum version mentioned in the quick start section is consistent
-    # Looking for: "1. Install [`Nextflow`](https://www.nextflow.io/docs/latest/getstarted.html#installation) (`>=22.10.1`)"
-    nf_version_re = r"1\.\s*Install\s*\[`Nextflow`\]\(https://www.nextflow.io/docs/latest/getstarted.html#installation\)\s*\(`>=(\d*\.\d*\.\d*)`\)"
-    match = re.search(nf_version_re, content)
-    if match:
-        nf_quickstart_version = match.group(1)
-        try:
-            if nf_quickstart_version != self.minNextflowVersion:
-                raise AssertionError()
-        except (AssertionError, KeyError):
-            failed.append(
-                f"README Nextflow minimium version in Quick Start section does not match config. README: `{nf_quickstart_version}`, Config `{self.minNextflowVersion}`"
-            )
-        else:
-            passed.append(
-                f"README Nextflow minimum version in Quick Start section matched config. README: `{nf_quickstart_version}`, Config: `{self.minNextflowVersion}`"
-            )
-    else:
-        warned.append("README did not have a Nextflow minimum version mentioned in Quick Start section.")
-
     return {"passed": passed, "warned": warned, "failed": failed}

nf_core/modules/__init__.py

@@ -8,7 +8,6 @@
 from .modules_repo import ModulesRepo
 from .modules_test import ModulesTest
 from .modules_utils import ModuleException
-from .mulled import MulledImageNameGenerator
 from .patch import ModulePatch
 from .remove import ModuleRemove
 from .test_yml_builder import ModulesTestYmlBuilder

nf_core/modules/lint/__init__.py

@@ -91,6 +91,8 @@ def __init__(
             modules_json.check_up_to_date()
             self.all_remote_modules = []
             for repo_url, components in modules_json.get_all_components(self.component_type).items():
+                if remote_url is not None and remote_url != repo_url:
+                    continue
                 for org, comp in components:
                     self.all_remote_modules.append(
                         NFCoreModule(

nf_core/modules/modules_json.py

@@ -427,8 +427,8 @@ def parse_dirs(self, dirs, missing_installation, component_type):
             git_url = None
             for repo in missing_installation:
                 if component_type in missing_installation[repo]:
-                    for dir_name in missing_installation[repo][component_type]:
-                        if component in missing_installation[repo][component_type][dir_name]:
+                    if install_dir in missing_installation[repo][component_type]:
+                        if component in missing_installation[repo][component_type][install_dir]:
                             component_in_file = True
                             git_url = repo
                             break

nf_core/pipeline-template/.editorconfig

@@ -8,7 +8,7 @@ trim_trailing_whitespace = true
 indent_size = 4
 indent_style = space
 
-[*.{md,yml,yaml,html,css,scss,js,cff}]
+[*.{md,yml,yaml,html,css,scss,js}]
 indent_size = 2
 
 # These files are edited and tested upstream in nf-core/modules

nf_core/pipeline-template/.github/ISSUE_TEMPLATE/bug_report.yml

@@ -45,6 +45,6 @@ body:
         * Nextflow version _(eg. 22.10.1)_
         * Hardware _(eg. HPC, Desktop, Cloud)_
         * Executor _(eg. slurm, local, awsbatch)_
-        * Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter or Charliecloud)_
+        * Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud, or Apptainer)_
         * OS _(eg. CentOS Linux, macOS, Linux Mint)_
         * Version of {{ name }} _(eg. 1.1, 1.5, 1.8.2)_

nf_core/pipeline-template/.github/workflows/clean-up.yml

@@ -0,0 +1,24 @@
+name: "Close user-tagged issues and PRs"
+on:
+  schedule:
+    - cron: "0 0 * * 0" # Once a week
+
+jobs:
+  clean-up:
+    runs-on: ubuntu-latest
+    permissions:
+      issues: write
+      pull-requests: write
+    steps:
+      - uses: actions/stale@v7
+        with:
+          stale-issue-message: "This issue has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment otherwise this issue will be closed in 20 days."
+          stale-pr-message: "This PR has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor. Remove stale label or add a comment if it is still useful."
+          close-issue-message: "This issue was closed because it has been tagged as awaiting-changes or awaiting-feedback by an nf-core contributor and then staled for 20 days with no activity."
+          days-before-stale: 30
+          days-before-close: 20
+          days-before-pr-close: -1
+          any-of-labels: "awaiting-changes,awaiting-feedback"
+          exempt-issue-labels: "WIP"
+          exempt-pr-labels: "WIP"
+          repo-token: "{% raw %}${{ secrets.GITHUB_TOKEN }}{% endraw %}"