Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix linting when creating a pipeline skipping some parts of the template #2330

Merged
merged 5 commits into from
Jun 21, 2023
+66 −1
Merged

Fix linting when creating a pipeline skipping some parts of the template #2330

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
18aa621
add files to ignore if some parts of the tamplet are skipped
mirpedrol Jun 20, 2023
8a86bbc
add pytest create a pipeline and skip parts
mirpedrol Jun 20, 2023
a32fd94
update CHANGELOG
mirpedrol Jun 20, 2023
56ca280
run prettier on pipeline_template.yml
mirpedrol Jun 20, 2023
f5c4f4e
ignore bug_report.yml linting if pipeline is unbranded
mirpedrol Jun 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
- Error if module container specification has quay.io as prefix when it shouldn't have ([#2278])(https://github.com/nf-core/tools/pull/2278/files)
- Detect if container is 'simple name' and try to contact quay.io server by default ([#2281](https://github.com/nf-core/tools/pull/2281))
- Warn about null/None/empty default values in `nextflow_schema.json` ([#3328](https://github.com/nf-core/tools/pull/2328))
- Fix linting when creating a pipeline skipping some parts of the template and add CI test ([#2330](https://github.com/nf-core/tools/pull/2330))

### Modules

Expand Down
30 changes: 29 additions & 1 deletion nf_core/create.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,7 @@ def render_template(self):
if self.template_yaml:
with open(self.outdir / "pipeline_template.yml", "w") as fh:
yaml.safe_dump(self.template_yaml, fh)
run_prettier_on_file(self.outdir / "pipeline_template.yml")

def update_nextflow_schema(self):
"""
Expand Down Expand Up @@ -403,6 +404,12 @@ def fix_linting(self):
".github/workflows/awstest.yml",
".github/workflows/awsfulltest.yml",
],
"files_unchanged": [
"CODE_OF_CONDUCT.md",
f"assets/nf-core-{short_name}_logo_light.png",
f"docs/images/nf-core-{short_name}_logo_light.png",
f"docs/images/nf-core-{short_name}_logo_dark.png",
],
"nextflow_config": [
"manifest.name",
"manifest.homePage",
Expand All @@ -415,9 +422,26 @@ def fix_linting(self):
lint_config["files_exist"].extend(
[
".github/ISSUE_TEMPLATE/bug_report.yml",
".github/ISSUE_TEMPLATE/feature_request.yml",
".github/PULL_REQUEST_TEMPLATE.md",
".github/CONTRIBUTING.md",
".github/.dockstore.yml",
".gitignore",
]
)
lint_config["files_unchanged"].extend(
[
".github/ISSUE_TEMPLATE/bug_report.yml",
".github/ISSUE_TEMPLATE/config.yml",
".github/ISSUE_TEMPLATE/feature_request.yml",
".github/PULL_REQUEST_TEMPLATE.md",
".github/workflows/branch.yml",
".github/workflows/linting_comment.yml",
".github/workflows/linting.yml",
".github/CONTRIBUTING.md",
".github/.dockstore.yml",
]
)
lint_config["files_unchanged"] = [".github/ISSUE_TEMPLATE/bug_report.yml"]

# Add CI specific configurations
if not self.template_params["ci"]:
Expand Down Expand Up @@ -446,6 +470,10 @@ def fix_linting(self):
if not self.template_params["github_badges"] or not self.template_params["github"]:
lint_config["readme"] = ["nextflow_badge"]

# If the pipeline is unbranded
if not self.template_params["branded"]:
lint_config["files_unchanged"].extend([".github/ISSUE_TEMPLATE/bug_report.yml"])
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we can avoid just flat out ignoring this file, that would be nice. There is already a function to customise it to remove some nf-core specific stuf. I think the failure on this file is something to do with the custom prefix / name match at the end of the output.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the pipeline is unbranded we remove one chunk of this file containing links to nf-core documentation:

  - type: markdown
    attributes:
      value: |
        Before you post this issue, please check the documentation:

        - [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
        - [{{ name }} pipeline documentation](https://nf-co.re/{{ short_name }}/usage)

so it won't match the template, and can't be considered a partial match either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so it won't match the template, and can't be considered a partial match either.

Seems like this isn't possible so will merge this in and we can follow up in a separate PR / issue if required.


# Add the lint content to the preexisting nf-core config
config_fn, nf_core_yml = nf_core.utils.load_tools_config(self.outdir)
nf_core_yml["lint"] = lint_config
Expand Down
7 changes: 7 additions & 0 deletions tests/data/pipeline_create_template_skip.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
prefix: testprefix
skip:
- github
- ci
- github_badges
- igenomes
- nf_core_configs
29 changes: 29 additions & 0 deletions tests/test_create.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@

TEST_DATA_DIR = Path(__file__).parent / "data"
PIPELINE_TEMPLATE_YML = TEST_DATA_DIR / "pipeline_create_template.yml"
PIPELINE_TEMPLATE_YML_SKIP = TEST_DATA_DIR / "pipeline_create_template_skip.yml"


class NfcoreCreateTest(unittest.TestCase):
Expand Down Expand Up @@ -107,3 +108,31 @@ def test_pipeline_creation_initiation_customize_template(self, mock_questionary,
assert os.path.exists(pipeline_template)
with open(pipeline_template) as fh:
assert fh.read() == PIPELINE_TEMPLATE_YML.read_text()

@with_temporary_folder
def test_pipeline_creation_with_yml_skip(self, tmp_path):
pipeline = nf_core.create.PipelineCreate(
name=self.pipeline_name,
description=self.pipeline_description,
author=self.pipeline_author,
version=self.pipeline_version,
no_git=False,
force=True,
outdir=tmp_path,
template_yaml_path=PIPELINE_TEMPLATE_YML_SKIP,
plain=True,
default_branch=self.default_branch,
)
pipeline.init_pipeline()
assert not os.path.isdir(os.path.join(pipeline.outdir, ".git"))

# Check pipeline yml has been dumped and matches input
pipeline_template = os.path.join(pipeline.outdir, "pipeline_template.yml")
assert os.path.exists(pipeline_template)
with open(pipeline_template) as fh:
assert fh.read() == PIPELINE_TEMPLATE_YML_SKIP.read_text()

# Check that some of the skipped files are not present
assert not os.path.exists(os.path.join(pipeline.outdir, "CODE_OF_CONDUCT.md"))
assert not os.path.exists(os.path.join(pipeline.outdir, ".github"))
assert not os.path.exists(os.path.join(pipeline.outdir, "conf", "igenomes.config"))