add schema to validate meta.ymls

[mashehu]

In order to avoid this I am proposing the following JSON-schema. We will use this then during nf-core modules lint to check the meta.ymls (and in the future also for subworkflows). It also is integrated in the pre-commit check. If you are using the yaml extension for VS code you get also hints about expected entries when working on this file (by adding the following to your settings.json

"yaml.schemas": {
"yaml-schema.json": "modules/**/meta.yml"
},

[mirpedrol]

Looks good :)

We will still have the problem of "wrong" indentation. When the indentation is not wrong but not in the way we like, this makes the command nf-core modules info fail. For example:

  - html:
    type: file
    description: FastQC report
    pattern: "*_{fastqc.html}"

instead of

  - html:
      type: file
      description: FastQC report
      pattern: "*_{fastqc.html}"

[mashehu]

Looks good :)

We will still have the problem of "wrong" indentation. When the indentation is not wrong but not in the way we like, this makes the command nf-core modules info fail. For example:

I tried it out and my schema checker complains correctly about it (but doesn't fix it automatically, we still need to do that by ourselves).

[mirpedrol]

I tested this pre-commit hook with Gitpod and did not fail, but now having a look at the follow up PR nf-core/tools#2190 I see that the linting complains :)
I get this error message "The meta.yml of the module fastqc is not valid: None is not of type 'object'. Check that the child entries of output.html are indented correctly."

[mashehu]

I tested this pre-commit hook with Gitpod and did not fail, but now having a look at the follow up PR nf-core/tools#2190 I see that the linting complains :) I get this error message "The meta.yml of the module fastqc is not valid: None is not of type 'object'. Check that the child entries of output.html are indented correctly."

Yep, that is working as expected. Or were you thinking of something else?

[mirpedrol]

Yep, that is working as expected. Or were you thinking of something else?

No, works very nice 😄
I think it's fine then, even if the pre-commit hook doesn't mention the indentation because linting will.

[mashehu]

ah, you meant the git-hook doesn't do anything? let me check.

[edmundmiller]

Could we also add a line to all the meta yamls with something like this

---
# yaml-language-server: $schema=https://path-to-schema.json

[jfy133]

Overall looks good but I would rather make it more strict 😬

[jfy133]

Should we set an enum based on: https://nf-co.re/docs/contributing/modules#documentation point 6?

[jfy133]

(applies to output: too)

[mashehu]

Added it now locally and I get quite a few deviations. Because I see quite a lot of "list" entries coming up, should we add it to the options? (if that is the correct groovy data object)

[mirpedrol]

Looking for an example, I found justrma and the yml. Looks like celfiles_dir should be directory instead of list, no?
I don't know if there are other modules with different situations.

[jfy133]

I would rather say list isn't so informative for the type, it's the contents of the list is more important (is it a list of files?)

[mirpedrol]

I would opt for getting rid of list and just specifying the type of the list elements

[mashehu]

I think it is important to indicate if it is more then one item, so I would go for nested here, so we can actually check the items themselves as well.

[jfy133]

But shouldn't that be inthe description (that's normally how I do it)

type: file
description: A file or list of files that XYZ

[mashehu]

if we make it nested, we could then for example check the file extension of each individual file. More validation power 🦾

[jfy133]

🍝

[jfy133]

Maybe an enum using: http://bio-tools.github.io/biotoolsSchema/#Link21 ?

[mashehu]

Hmm, that list is a bit too long and would make the schema quite unwieldy I think.

[jfy133]

I thought with JSON schema you can refer to 'remote' enums? But agree wouldn't want to embed the whole list.

[jfy133]

I wonder how bioconda/conda does it 🤔 as they have a lint for that

[mashehu]

ah, true! will try that