Task config

[hynky1999]

What does this implement/fix? Explain your changes.

1. This PR adds new adds following new args to task config

• hf_revision (allows locking to certain commit)
• hf_filter (allows filtering the datasets)

2. Set defaults as early as possible for lighteval task config
3. Removing of unused args from task class:

      self.save_queries: bool = False
      self.logfile_name: Optional[Path] = None
      self.is_main_process: bool = False

4. Improved typings

5. I noticed that following args are unused, maybe we should create issue to implement that functionality or remove it ?

    frozen: bool = False
    output_regex: Optional[str] = None

[NathanHB]

why do we ignore the type here ?

[hynky1999]

Because it doesn't have correct type
load_dataset returns Dataset | DatasetDict | IterableDataset | IterableDatasetDict, based on the input args and afaik there is unspecified contract that if the we don't provide streaming and split arg we get DatasetDict. However there is no way to achieve this on typings level, so I just ignore this error.

If the question was why I put there the type: ignore it's because even tho we don't have a typechecker in the quality checks, I do have it on in my vscode (pyright) and it shows red when there is a typing problem.

[NathanHB]

Looks great !
the output_regex is legacy from the harness, we needed it for bigbench tasks but this is now handled by the metrics direclty. Frozen was to make sure the task did not change over time but we are now using task versioning for this.