chore(deps): bump datasets from 2.20.0 to 3.0.1

[dependabot]

Bumps datasets from 2.20.0 to 3.0.1.

Release notes

Sourced from datasets's releases.

3.0.1

What's Changed

• Modify add_column() to optionally accept a FeatureType as param by @​varadhbhatnagar in huggingface/datasets#7143
• Align filename prefix splitting with WebDataset library by @​albertvillanova in huggingface/datasets#7151
• Support ndjson data files by @​albertvillanova in huggingface/datasets#7154
• Support JSON lines with missing struct fields by @​albertvillanova in huggingface/datasets#7160
• Support JSON lines with empty struct by @​albertvillanova in huggingface/datasets#7162
• fix increase_load_count by @​lhoestq in huggingface/datasets#7165
• fix docstring code example for distributed shuffle by @​lhoestq in huggingface/datasets#7166
• Support JSON lines with missing columns by @​albertvillanova in huggingface/datasets#7170
• Add torchdata as a regular test dependency by @​albertvillanova in huggingface/datasets#7172

New Contributors

• @​varadhbhatnagar made their first contribution in huggingface/datasets#7143

Full Changelog: huggingface/datasets@3.0.0...3.0.1

3.0.0

Dataset Features

• Use Polars functions in .map()

  • Allow Polars as valid output type by @​psmyth94 in huggingface/datasets#6762

  • Example:

    >>> from datasets import load_dataset
    >>> ds = load_dataset("lhoestq/CudyPokemonAdventures", split="train").with_format("polars")
    >>> cols = [pl.col("content").str.len_bytes().alias("length")]
    >>> ds_with_length = ds.map(lambda df: df.with_columns(cols), batched=True)
    >>> ds_with_length[:5]
    shape: (5, 5)
    ┌─────┬───────────────────────────────────┬───────────────────────────────────┬───────────────────────┬────────┐
    │ idx ┆ title                             ┆ content                           ┆ labels                ┆ length │
    │ --- ┆ ---                               ┆ ---                               ┆ ---                   ┆ ---    │
    │ i64 ┆ str                               ┆ str                               ┆ str                   ┆ u32    │
    ╞═════╪═══════════════════════════════════╪═══════════════════════════════════╪═══════════════════════╪════════╡
    │ 0   ┆ The Joyful Adventure of Bulbasau… ┆ Bulbasaur embarked on a sunny qu… ┆ joyful_adventure      ┆ 180    │
    │ 1   ┆ Pikachu's Quest for Peace         ┆ Pikachu, with his cheeky persona… ┆ peaceful_narrative    ┆ 138    │
    │ 2   ┆ The Tender Tale of Squirtle       ┆ Squirtle took everyone on a memo… ┆ gentle_adventure      ┆ 135    │
    │ 3   ┆ Charizard's Heartwarming Tale     ┆ Charizard found joy in helping o… ┆ heartwarming_story    ┆ 112    │
    │ 4   ┆ Jolteon's Sparkling Journey       ┆ Jolteon, with his zest for life,… ┆ celebratory_narrative ┆ 111    │
    └─────┴───────────────────────────────────┴───────────────────────────────────┴───────────────────────┴────────┘

• Support NumPy 2
  • Allow numpy-2.1 and test it without audio extra by @​albertvillanova in huggingface/datasets#7118

Cache Changes

• Use huggingface_hub cache by @​lhoestq in huggingface/datasets#7105
  • use the huggingface_hub cache for files downloaded from HF, by default at ~/.cache/huggingface/hub
  • cached datasets (Arrow files) will still be reloaded from the datasets cache, by default at ~/.cache/huggingface/datasets

... (truncated)

Commits

• 679562d Release: 3.0.1 (#7173)
• 46c39d8 Add torchdata as a regular test dependency (#7172)
• e450bd3 Support JSON lines with missing columns (#7170)
• e9ec56c fix docstring code example for distributed shuffle (#7166)
• 548d2d2 fix increase_load_count (#7165)
• b032702 Support JSON lines with empty struct (#7162)
• 2eb4edb Support JSON lines with missing struct fields (#7160)
• 13f18e3 Support ndjson data files (#7154)
• d70c902 Align filename prefix splitting with WebDataset library (#7151)
• 43b1fe1 Modify add_column() to optionally accept a FeatureType as param (#7143)
• Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

---

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

• @dependabot rebase will rebase this PR
• @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
• @dependabot merge will merge this PR after your CI passes on it
• @dependabot squash and merge will squash and merge this PR after your CI passes on it
• @dependabot cancel merge will cancel a previously requested merge and block automerging
• @dependabot reopen will reopen this PR if it is closed
• @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
• @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
• @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
• @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

[dependabot]

Superseded by #91.