Update datasets to 2.11.0

[severo]

See https://github.com/huggingface/datasets/releases/tag/2.11.0

TODO: See discussions below

• Update datasets dependency to 2.11.0 version #1009
• Use soundfile for mp3 decoding instead of torchaudio #1280
• Remove dependency on torch and torchaudio #1281
• Use writer_batch_size for ArrowBasedBuilder
• Use direct cast from binary to Audio/Image
• Refresh datasets that use numpy.load

Useful changes for the datasets server (please complete if there are more, @huggingface/datasets)

Use soundfile for mp3 decoding instead of torchaudio by @polinaeterna in huggingface/datasets#5573

• this allows to not have dependencies on pytorch to decode audio files
• this was possible with soundfile 0.12 which bundles libsndfile binaries at a recent version with MP3 support

should we remove the dependency to torch and torchaudio? cc @polinaeterna

Add writer_batch_size for ArrowBasedBuilder by @lhoestq in huggingface/datasets#5565

• allow to specofy the row group / record batch size when you download_and_prepare() a dataset

Needed for #833 I think; cc @lhoestq

Allow direct cast from binary to Audio/Image by @mariosasko in huggingface/datasets#5644

Should we adapt the code in https://github.com/huggingface/datasets-server/blob/main/services/worker/src/worker/features.py due to that?

Support streaming datasets with numpy.load by @albertvillanova in huggingface/datasets#5626

should we refresh some datasets after that?

[albertvillanova]

I would suggest splitting this issue into several subtasks. I'm editing the description above...

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[severo]

Keep open. Note also #1099 to upgrade to 2.12.0

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[albertvillanova]

Keep open

[albertvillanova]

I think "Use writer_batch_size for ArrowBasedBuilder" is already done by:

• Lower parquet row group size for image datasets #833

CC: @lhoestq @severo

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.