Merge the workers that rely on the datasets library

[severo]

The docker image size for a worker based on the datasets library is large, mainly due to two dependencies: PyTorch and TensorFlow (about 4GB).
We create /workers/datasets_based for all the workers that depend on datasets and chose the processing step with the DATASETS_BASED_ENDPOINT env var.

[codecov-commenter]

Codecov Report

Base: 85.56% // Head: 89.04% // Increases project coverage by +3.48% 🎉

Coverage data is based on head (239ffff) compared to base (4ab24c5).
Patch coverage: 100.00% of modified lines in pull request are covered.

❗ Current head 239ffff differs from pull request most recent head 931d853. Consider uploading reports for the commit 931d853 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #656      +/-   ##
==========================================
+ Coverage   85.56%   89.04%   +3.48%     
==========================================
  Files          54       10      -44     
  Lines        2597      639    -1958     
==========================================
- Hits         2222      569    -1653     
+ Misses        375       70     -305     

Flag	Coverage Δ
jobs_mongodb_migration	?
libs_libcommon	?
services_admin	?
services_api	?
workers_datasets_based	89.04% <100.00%> (?)
workers_first_rows	?
workers_splits	?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
workers/datasets_based/src/datasets_based/asset.py	100.00% <ø> (ø)
workers/datasets_based/tests/fixtures/datasets.py	100.00% <ø> (ø)
...orkers/datasets_based/src/datasets_based/config.py	100.00% <100.00%> (ø)
...kers/datasets_based/src/datasets_based/features.py	73.43% <100.00%> (ø)
...orkers/datasets_based/src/datasets_based/worker.py	100.00% <100.00%> (ø)
...ets_based/src/datasets_based/workers/first_rows.py	82.87% <100.00%> (ø)
...atasets_based/src/datasets_based/workers/splits.py	97.36% <100.00%> (ø)
workers/datasets_based/tests/test_features.py	100.00% <100.00%> (ø)
...rs/datasets_based/tests/workers/test_first_rows.py	91.13% <100.00%> (ø)
...orkers/datasets_based/tests/workers/test_splits.py	89.85% <100.00%> (ø)
... and 50 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.