Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: DataFrameTreeReduction is no longer part of dask #566

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

lgray
Copy link
Collaborator

@lgray lgray commented Jan 18, 2025

This class was removed from dask itself, and dask-awkward is presently broken with latest dask because of that.

@lgray
Copy link
Collaborator Author

lgray commented Jan 18, 2025

@douglasdavis @martindurant now that the legacy dask dataframe is no longer there, this causes a bunch of tests to fail as well, and forces dependency on dask-expr.

@martindurant
Copy link
Collaborator

Is the following enough for those failures:

from dask.dataframe.core import DataFrame
-> 
from dask.dataframe import DataFrame

?

@ikrommyd
Copy link
Contributor

Is the following enough for those failures:

from dask.dataframe.core import DataFrame
-> 
from dask.dataframe import DataFrame

?

It should be. This import was removed recently and works fine for me locally

Python 3.12.7 | packaged by conda-forge | (main, Oct  4 2024, 15:57:01) [Clang 17.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from dask.dataframe.core import DataFrame
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'DataFrame' from 'dask.dataframe.core' (/Users/iason/miniforge3/envs/egamma-dev/lib/python3.12/site-packages/dask/dataframe/core.py)
>>> from dask.dataframe import DataFrame
>>> exit()

However, please make sure that from dask.dataframe import DataFrame is compatible with older versions of dask/distributed. We typically run on clusters that don't have the latest versions available.

@martindurant
Copy link
Collaborator

However, please make sure that from dask.dataframe import DataFrame is compatible with older versions of dask/distributed.

Yes, it has been this way a long time.

@martindurant
Copy link
Collaborator

from dask.dataframe import DataFrame

On further reading, this is no longer the same DataFrame as before at all ("DataFrame-like Expr Collection."). There is still a function, dask.dataframe.from_graph that supports making a thing, that might be what we need. What a mess!

(I guess akimbo.dask is broken too)

@martindurant
Copy link
Collaborator

@lgray , let me know when you are working on this. I suppose dask-histogram is broken too; but the copy of tree-reduction should live here, so I'll wait for this PR and then fix the other.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants