Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic handling for X,Y,band,time dimensions for transpose and rep… #244

Merged
merged 4 commits into from
May 6, 2024

Conversation

jzvolensky
Copy link
Contributor

@jzvolensky jzvolensky commented Apr 5, 2024

Hi all,

Corresponding issue which this will cover as well: #243.

I am currently working on adding the align parameter to the resample_spatial function as per the OpenEO processes documentation.

Part of this also involves handling cubes with different dimensions setups. For now, I edited the original fixed transpose to check the shape in the 4 basic dimensions.

Currently I added the align as a dummy parameter which does not do anything but allows us to use the function.

Couple of questions and ideas:

  1. Regarding the transpose/handling of the cube dimensions, how could we go about handling other dimensions since there are a lot of possibilities?
  2. How should the align parameter be implemented? what should the result be by using different align options?
  3. Since we already use ODC reprojection there is parameter anchor available which appears to do a similar operation to align link to the docs

Any feedback, ideas and suggestions are welcome!

Thanks!

@clausmichele
Copy link
Member

It would be good to have feedbacks from the VITO team (@jdries @soxofaan) about how they implemented resample_spatial on their side. Do you handle N-dimensional datacubes? Like x,y,bands,t + other dimensions that can be added using add_dimension?

@jdries
Copy link

jdries commented Apr 5, 2024

we don't do adding other dimensions, resample spatial just acts upon the x and y dimensions. The t and bands dimension are of course supported and basically left untouched.

@clausmichele
Copy link
Member

I didn't mean that you add dimensions within this process, but I was wondering if you support as input datacubes with arbitrary number of dimensions (additional to x and y), which seems so from your reply! Thanks

@jdries
Copy link

jdries commented Apr 5, 2024

to be clear: our backend does not at all support adding arbitrary number of dimensions.
We mainly support these combinations for raster cubes:
x,y
x,y,bands
x,y,bands,t
x,y,t

data_cp = data.transpose('y', 'x')
except Exception:
raise OpenEOException(
f"Data cube has unexpected dimensions: {dims}"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your inputs on the process and adding the align parameter!
I think, we can keep using the data.openeo here, as this checks the dimensions in the dataset - and for example uses t instead of time if the temporal dimension is called like that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, sure but using data.openeo limits you to using band , time, x and y only now no? Because the moment you add a dimension and run resample it fails. We are looking into ways to extend the support of other dimensions. We can certainly reuse data.openeo for something but ultimately extend the resample_spatial dim support.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could do something like this maybe:

    dims = list(data.dims)

    known_dims = [
        data.openeo.band_dims[0],
        data.openeo.temporal_dims[0],
        data.openeo.y_dim,
        data.openeo.x_dim,
    ]

    other_dims = [dim for dim in dims if dim not in known_dims]

    data_cp = data.transpose(*other_dims, *known_dims)

This way we can keep the data.openeo features plus add other dimensions. Of course, some limits and checks would need to be performed on the other dimensions as we cannot realistically allow and support everything.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Valentina!

I updated the latest commit based on your suggestion to keep the data.openeo functionality. The dummy align is set as well to prevent errors from occurring. Let me know what you think!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

The tests are currently failing because you need to pre-commit run --all-files on the branch.

@jzvolensky jzvolensky marked this pull request as ready for review May 6, 2024 12:15
@ValentinaHutter
Copy link
Collaborator

Thanks for the contribution! Will add this to the latest release!

@ValentinaHutter ValentinaHutter merged commit ea996ba into Open-EO:main May 6, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants