Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Compatibility with xarray for #18 and R CMD check cleanup #85

Merged
merged 20 commits into from
Jun 12, 2024

Conversation

dblodgett-usgs
Copy link
Collaborator

Likely some other checks, I'll comment in #18 with what I checked so far.

@dblodgett-usgs
Copy link
Collaborator Author

pizzar_chain copy

Here's a social media sized logo to add to the repository in settings @keller-mark

@dblodgett-usgs
Copy link
Collaborator Author

dblodgett-usgs commented Jun 3, 2024

Been doing a whole bunch of testing and investigation of the array and nested array internals. I uncovered a handful of bugs in the process.

  1. fixed an issue with the zarr version "2" being "[2]" which stopped zarr-python from reading pizzarr zarr files.
  2. added code to deal with step size and added testing for step size selections
  3. got direct R array values working in NestedArray$set() and tested (this code appeared to be very lightly tested if at all?)
  4. added some inline documentation for selection and indexing
  5. rewrote is_scalar() to be what I think you mean it to be?!? JKLOL -- I need to learn more about the use of jsonlite::unbox and the scalar class.
  6. got selection working and tested with ":" and integers.
  7. contributed the vignette I've been working on to organize my understanding of all this stuff.

Open questions in test-nested-array.R:

  1. Should the set method of NestedArray support integer and "..."/":" indexing?
  2. Should the set method of NestedArray update the parent array ?
  # Should this be the case?!?
  expect_error(a[1:10, 1:20]$set("...", new_vals), 
               "selection must be a list of slices")
a[1:10, 1:20]$set(list(slice(1,10), slice(1,20)), new_vals)

  # it does not update the original array -- should it?
  expect_equal(a[1:10, 1:20]$as.array(),
               vals)

@dblodgett-usgs
Copy link
Collaborator Author

This latest commit gets the tests passing, but the handling of is_scalar vs is_integer is maybe not 100% -- very much worth a review on those details.

@dblodgett-usgs dblodgett-usgs changed the title Compatibility with xarray for #18 Compatibility with xarray for #18 and R CMD check cleanup Jun 7, 2024
@dblodgett-usgs
Copy link
Collaborator Author

With my latest commits, I see no errors, no warnings, and one note in check. The note is the dog.ome.zarr size issue noted in #87.

@keller-mark
Copy link
Owner

keller-mark commented Jun 12, 2024

Should the set method of NestedArray support integer and "..."/":" indexing?
Should the set method of NestedArray update the parent array ?

Good point, this could get confusing, especially when using the bracket access notation on the ZarrArray since it may be less obvious that what you get back is a NestedArray. We could check how the analogous pattern in Python behaves and then follow that. Alternatively, maybe we need a mechanism for the user to opt-in or opt-out of updating the parent (might be useful/needed in cases where the Store is read-only). Another option may be to try to get away from the NestedArray by returning plain arrays in more cases.

@keller-mark
Copy link
Owner

the handling of is_scalar vs is_integer is maybe not 100%

Good point, these and is_integer_vec should to be renamed to be more clear what they are doing. I will do that and add some more comments/tests. Luckily I think they only need to be used in a handful of cases.

Copy link
Owner

@keller-mark keller-mark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these changes / improved docs / tests are awesome

tests/testthat/test-compat.R Outdated Show resolved Hide resolved

d <- g$get_item("volcano")$get_item(list(s, s))$data

expect_equal(dim(d), c(11, 11))
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test that checks equality of the contents as well?

@@ -4,3 +4,23 @@ setup({
teardown({

})

zarr_volcano <- function() {
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice!

dblodgett-usgs and others added 2 commits June 12, 2024 15:01
Co-authored-by: Mark Keller <7525285+keller-mark@users.noreply.github.com>
@keller-mark keller-mark merged commit b4a3722 into keller-mark:main Jun 12, 2024
2 checks passed
This was referenced Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants