Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

xarray .count() gives -1 when it should be 1? #2770

Closed
ru111 opened this issue Feb 13, 2019 · 5 comments · Fixed by #2892
Closed

xarray .count() gives -1 when it should be 1? #2770

ru111 opened this issue Feb 13, 2019 · 5 comments · Fixed by #2892

Comments

@ru111
Copy link

ru111 commented Feb 13, 2019

Data array da I'm working with looks like:

<xarray.DataArray 'T' (time: 365, latitude: 256, longitude: 512)>
[47841280 values with dtype=float32]
Coordinates:
  * time         (time) datetime64[ns] 1979-01-01T09:00:00 ... 1979-12-31T09:00:00
  * latitude     (latitude) float32 89.462944 88.766945 ... -89.462944
  * longitude    (longitude) float32 0.0 0.703125 ... 358.59375 359.29688

Now int(da.time.count()) gives 365 which is correct. However when I select a time coordinate da2 = da.isel(time=0) the resulting array has one time coordinate:

<xarray.DataArray 'T' (latitude: 256, longitude: 512)>
[131072 values with dtype=float32]
Coordinates:
    time         datetime64[ns] 1979-01-01T09:00:00
  * latitude     (latitude) float32 89.462944 88.766945 ... -89.462944
  * longitude    (longitude) float32 0.0 0.703125 ... 358.59375 359.29688

and int(da2.time.count()) gives me -1 instead of 1. I wasn't sure if this was a bug or I wasn't selecting the coordinate properly - if this is the case how should I call the time coordinate to count it correctly?

@ru111 ru111 changed the title xarray .count() gives -1 when there it should be 1? xarray .count() gives -1 when it should be 1? Feb 13, 2019
@max-sixty
Copy link
Collaborator

Thanks for the issue @ru111

That does seem weird. Do you have a minimally reproducible example?

@andrew-c-ross
Copy link
Contributor

I can reproduce OP when the coordinate is a datetime64, but the problem does not occur with int64s (or float64s).

datetime coordinate:

In [1]: import xarray

In [2]: xarray.__version__
Out[2]: '0.11.2'

In [3]: import datetime as dt

In [4]: da = xarray.DataArray([0], coords=[('time', [dt.datetime(1979, 1, 1)])])

In [5]: da.isel(time=0).time.count()
Out[5]:
<xarray.DataArray 'time' ()>
array(-1)
Coordinates:
    time     datetime64[ns] 1979-01-01

int coordinate:

In [6]: da = xarray.DataArray([0], coords=[('time', [0])])

In [7]: da.isel(time=0).time.count()
Out[7]:
<xarray.DataArray 'time' ()>
array(1)
Coordinates:
    time     int64 0

@shoyer
Copy link
Member

shoyer commented Feb 15, 2019

The problem seems to come up for scalar datetime64 arrays:

>>> xarray.Variable((), np.datetime64('2000-01-01')).count()
-1

@shoyer
Copy link
Member

shoyer commented Feb 15, 2019

Apparently isnull(datetime64_scalar_array) returns a Python boolean False. This gets inverted by ~ to -1 instead of True in our implementation of count:

return np.sum(~isnull(data), axis=axis)

To fix this, we should either switch ~x to np.logical_not(x).

@shoyer shoyer added the bug label Feb 15, 2019
@dcherian
Copy link
Contributor

Though for datetime64 we should use ~np.nat?

dnowacki-usgs pushed a commit to dnowacki-usgs/xarray that referenced this issue Apr 13, 2019
BUG: Fix pydata#2770 by changing ~ to np.logical_not()
TST: Add test for scalar datetime64 value
DOC: Add to whats-new.rst
shoyer pushed a commit that referenced this issue Apr 14, 2019
BUG: Fix #2770 by changing ~ to np.logical_not()
TST: Add test for scalar datetime64 value
DOC: Add to whats-new.rst
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants