Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add elemwise ops for GCXS #412

Merged
merged 3 commits into from
Oct 2, 2020
Merged

Add elemwise ops for GCXS #412

merged 3 commits into from
Oct 2, 2020

Conversation

daletovar
Copy link
Contributor

  • Moves element-wise operations
  • For now, just converts GCXS to COO to compute elemwise and then converts again at the end
  • If COO and GCXS interact, the output will be COO
  • Moves methods that rely on elemwise to _sparse_array: round, clip, astype, mean, var, std
    In the future I like the idea of converting all SparseArrays to a linearized representation and do the computations on that and then convert the result to the desired format. It looks like this is basically what's done anyway considering that _match_arrays works with linearized coords. I just think it's currently wasteful to convert GCXS -> COO -> linearized -> COO -> GCXS.

@codecov
Copy link

codecov bot commented Sep 24, 2020

Codecov Report

Merging #412 into master will increase coverage by 0.10%.
The diff coverage is 96.62%.

@@            Coverage Diff             @@
##           master     #412      +/-   ##
==========================================
+ Coverage   95.06%   95.17%   +0.10%     
==========================================
  Files          19       19              
  Lines        2819     2818       -1     
==========================================
+ Hits         2680     2682       +2     
+ Misses        139      136       -3     

@hameerabbasi
Copy link
Collaborator

In the future I like the idea of converting all SparseArrays to a linearized representation and do the computations on that and then convert the result to the desired format.

Unfortunately linearized coords don't work for broadcasting cases.

Copy link
Collaborator

@hameerabbasi hameerabbasi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @daletovar. Overall, this looks pretty awesome. I've suggested a couple of minor changes.

Comment on lines 157 to 161
self.data = other.data
self.indices = other.indices
self.indptr = other.indptr
self.compressed_axes = other.compressed_axes
super().__init__(other.shape, fill_value=other.fill_value)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
self.data = other.data
self.indices = other.indices
self.indptr = other.indptr
self.compressed_axes = other.compressed_axes
super().__init__(other.shape, fill_value=other.fill_value)
return self.copy(deep=False)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't quite see how this makes a copy of other. I tried this and it makes most of the tests fail.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a misunderstanding on my part.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gotcha, no worries.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you try self.__dict__ = other.__dict__.copy()?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, definitely. Should I change it for COO as well?

sparse/_sparse_array.py Show resolved Hide resolved
sparse/tests/test_elemwise.py Outdated Show resolved Hide resolved
sparse/tests/test_elemwise.py Outdated Show resolved Hide resolved
sparse/tests/test_elemwise.py Outdated Show resolved Hide resolved
sparse/tests/test_elemwise.py Outdated Show resolved Hide resolved
@daletovar
Copy link
Contributor Author

Thanks so much for the review, @hameerabbasi.

@hameerabbasi hameerabbasi merged commit 3d4453a into pydata:master Oct 2, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants