Improve consistency of similar methods operating on sparse matrices and flesh them out

[Sacha0]

This pull request's first commit: (1) makes similar methods operating on SparseMatrixCSCs more consistent both among themselves and with other similar methods in Base; and (2) fleshes those methods out. The second commit thoroughly tests those similar methods. The first commit fixes #17456, and the second covers (tests) #17456's underlying cause. Best!

[Sacha0]

The AV i686 failure appears unrelated, but the Travis i686 failure appears real. I can't reproduce the latter locally though, and inspecting the test output

From worker 4:       * sparse               Test Failed
From worker 4:    Expression: sparse(Fs[:L]) ≈ Lfp
From worker 4:     Evaluated: 
From worker 4:      [1, 1]  =  6.08276
From worker 4:      [2, 1]  =  -7.06916
From worker 4:      [3, 1]  =  1.97279
From worker 4:      [2, 2]  =  6.93015
From worker 4:      [3, 2]  =  -0.296394
From worker 4:      [3, 3]  =  0.142334 isapprox [6.08276 0.0 0.0; -7.06916 6.93015 0.0; 1.97279 -0.296394 0.142334]

suggests that the result is correct, but perhaps not within some tolerance? Thanks!

[Pbellive]

Hi. I'm wondering if there's still a plan to fix the CI issues with this PR and get it merged? I just ran into #17456 on Julia 0.5.1 (commit a6c55c5*). Also checked that it still exists on master (commit d6088fc*). I feel a bit silly asking about this when I don't know enough to debug why the CI checks are failing but I thought I'd check in to see if it's just been forgotten about. Is it worth merging this work into an up to date version of Base and submitting a new PR? If so I could try doing that. Thanks!

[Sacha0]

(Leaving a breadcrumb to #22009 and #23905 for changes during resuscitation of this pull request.)

[Sacha0]

Rewritten. What say you now, CI?

[mbauman]

Possible alternative: have similar return a reshaped sparse vector matrix for cases where N > 2. This has two advantages:

• Doesn't unexpectedly blow up memory use
• Will get better performance over time as we add optimizations for reshaped sparse arrays.

[Sacha0]

@mbauman, did you intend the above comment for #23905? How to handle Dim{0} and Dim{N>2} are somewhat orthogonal to this pull request, which primarily addresses #17456 and related issues. Best!

[mbauman]

I guess, sure, but doesn't this PR also implicitly fix #23905 by removing the definition for similar(S::SparseMatrixCSC, ::Type{Tv}, d::Dims)? I guess it doesn't matter so much where we discuss it; it was just a brainstorm suggestion. I'd wager folks who use sparse matrices would almost rather an immediate error here rather than blowing up their storage. But I'm not one of those folks, so it's not a strong suggestion.

[Sacha0]

Yes, as a side effect this pull request partially addresses #23906, and the sequel for SparseVectors should similarly cover another part as a side effect. But this pull request's primary target is #17456 and related issues, and I would much appreciate keeping this thread tight and decoupled from broader design questions :). (Your suggestion is quite clever by the by!)

[Sacha0]

All test failures appear unrelated, except for the CircleCI i686 failure, which is bizarre:

Some tests did not pass: 1817 passed, 1 failed, 0 errored, 0 broken.sparse/sparse: Test Failed
  Expression: transpose(view(speye(10), 1:5, 1:5)) ≈ eye(5, 5)
   Evaluated: 
  [1, 1]  =  1.0
  [2, 2]  =  1.0
  [3, 3]  =  1.0
  [4, 4]  =  1.0
  [5, 5]  =  1.0 ≈ [1.0 0.0 0.0 0.0 0.0; 0.0 1.0 0.0 0.0 0.0; 0.0 0.0 1.0 0.0 0.0; 0.0 0.0 0.0 1.0 0.0; 0.0 0.0 0.0 0.0 1.0]

On the one hand, this failure is quite similar to the failure that blocked this pull request better than a year ago on Travis i686: The failure is also on i686, the report suggests that the test should pass, and I cannot reproduce the failure locally. On the other hand, and what makes the failure seem particularly strange, is that Travis i686 now passes. Any insight would be much appreciated!

[Sacha0]

(Seems the failure was either unrelated or related but nondeterministic (though the test is deterministic), as CircleCI i686 passed this round.)

[Sacha0]

Another form of the same failure appeared this round on Travis x86_64 Linux:

Error in testset sparse/sparse:
Test Failed
  Expression: f(arr, 1) ≈ f(farr, 1)
   Evaluated: 
  [1, 2]  =  -0.47132
  [1, 3]  =  -0.463003
  [1, 4]  =  0.0
  [1, 5]  =  -0.848927
  [1, 6]  =  -0.509547
  [1, 7]  =  -0.478014 ≈ [0.0 -0.47132 -0.463003 0.0 -0.848927 -0.509547 -0.478014]

So this issue is not confined to 32-bit platforms. I will see whether I can reproduce this locally via repeated runs.

Update: On the sixth consecutive local (macOS) run of Base.runtests("sparse/sparse") I encountered another form of this failure. So this failure is not confined to linux either.

[Sacha0]

Reduced to an issue with vecnorm(A::SparseMatrixCSC, ...), which should be fixed by #24204. Best!

[Sacha0]

Confirming that #24204 seems to fix the stress test I have locally exercising the nondeterministic CI failure. Cycling CI. Best!

[Sacha0]

Absent objections or requests for time, I plan to merge these changes this evening (assuming CI clears). Best!

[Sacha0]

Thanks all! :)