replicateA_, replicateM, replicateM_

[s5bug]

Spawned from a discussion about replicateA from Gitter.

Current progress:

• Implement replicateA_
• Implement replicateM
• Implement replicateM_
• Document replicateA_
• Document replicateM
• Document replicateM_
• Add replicateA_ to ApplicativeOps
• Add replicateM and replicateM_ to MonadOps

[s5bug]

Alright: I'm not aware of any useful examples of replicateM instead of replicateA, and I can't think of a simple one for replicateM_ (at least not for cats-core). Any ideas, so I can finish up the documentation and mark ready for review?

[johnynek]

Two comments:

1. For any lawful monad isn't replicateM the same as replicateA?
2. make collection traversals stack safe #3521 does an optimization to preserve stack safety. I think this implementation of replicateA undoes this and it becomes stack unsafe.

[s5bug]

If Traverse becomes stack-safe, then yes, replicateA and replicateM should have the same behavior (however replicateA_ could still blow the stack when given a huge number).

Regarding "undoing": I did not implement replicateA myself, perhaps my branch is out-of-date with master.

[johnynek]

Sorry I meant the current PR has a stack unsafe replicateA_.

Master does have a stack safe traverse for collections so replicate will pick that up but this implementation won't get it.

As a default implementation I'd rather call void on the original value, then replicateA then void.

That would be stack-safe and allow implementations that optimize void (say discard trailing maps) to leverage.

[s5bug]

You're sure that it won't still construct the intermediate list? I'm worried about the heap blowing up with a big replicateA.

[johnynek]

what I suggested definitely would create the intermediate list on the heap. But the code in this PR will create the same size on the stack. The stack is much smaller than the heap, so it will blow up first. But keep in mind: it uses O(N) heap where N is the size of the object in heap now, so it is a constant multiple of heap. You might OOM, but that means you were close before.

To recap, I would prefer my approach (since it will temporarily use heap, which is much greater than stack as a resource).

I don't know how to implement this using O(1) heap and stack. That would be cool. I think you can probably do O(log N) stack and O(log N) heap by adapting this method:

cats/core/src/main/scala/cats/data/Chain.scala

Line 687 in ce6106a

def traverseViaChain[G[_], A, B](

But I don't exactly see how. That function relies on materializing the data-structure into an immutable.IndexedSeq and then recursively slicing it into a tree. So, to avoid that materialization, you need to be able to turn the original data structure into a tree.

You could imagine something like:

class Linear[A] {
  type F
  def chunk: F[A]
  def recurse: Splitable[F]
}

trait Splitable[G[_]] extends Foldable[G] {
  def split[A](ga: G[A], items: Int): Option[Seq[Linear[A]]]
}

If you had that, you could recursively split something and walk the tree, aggregating the F[Unit] back up so that the depth only gets logarithmically deep.

[s5bug]

Would it be terrible to implement the current stack-unsafe method by wrapping it in Eval? Because that would trampoline and also not have an intermediate data structure, right?

I'll take a look at the current way Traverse works right now and check if that can be adapted to not create an intermediate structure.

[s5bug]

Ah! Using Eval is exactly what traverseViaChain does. I'll see about updating my branch and giving that a go.

[s5bug]

I ended up having an idea: replicateA_ is like folding over a range with an initial ().pure[F] and with *> as a binary operation. This means that you can implement it with constant stack space without Eval, if I'm correct.

Would appreciate some other eyes on this though, I'm quite likely to be wrong.

[larsrh]

Stray closing brace:

Suggested change

}

[s5bug]

Ah, I'm missing an opening brace. I don't know how I didn't catch that.

[johnynek]

for any lawful monad, these are exactly the same as replicateA aren't they? I am opposed to adding a function that is only different when the monad is not lawful.

[s5bug]

Yes, but it does not have the overhead of traverseViaChain/Eval/etc., if that would make a difference (would it?)

[johnynek]

People can override for efficiency but my opinion is that we should not have duplicative functions that are equivalent in general.

Secondly, generally I think we should not trade stack safety for performance since lack of stack safety makes the library randomly fail when sizes get a bit larger.

I think we should have it be as fast as possible that is also stack safe, IMO.

[s5bug]

Sounds good. Should I remove replicateM or just alias it?

Should I override replicateM with this version in StackSafeMonad?

[johnynek]

nit: pure(()) is the same as unit.

nit: if we keep this implementation, I'd like to add the tailrec annotation.

instead of using fa can we do make: val fvoid = void(fa) outside of def go, and use that?

I still want to express this function in a stack safe way if we can.

[s5bug]

I was thinking about adding @tailrec but didn't know if it was used elsewhere. Will do.

[Odomontois]

How about moving replicateA and replicateA_ to the InvariantMonoidal ?
replicateA_ can be done since (Unit, Unit) is isomorphic to the Unit
And replicateA may assume non-emptiness of List[A] in the
(_ : F[(A, List[A])]).imap{case (h, t) => h :: t}{case h :: t => (h, t)}

[armanbilge]

Superseded by #4208.