Change curry to use functools.partial

[eriknw]

curry instances always return other curry instance if the arguments are not fully applied; previously, they could sometimes return functools.partial instances.

The performance of calling curry instances is better than before, but still not as good as calling partial. To achieve the performance of partial, use curried_func.call.

This PR addresses the idempotency issue from #147. It does not, however, split curry into class Curry/def curry as discussed in #147 (or as indicated by the name of this branch). I think curry performs just fine as a single class.

[eriknw]

I performed the following benchmarks for old and new curry, which shows the new curry is significantly improved, but still significantly slower (x2-x3) than partial:

from toolz import curry
from functools import partial

def f(*args, **kwargs):
    pass

c1 = curry(f)
p1 = partial(f)

%timeit p1()
%timeit c1()
if hasattr(c1, 'call'):
    %timeit c1.call()

%timeit p1(1)
%timeit c1(1)
%timeit p1(1, 2)
%timeit c1(1, 2)
%timeit p1(x=1)
%timeit c1(x=1)
%timeit p1(x=1, y=2)
%timeit c1(x=1, y=2)

c2 = curry(f, 1, 2)
p2 = partial(f, 1, 2)

%timeit p2()
%timeit c2()
%timeit p2(3)
%timeit c2(3)
%timeit p2(3, 4)
%timeit c2(3, 4)
%timeit p2(x=3)
%timeit c2(x=3)
%timeit p2(x=3, y=4)
%timeit c2(x=3, y=4)

c3 = curry(f, a=1, b=2)
p3 = partial(f, a=1, b=2)

%timeit p3()
%timeit c3()
%timeit p3(3)
%timeit c3(3)
%timeit p3(3, 4)
%timeit c3(3, 4)
%timeit p3(x=3)
%timeit c3(x=3)
%timeit p3(x=3, y=4)
%timeit c3(x=3, y=4)

[mrocklin]

While I like the idea of precomputing a partial and keeping it around in call I think I'm mostly against this change.

To me a 2-3x slowdown on curried function calling is a big deal. I actually use map(get(...)) quite a bit and if all I'm doing is basic data selection then it does occasionally become a bottleneck. This slowdown would make me switch down to using list comprehensions.

At the same time I only rare notice that I've been given a partial rather than a curry object, and I don't think I've ever actually cared. Maybe I need more motivation here to understand the value. I guess this would be an issue due to docstrings, function names, and pickling. The inconsistent output interface doesn't actually bother me that much (I'm not sure why not though, I think that it should).

Anyway, I'm -0.5 on this as is. Thoughts?

[eriknw]

Take two. See latest push. I'll discuss more later.

[eriknw]

I expected this PR would need vetting, and I'm sure it will continue to develop.

I agree that a factor of 2x is important. One approach to achieve this is to sometimes return a partial object. The approach in 9fc06dd is to never return a partial, but instead adds a call attribute, which can be used to quickly call any curried function. The code is simple, the behavior is consistent, and the efficient calling of a curried function is explicit.

Let me illustrate what prompted some of the changes in this PR:

curried_get = curry(get)
fast_cheetah = curried_get('cheetah')
fast_cheetah(zoo)  # this is a fast partial

slow_cheetah = curry(get, 'cheetah')
slow_cheetah(zoo)  # this is a slow curry with the current implementation

While developing the class Curry/def curry split, I thought, "Aha, I can make slow_cheetah a fast partial just like fast_cheetah, and it's also intuitive that curry(get)('cheetah') is equivalent to curry(get, 'cheetah')," so I made it so. With curry as a function, it is able to return a Curry or partial instance. However, this caused some tests to fail.

@curry
def memoize(f, cache=None, key=None):
    ...

add_cache = {}
@memoize(cache=add_cache)
def add(x, y):
    return x + y

@memoize(cache=add_cache) didn't work, because memoize is a partial instance that doesn't incrementally add arguments like curry. This sucks! The approach in 2ca1c7c is similar to the current behavior, except it only returns a partial if the function doesn't have keywords. curry(f, x) and curry(f)(x) are equivalent, @memoize(cache=cache) works, and curry(binop)(1) and curry(binop, 1) are both fast partials. However, your example map(get(...), ...) is a slow Curry, because get has keyword arguments. Hence, this solution may not be ideal either.

I'll continue this discussion later. Allowing curry to sometimes return a partial introduces many questions (more than I initially realized), and there are a large permutation of possible behaviors. Hopefully this post helped illustrate some of the issues and different behaviors that can arise. When faced with so many questions and possibilities, I instinctively turned to a simple, consistent, and explicit implementation-- 9fc06dd --that introduces a way to always call the curried function efficiently.

[eriknw]

Closing. Superseded by #170.