using asyncio.iscoroutinefunction() on a functools.partial object

[mathieui]

BPO	23519
Nosy	@gvanrossum, @vstinner, @mathieui, @1st1
Files	example_partial.py

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2015-02-26.17:52:33.200>
created_at = <Date 2015-02-25.11:00:39.941>
labels = ['type-feature', 'invalid', 'expert-asyncio']
title = 'using asyncio.iscoroutinefunction() on a functools.partial object'
updated_at = <Date 2017-06-09.02:28:01.054>
user = 'https://github.com/mathieui'

bugs.python.org fields:

activity = <Date 2017-06-09.02:28:01.054>
actor = 'gvanrossum'
assignee = 'none'
closed = True
closed_date = <Date 2015-02-26.17:52:33.200>
closer = 'gvanrossum'
components = ['asyncio']
creation = <Date 2015-02-25.11:00:39.941>
creator = 'mathieui'
dependencies = []
files = ['38246']
hgrepos = []
issue_num = 23519
keywords = []
message_count = 8.0
messages = ['236569', '236570', '236592', '236660', '236668', '236693', '295475', '295478']
nosy_count = 5.0
nosy_names = ['gvanrossum', 'vstinner', 'mathieui', 'yselivanov', 'curtmcd']
pr_nums = []
priority = 'normal'
resolution = 'not a bug'
stage = None
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23519'
versions = ['Python 3.4', 'Python 3.5']

[mathieui]

Using iscoroutinefunction() on an object returned by functools.partial() should return True if the function wrapped was a coroutine function.

(a recursive loop like the one in asyncio/events.py get_function_source() may be what needs to be done)

[vstinner]

Take a look at asyncio.events._get_function_source() which supports wrapped functions and functools.partial.

Can you propose a patch?

If iscoroutinefunction() becomes "expensive", we might make checks optional, only run them in debug mode.

[gvanrossum]

I recall discussing this before (maybe on the tulip list). I am firmly against. It is a slippery slope -- why inspect a partial but not a lambda? Plus there is no use case.

[vstinner]

Plus there is no use case.

Mathieu: can you maybe give some examples? How are you using functools.partial with coroutines?

[mathieui]

Using functools.partial with coroutines would be mostly out of convenience, in order to avoid having factories in that return parametrized coroutine functions. I guess in such cases it might be better to create a two-lines wrapper around partial() to make it return a coroutine rather than change the stdlib for that.

In the attached file is an example of such use, where EventNotifier is a Protocol which receives external events and triggers event handlers based on that, and where the add_event_handler function checks if the handler is a coroutine function. In which case it uses asyncio.async to schedule the handler call; otherwise it uses loop.call_soon.

You can close this, I guess.

[gvanrossum]

Yeah, your "add event handler" routine shouldn't be so picky to insist that iscoroutinefunction() returns True. It should just call the thing and verify that it has returned a coroutine object (asyncio.iscoroutine()).

[curtmcd]

There are use cases for this. I hit the problem and eventually wound up here, so I'd like to make a case to re-open.

My project has a callback registry of asyncio handler routines.

register_callback(condition1, handler1)
register_callback(condition2, handler2)
...

I want to register many callbacks, but use only one handler and an argument to differentiate it. I realize our callback systems should provide for a cookie, but it doesn't.

register_callback(condition1, functools.partial(handler, 'detail1'))
register_callback(condition2, functools.partial(handler, 'detail2'))

The callback registry makes sure iscoroutinefunction(handler) because we don't want to defer error checking to the distant future. But iscoroutinefunction() returns False for the partial function. I was hopeful that this might work, but alas, no:

register_callback(condition1,
                  asyncio.coroutine(functools.partial(handler, 'detail1')))

[gvanrossum]

Use a lambda instead of partial. It's more pythonic.

[graingert]

this is fixed in #79071

[graingert]

looking at this more closely - this is a bug in asyncio.iscoroutinefunction that was fixed for inspect.iscoroutinefunction

this code will now work - if you use async def handler(detail):

register_callback(condition1, functools.partial(handler, 'detail1'))
register_callback(condition2, functools.partial(handler, 'detail2'))

here's a demo:

import inspect
import functools
import asyncio

async def demo(v):
    pass


@asyncio.coroutine
def demo_old(v):
    pass


print(f"{inspect.iscoroutinefunction(functools.partial(demo, v=1))=}")
print(f"{asyncio.iscoroutinefunction(functools.partial(demo, v=1))=}")
print(f"{inspect.iscoroutinefunction(functools.partial(demo_old, v=1))=}")
print(f"{asyncio.iscoroutinefunction(functools.partial(demo_old, v=1))=}")

# this prints:
# /home/graingert/projects/bar.py:10: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead
#   def demo_old(v):
# inspect.iscoroutinefunction(functools.partial(demo, v=1))=True
# asyncio.iscoroutinefunction(functools.partial(demo, v=1))=True
# inspect.iscoroutinefunction(functools.partial(demo_old, v=1))=False
# asyncio.iscoroutinefunction(functools.partial(demo_old, v=1))=False

[gvanrossum]

So can I close this? (I'm not sure what happened here exactly, I don't recall reopening, maybe GitHub did something funky?)

[graingert]

So can I close this? (I'm not sure what happened here exactly, I don't recall reopening, maybe GitHub did something funky?)

ah I thought you were cryptically telling me the issue was still valid when you re-opened this - it caused me to re-read the issue and find that it is actually still a bug - although now only relevant on python 3.10 and maybe on 3.11

[graingert]

I have a PR to fix it here #94907 but I think it would be better to drop asyncio.iscoroutinefunction, asyncio.coroutines._is_coroutine and all their usages.

This would require making _wrap_awaitable a real async fn - or a @types.coroutine function

[gvanrossum]

Honestly if this is fixed in 3.11 by deleting asyncio.coroutine I would leave 3.10 alone.

Deleting asyncio.iscoroutinefunction would be problematic since it is ostensibly public so would have to be deprecated. But it sounds like you believe it could just become an alias for inspect.iscoroutinefunction if we change _wrap_awaitable? That sounds like we could do (though it's a bit late for 3.11 -- in main a.k.a. 3.12 I'd be okay with this though).

[graingert]

I opened an issue for that removal #94912

[gvanrossum]

Okay, closing this issue as fixed.