Initial proof-of-concept for uvicorn compatibility

[tomchristie]

What do these changes do?

This pull request adds a (message, channels) callable interface to the App instance, as described by the uvicorn messaging interface... http://www.uvicorn.org/#messaging-interface

There are several aspects that are currently left incomplete:

• Needs to pass a payload interface object, which bridges the body stream data in channels[body] into the StreamReader interface expected by aiohttp.
• Needs to pass a writer instance, which bridges channels[reply] into the interface expected by aiohttp.
• Needs to pass a task instance. I'm not clear where that interface gets used in aiohttp. Not something that we currently expose in uvicorn's proposed messaging interface.

There are a few flags currently left hardcoded in the RawRequestMessage that we instantiate.

Are there changes in behavior for the user?

Running a basic "hello world" application, there's a significant performance improvement...

import asyncio
from aiohttp import web

async def hello(request):
    return web.Response(body=b"Hello, world")

app = web.Application()
app.router.add_route('GET', '/', hello)

if __name__ == '__main__':
    web.run_app(app)

Running under the aiohttp server, with wrk -d20s -t10 -c200 http://127.0.0.1:8080/...

$ python ./app.py
Latency: 82.38ms
Requests/sec: 2417.26

Running under uvicorn (single worker)...

$ uvicorn app:app
Latency: 56.99ms
Requests/sec: 6661.89

This shouldn't be taken as definitive, there are a couple more objects that we'd need to instantiate in order to present the request body to aiohttp applications, and to allow streaming responses, but I'm raising the pull request as an initial demonstration, to see if there's any interest in my pursuing this any further or not.

It's also worth noting that the uvicorn implementation does include high/low watermark flow control on both incoming and outgoing data, keepalive/close behavior, websocket upgrades & chunked requests/response.

The messaging interface should currently still be considered a moving target, still in a design phase, but I'm interested in knowing if it's something that you might consider, if we could demonstrate that it'd allow aiohttp to run under a faster server implementation?

Checklist

n/a - the pull request in its current state is intended as an initial proof-of-concept.

[asvetlov]

Hmm.
Your benchmarks are not correct.

1. Run your app.py snippet untouched: 5500 RPS

2. Install uvloop by adding the following lines:

import uvloop
asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

uvcorn installs uvloop by default, why not do it for benchmark?
As result we have 6600 RPS

3. Disable access log: web.run_app(app, access_log=None)
   RPS: 8000

Thus I don't see any performance improvement from guvcorn.

P.S.
Python 3.6.1
aiohttp master
uvloop 0.8.0
tested on local laptop

[tomchristie]

Yup, noted - so I see them aiohttp under the two different deployments running at comparable throughputs now. (both at 6400r/s on machine)

The point of having a well specified interface remains: It'd give us a baseline that multiple applications can build on, and that multiple different server implementations can expose. I'd love to have a productive conversation with you folks about what you'd expect that interface to look like, tho I'll likely do that over on aio-libs if that is something we're able to move forward with.

I'll close off this pull request for now.

[fafhrd91]

seems you'd like to use aiohttp.web on top uvicorn, but that is just one file web_protocol.py. is it worse of new dependency.
I checked uvicorn code, it looks nice and clean but too immature. also I could not find tests.

[tomchristie]

it looks nice and clean but too immature.

I'm not expecting aiohttp to adopt this immediately. What I am trying to do tho is to kickstart a productive conversation around having a clean, minimal and well-specified interface between server implementations and application frameworks.

We have that for synchronous frameworks with WSGI, but there's no equivalent for asyncio, and as a result all the asyncio web frameworks that exist are each reimplementing all the gnarly details of server handling from scratch.

A far better situation would be if we had a really nicely specified interface, so that we can properly separate server implementations from application frameworks. The messaging interface described in the uvicorn documentation is an example of what that interface could look like.

that is just one file web_protocol.py

It's at least worker.py, web_protocol.py, _http_parser.pyx, _websocket.pyx and possibly some other bits. Given aiohttp's maturity I'm not expecting that you folks would want to remove the existing built-in server anytime soon, which is why I'm starting off by demonstrating a proof-of-concept that exposing a (message, channels) coroutine interface alongside the existing built-in server is something that you'd be able to do.

[fafhrd91]

i like the idea, but stringly typed components looks like step backward. and we should be able to justify another obstruction layer.

also do we really need something like WSGI? I don't see python as a first choice for new high-load projects nowadays. messaging interface introduces new abstraction layer which increases already high learning curve for asyncio projects.

regarding files, we use _http_parser.pyx and _websocket.pyx on both side (client and server)

[tomchristie]

do we really need something like WSGI?

We absolutely need a clearly specified server/application interface layer, yes.

Suppose you're a python developer and you've got a great new idea for a web framework, what's your starting point at the moment? Implement the protocol, choose and integrate an http parser, figure out how to get flow control right (in both directions), add ssl contexts and signal handling, design a messaging interface that abstracts those details away from the rest of the codebase, add request logging, figure out clean shutdown etc.etc.etc.

Equally consider existing WSGI based Python web frameworks that are considering evolving to support asyncio deployment modes. Right now they'd either have to understand and implement all of those different aspects, and clone a large amount of existing implementation code across from one of the existing asyncio frameworks.

Have a standard interface would also allow use to have shared middleware, eg. equivalents to werkzeug's debug WSGI middleware or whitenoise's static file serving WSGI middleware.

increases already high learning curve for asyncio projects.

I'd argue the opposite. By getting the abstractions right, we remove a huge number of details that developers would otherwise need to get their heads around.

Consider a hello world application, with the proposed interface...

async def app(message, channels):
    await channels['reply'].send({
        'status': 200,
        'headers': [
            [b'content-type', b'text/plain'],
        ],
        'content': b'Hello, world'
    })

Without having some kind of standardized interface at that level we're instead telling developers "implementing a server is easy, just subclass asyncio.Protocol, implement data_recieved, hook up an http parser, and you're good to go" (*)

(*) vastly simplified.

introduces new abstraction layer

Even if you don't have a tightly specified server/application interface, you still end up having an interface boundary between the two. It's just that it ends up being different in each web framework implementation. It'll just be less thoroughly designed as a result, and not interoperable in any way.

I don't see python as a first choice for new high-load projects nowadays.

The need for a clean interface layer stands whether or not that's true. However one of the reasons that I think this work is particularly important at the moment is because I think Python's on the cusp of being able to change that, and have an ecosystem of asyncio frameworks that are highly performance-competitive against the rest of the web landscape.

[asvetlov]

@tomchristie do you visit EuroPython this year?
I'd like to discuss questions raised by you.

[tomchristie]

@asvetlov - Yup, although only briefly. I'll be around Monday afternoon, and all day Tuesday.

[fafhrd91]

In general I agree, unified interface should help in long run. Problem I see is time, async much more complicated than simple wsgi interface. In best case stabilization of interface and adoption would take couple years. I am not sure if in two years python will be good choice for high profile/high load projects. I see tendency, python get pushed out of web dev. Well, at least in SF. Regarding performance, from my experience - python is slow. And all we do here just optimizing web framework, which is in project bigger than hello world accommodate very few percent of overall cpu usage. it is really hard to reason about performance. Also python does not scale with development team. This is just my opinion, nothing else. And this is the reason why I am skeptical about wsgi for asyncioz

…

Sent from my iPhone

On Jun 30, 2017, at 3:49 AM, Tom Christie ***@***.***> wrote: do we really need something like WSGI? We absolutely need a clearly specified server/application interface layer, yes. Suppose you're a python developer and you've got a great new idea for a web framework, what's your starting point at the moment? Implement the protocol, choose and integrate an http parser, figure out how to get flow control right (in both directions), add ssl contexts and signal handling, design a messaging interface that abstracts those details away from the rest of the codebase, add request logging, figure out clean shutdown etc.etc.etc. Equally consider existing WSGI based Python web frameworks that are considering evolving to support asyncio deployment modes. Right now they'd either have to understand and implement all of those different aspects, and clone a large amount of existing implementation code across from one of the existing asyncio frameworks. Have a standard interface would also allow use to have shared middleware, eg. equivalents to werkzeug's debug WSGI middleware or whitenoise's static file serving WSGI middleware. increases already high learning curve for asyncio projects. I'd argue the opposite. By getting the abstractions right, we remove a huge number of details that developers would otherwise need to get their heads around. Consider a hello world application, with the proposed interface... async def app(message, channels): await channels['reply'].send({ 'status': 200, 'headers': [ [b'content-type', b'text/plain'], ], 'content': b'Hello, world' }) Without having some kind of standardized interface at that level we're instead telling developers "implementing a server is easy, just subclass asyncio.Protocol, implement data_recieved, hook up an http parser, and you're good to go" (*) (*) vastly simplified. introduces new abstraction layer Even if you don't have a tightly specified server/application interface, you still end up having an interface boundary between the two. It's just that it ends up being different in each web framework implementation. It'll just be less thoroughly designed as a result, and not interoperable in any way. I don't see python as a first choice for new high-load projects nowadays. The need for a clean interface layer stands whether or not that's true. However one of the reasons that I think this work is particularly important at the moment is because I think Python's on the cusp of being able to change that, and have an ecosystem of asyncio frameworks that are highly performance-competitive against the rest of the web landscape. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[tomchristie]

@asvetlov I can't make EuroPython after all, due to some family issues that I need to be around for. Very happy to talk via video or on the discussion group anytime. All the best!

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.