Questions about proxying with nginx and keep-alives

[wgwz]

Hey @singingwolfboy, I am hoping you can offer some advice on how to implement keep-alives; via ping and pong between the client and server. Here's the event-source RFC for ping and pong. I've come to the conclusion that I need to implement keep-alive's. But just in case I am asking for the wrong information here are some specifics.

I am getting a "504 gateway timeout error" when proxy-serving with nginx. When accessing the stream via gunicorn, I do not get a 504 timeout. So it's definitely on the nginx side. The 504 timeout happens on /stream and on endpoints like /stream?channel=pocoo. Here is the full nginx config I am using.

I've followed this serverfault answer for setting up the nginx configuration, which recommends dispatching the X-Accel-Buffering and Cache-Control headers for the sse endpoints. But even with this, I still get the 504 timeout.

@sse.after_request                                                                                            
def add_nginx_sse_headers(response):                                                                          
    response.headers['X-Accel-Buffering'] = 'no'                               
    response.headers['Cache-Control'] = 'no-cache'                             
    return response

At this point I'm not exactly sure what I'm doing wrong. I'm taking shots in the dark :) but I am glad to distill some of this into docs for the Flask-SSE (once I've figured it out). I would really like to hear your advice. Thanks!

Also I have tried setting keepalive_timeout 0; in the nginx config. I thought that would prevent the timeout, but no, I still get the 504 :( And that is what brought to implementing keep-alive's.

[singingwolfboy]

Hi @wgwz, I think you may be mixing up different technologies. Flask-SSE is an implementation of the Server Sent Events specification, but the RFC you linked to for ping and pong is part of the WebSockets specification. As the name implies, server sent events are sent from the server -- it is one-way communication. It's not possible for the client to send a message to the server as part of server sent events.

If you want to use WebSockets with Flask, you may want to look into Flask-Sockets instead.

[wgwz]

Oh, thanks for clarifying that. I got a bit lost in all this as you can tell. But I am definitely using Flask-SSE :) and not using websockets. So everything aside from the ping/pong stuff is still relevant.

[wgwz]

I do realize this is kind of outside of the scope of the project. But I thought I'd try asking here.

[singingwolfboy]

Configuring nginx appropriately would definitely be a good thing to add to the documentation, once we figure out how to do it! However, I will admit that I've never actually run this project in production, so I don't know the correct way to do this. Has anyone else done this successfully?

[wgwz]

The serverfault example is the most in-depth answer I could find. There is also a segment in MDN's SSE article that hints at being able to keep the connection alive:

Note: The comment line can be used to prevent connections from timing out; a server can send a comment periodically to keep the connection alive.

But I find that a bit vague. I'll keep posting as I learn more, but I hope to hear from others!

[wgwz]

Here is another thread with a possible solution. It configures nginx to catch the 504 error and return a 200. This rule is only applied for the event-source endpoints.

[wgwz]

Well I made some good progress on this. I need to clear up the minimal configuration for nginx, and understand exactly why each piece is needed. But one thing that does resolve this issue, is using a keep-alive message (no hacking of the nginx config).

Without a keep alive message the nginx server will use keepalive_timeout when its value is reached, resulting in 504 gateway timeout. With a keep-alive message you're periodically publishing something to the stream. This prevents the connection from closing. The way I've implemented this, currently, is by using Celery-beat.

@celery.task                                                                                                  
def publish_keep_alive():                                                                                     
    sse.publish({'message': 'keep alive'})

celery.conf.CELERYBEAT_SCHEDULE = {                                                                       
    'keep-alive-every-30-seconds': {                                                                      
        'task': 'chat_app.periodic_tasks.publish_keep_alive',              
        'schedule': 30.0,                                                                                 
        'args': None                                                                                      
    },                                                                                                    
}

This needs to be done for each channel. But I would enjoy finding a solution not using Celery. If anyone has ideas... I think it'd be better for the docs to have a lower-level solution than having to say "hey, you're going to have to use Celery for this" :)

[singingwolfboy]

Yeah, that would work! You could even use the type argument to sse.publish() so that keep-alive messages are their own type. That way, you could publish your meaningful messages as a separate type, and code up your Javascript to only pay attention to messages of the meaningful type. As an example:

@celery.task                                                                                                  
def publish_keep_alive():                                                                                     
    sse.publish({'message': 'keep alive'}, type='keepalive')

@app.route('/test')
def testing():
    sse.publish({'value': 42}, type='data')

var source = new EventSource("{{ url_for('sse.stream') }}");
// Because I'm adding an event listener for 'data', it will only respond
// to events of type 'data', and ignore the 'keepalive' events! 
source.addEventListener('data', function(event) {
  // ...
}, false);

[wgwz]

Good point on the message type, thanks!