Setting up nginx + multiple NiceGUI instances

[me21]

I'm setting up a server to run the NiceGUI application on. So far so good: I installed nginx and set it up to proxy the requests to NiceGUI app (didn't use Docker container for NiceGUI - maybe in the future). Seems to work.
There's a note at https://nicegui.io/documentation/section_configuration_deployment#server_hosting, specifically:

You may also have a look at our demo for using a custom FastAPI app. This will allow you to do very flexible deployments as described in the FastAPI documentation. Note that there are additional steps required to allow multiple workers.

If I understand correctly, NiceGUI runs with one worker by default (https://github.com/zauberzeug/nicegui/blob/main/examples/fastapi/start.sh). Does that note imply that it's possible to run multiple worker processes for NiceGUI app? If so, how do we make sure that the websocket connection maps to the same worker process that served the page to the client?

[rodja]

Does that note imply that it's possible to run multiple worker processes for NiceGUI app? If so, how do we make sure that the websocket connection maps to the same worker process that served the page to the client?

You are right. It should be possible to serve several instances of the same NiceGUI app if you use a reverse proxy (load balancer) which supports session affinity. The open feature request #1539 is more about Gunicorn/Uvicorn which do not have this capability.

Specifically, it should be fairly simple to set something up with ip_hash for nginx: https://nginx.org/en/docs/http/load_balancing.html#nginx_load_balancing_with_ip_hash. It would be very cool to have such an example in our repository. Would you like to try it out and create a pull request @me21?

[me21]

I'll try to understand how it should work because we're currently talking not about multiple servers, rather about multiple worker processes on the same server, right?
I think in that case we should run multiple uvicorn processes, each with its own socket and configured for worker number == 1. And we will pass these sockets to the nginx configuration. Or do we need uvicorn in that case at all? We could run multiple systemd services for running our NiceGUI app instances, each with its own listening port.
I'll see if I can come up with something.

[xaptronic]

It seems that currently nicegui prevents you from running with multiple workers on a single machine:

https://github.com/zauberzeug/nicegui/blob/main/nicegui/ui_run.py#L157

Even though if you remove this and make a small adjustment to config.app running with multiple workers is handled a few lines later:

https://github.com/zauberzeug/nicegui/blob/main/nicegui/ui_run.py#L186

Are there implications of doing this beyond global state loss / non session affinity balancer in front?

Would it be possible to change that raise to a warning that could be disabled perhaps by env var in the case that "I know what I'm doing" and wish to run in such a configuration?

[rodja]

I'm sorry but without session affinity you will constantly get reloads/connection errors when first serving a page. The browser needs to connect back to the server which created the html. If you have multiple workers this reconnecting will fail until -- by sheer luck -- the connection lands on the right worker.

What would be a valid scenario where you "know what you are doing"?

[xaptronic]

(edit: in writing this I think I better understand the problem of doing this.. I briefly tested this out yesterday and it seemed like it was "working" but I will try a more thorough test of this)

Scenario is that I am running a NiceGUI app on a single large machine and have an nginx proxy in front of it that uses an affinity based routing strategy.

Nginx proxy also serves as an asset cache which avoids having the NiceGUI main thread tied up with serving static assets in the event a large number of users access the app.

Essential, I have a use case which transfers a largish (~10mb payload) amount of data on initial load. If I deploy new source code and there are many (50-100 and beyond) p ateople with the app open, the single process struggles initially before finally settling. I believe that having additional processes running would help spread this load.

I am also considering some other strategies related to how I can get the initial payload into the app, but does the above make a reasonable case where changing the raise to a warning make sense? (I'm thinking it would be a similar pattern elsewhere in the codebase where a warning is issued and offers a link to GH issue for more detail).

[rodja]

Glad you found a way. Could you share the setup/config for your nginx with session affinity? I thought that was something nginx could not do...

[xaptronic]

So - it is indeed not entirely possible (at least yet) just with uvicorn workers and nginx as a simple load balancer, but I do have something setup and running - is the preferred way to share that just here in thread, or would an example PR against nicegui be preferrable?

[rodja]

Depends on the size. If its only a snipped of config you can share it here. If it is not more than 200 lines overall (possibly distributed over several files) an example might be the right way. If it is even more I suggest you create a separate GitHub project, gist or explain it in the wiki: https://github.com/zauberzeug/nicegui/wiki