Scale the application

[severo]

Both uvicorn and pm2 allow specifying the number of workers. pm2 seems interesting since it provides a way to increase or decrease the number of workers without restart.

But before using multiple workers, it's important to instrument the app in order to detect if we need it (eg: monitor the response time).

[julien-c]

note that orthogonally to scalability, we need to "guarantee" that row fetches from moon-landing will be under a specified latency (to be discussed), even in the case of cache misses in datasets-preview-backend

because the data will be needed at server-rendering time, for content to be parsed by Google

What's a reasonable latency you think you can achieve?

If it's too long we might want to pre-warm the cache for all (streamable) dataset, using a system based on webhooks from moon-landing for instance

[julien-c]

Re. scalability, let's not focus on this too much because we'll work with the Infra team to productionize this when the time comes, and architecture will surely change.

The current scope of work is to deploy a prototype that lets us iterate on the product definition, interaction with the datasets lib, etc (cc @n1t0 for visibility and context!)

[severo]

Philip's recommendation on https://huggingface.slack.com/archives/CTKK32GE8/p1630590091040300?thread_ts=1630588820.038000&cid=CTKK32GE8:

and for scaling I would definitely recommend to switch to docker/container and then there are many options how you can make your service scalable (edited)

[severo]

Replaced with #223