From 8fd7e929a27553dc8aa56d04c0837dba0b9f68cb Mon Sep 17 00:00:00 2001 From: Nathan Stitt Date: Wed, 30 Aug 2023 09:32:19 -0500 Subject: [PATCH] Document turning off proxy_buffering when api is streaming I hit this when running the api service behind an nginx proxy. The responses were streamed but all arrived at once. I'm guessing this configuration is a pretty standard way to add fast chat to an existing domain with SSL and mentioning it here might help someone out. --- docs/openai_api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/openai_api.md b/docs/openai_api.md index f69cc4f00..0c555a60e 100644 --- a/docs/openai_api.md +++ b/docs/openai_api.md @@ -62,7 +62,7 @@ completion = openai.ChatCompletion.create( print(completion.choices[0].message.content) ``` -Streaming is also supported. See [test_openai_api.py](../tests/test_openai_api.py). +Streaming is also supported. See [test_openai_api.py](../tests/test_openai_api.py). If your api server is behind a proxy you'll need to turn off buffering, you can do so in Nginx by setting `proxy_buffering off;` in the location block for the proxy. ### cURL cURL is another good tool for observing the output of the api.