[FIX] The chat fails when querying with /notes while running LLAMA3 API #831

stevennt · 2024-06-21T05:23:29Z

stevennt · 2024-06-21T05:36:18Z

When I add the OpenAi API (not the OpenAI-compatible LLAMA3 API), then it works properly.

So the issue appears when the /notes mode is used together with an OpenAi-compatible API.

sabaimran · 2024-06-21T05:52:57Z

Thanks for reporting @stevennt , I'll see if I can get a repro of the error on my end. Meanwhile, could you share any stack traces from your local server?

stevennt · 2024-06-21T05:58:47Z

Seems that it is something to do with the tokenizer: Configure tokenizer for unsupported model: llama3-70b-8192 in Khoj settings

myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | myaiabnkhoj-server-1 myaiabnkhoj-server-1 myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | [05:56:17.344098] DEBUG uvicorn.error: < TEXT 'How many protocol.py:1172 myaiabnkhoj-server-1 | items are in wikipedia' [31 bytes] myaiabnkhoj-server-1 | [05:56:17.348774] DEBUG uvicorn.error: > TEXT '{"type": protocol.py:1178 myaiabnkhoj-server-1 | "status", "message": myaiabnkhoj-server-1 | "**Understanding ...e": myaiabnkhoj-server-1 | "application/json"}' [125 bytes] myaiabnkhoj-server-1 | [05:56:17.355892] WARNING khoj.processor.conversation.utils: utils.py:217 myaiabnkhoj-server-1 | Fallback to default chat model myaiabnkhoj-server-1 | tokenizer: None. myaiabnkhoj-server-1 | Configure tokenizer for unsupported myaiabnkhoj-server-1 | model: llama3-70b-8192 in Khoj settings myaiabnkhoj-server-1 | to improve context stuffing. myaiabnkhoj-server-1 | [05:56:17.798986] DEBUG khoj.processor.conversation.openai before_sleep.py:65 myaiabnkhoj-server-1 | .utils: Retrying myaiabnkhoj-server-1 | khoj.processor.conversation.openai myaiabnkhoj-server-1 | .utils.completion_with_backoff in myaiabnkhoj-server-1 | 0.41305954873250217 seconds as it myaiabnkhoj-server-1 | raised BadRequestError: Error myaiabnkhoj-server-1 | code: 400 - {'error': {'message': myaiabnkhoj-server-1 | 'response_format does not support
streaming', 'type':
'invalid_request_error'}}.
| [05:56:18.498893] DEBUG khoj.processor.conversation.openai before_sleep.py:65
.utils: Retrying
khoj.processor.conversation.openai
.utils.completion_with_backoff in
0.6010106707150216 seconds as it
raised BadRequestError: Error
code: 400 - {'error': {'message':
'response_format does not support myaiabnkhoj-server-1 | streaming', 'type': myaiabnkhoj-server-1 | 'invalid_request_error'}}. myaiabnkhoj-server-1 | [05:56:19.389462] DEBUG khoj.routers.helpers: Chat actor: helpers.py:175 myaiabnkhoj-server-1 | Infer information sources to refer: myaiabnkhoj-server-1 | 2.038 seconds myaiabnkhoj-server-1 | [05:56:19.395893] ERROR uvicorn.error: Exception in websockets_impl.py:188 myaiabnkhoj-server-1 | ASGI application myaiabnkhoj-server-1 | myaiabnkhoj-server-1 | ╭─ Traceback (most recent ca─╮ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/uvicorn/prot │ myaiabnkhoj-server-1 | │ ocols/websockets/websocket │ myaiabnkhoj-server-1 | │ s_impl.py:184 in run_asgi │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 181 │ │ termination │ myaiabnkhoj-server-1 | │ 182 │ │ """ │ myaiabnkhoj-server-1 | │ 183 │ │ try: │ myaiabnkhoj-server-1 | │ ❱ 184 │ │ │ result = │ myaiabnkhoj-server-1 | │ 185 │ │ except BaseE │ myaiabnkhoj-server-1 | │ 186 │ │ │ self.clo │ myaiabnkhoj-server-1 | │ 187 │ │ │ msg = "E │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/uvicorn/midd │ myaiabnkhoj-server-1 | │ leware/proxy_headers.py:75 │ myaiabnkhoj-server-1 | │ in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 72 │ │ │ │ │ p │ myaiabnkhoj-server-1 | │ 73 │ │ │ │ │ s │ myaiabnkhoj-server-1 | │ 74 │ │ │ myaiabnkhoj-server-1 | │ ❱ 75 │ │ return await │ myaiabnkhoj-server-1 | │ 76 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/fastapi/appl │ myaiabnkhoj-server-1 | │ ications.py:1054 in │ myaiabnkhoj-server-1 | │ __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 1051 │ async def __cal │ myaiabnkhoj-server-1 | │ 1052 │ │ if self.roo │ myaiabnkhoj-server-1 | │ 1053 │ │ │ scope[" │ myaiabnkhoj-server-1 | │ ❱ 1054 │ │ await super │ myaiabnkhoj-server-1 | │ 1055 │ │ myaiabnkhoj-server-1 | │ 1056 │ def add_api_rou │ myaiabnkhoj-server-1 | │ 1057 │ │ self, │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ap │ myaiabnkhoj-server-1 | │ plications.py:123 in │ myaiabnkhoj-server-1 | │ __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 120 │ │ scope["app"] │ myaiabnkhoj-server-1 | │ 121 │ │ if self.midd │ myaiabnkhoj-server-1 | │ 122 │ │ │ self.mid │ myaiabnkhoj-server-1 | │ ❱ 123 │ │ await self.m │ myaiabnkhoj-server-1 | │ 124 │ │ myaiabnkhoj-server-1 | │ 125 │ def on_event(sel │ myaiabnkhoj-server-1 | │ 126 │ │ return self. │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │ myaiabnkhoj-server-1 | │ ddleware/errors.py:151 in │ myaiabnkhoj-server-1 | │ __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 148 │ │ myaiabnkhoj-server-1 | │ 149 │ async def __call │ myaiabnkhoj-server-1 | │ 150 │ │ if scope["ty │ myaiabnkhoj-server-1 | │ ❱ 151 │ │ │ await se │ myaiabnkhoj-server-1 | │ 152 │ │ │ return │ myaiabnkhoj-server-1 | │ 153 │ │ │ myaiabnkhoj-server-1 | │ 154 │ │ response_sta │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │ myaiabnkhoj-server-1 | │ ddleware/sessions.py:85 in │ myaiabnkhoj-server-1 | │ __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 82 │ │ │ │ │ h │ myaiabnkhoj-server-1 | │ 83 │ │ │ await sen │ myaiabnkhoj-server-1 | │ 84 │ │ │ myaiabnkhoj-server-1 | │ ❱ 85 │ │ await self.ap │ myaiabnkhoj-server-1 | │ 86 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │ myaiabnkhoj-server-1 | │ ddleware/authentication.py │ myaiabnkhoj-server-1 | │ :49 in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 46 │ │ if auth_resul │ myaiabnkhoj-server-1 | │ 47 │ │ │ auth_resu │ myaiabnkhoj-server-1 | │ 48 │ │ scope["auth"] │ myaiabnkhoj-server-1 | │ ❱ 49 │ │ await self.ap │ myaiabnkhoj-server-1 | │ 50 │ │ myaiabnkhoj-server-1 | │ 51 │ @staticmethod │ myaiabnkhoj-server-1 | │ 52 │ def default_on_er │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │ myaiabnkhoj-server-1 | │ ddleware/cors.py:77 in │ myaiabnkhoj-server-1 | │ __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 74 │ │ myaiabnkhoj-server-1 | │ 75 │ async def __call │ myaiabnkhoj-server-1 | │ 76 │ │ if scope["ty │ myaiabnkhoj-server-1 | │ ❱ 77 │ │ │ await se │ myaiabnkhoj-server-1 | │ 78 │ │ │ return │ myaiabnkhoj-server-1 | │ 79 │ │ │ myaiabnkhoj-server-1 | │ 80 │ │ method = sco │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │ myaiabnkhoj-server-1 | │ ddleware/exceptions.py:65 │ myaiabnkhoj-server-1 | │ in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 62 │ │ else: │ myaiabnkhoj-server-1 | │ 63 │ │ │ conn = We │ myaiabnkhoj-server-1 | │ 64 │ │ │ myaiabnkhoj-server-1 | │ ❱ 65 │ │ await wrap_ap │ myaiabnkhoj-server-1 | │ 66 │ │ myaiabnkhoj-server-1 | │ 67 │ def http_exceptio │ myaiabnkhoj-server-1 | │ 68 │ │ assert isinst │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │ myaiabnkhoj-server-1 | │ xception_handler.py:64 in │ myaiabnkhoj-server-1 | │ wrapped_app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │ myaiabnkhoj-server-1 | │ 62 │ │ │ │ myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │ myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │ myaiabnkhoj-server-1 | │ 65 │ │ │ │ myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │ myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │ myaiabnkhoj-server-1 | │ xception_handler.py:53 in │ myaiabnkhoj-server-1 | │ wrapped_app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │ myaiabnkhoj-server-1 | │ 51 │ │ │ myaiabnkhoj-server-1 | │ 52 │ │ try: │ myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │ myaiabnkhoj-server-1 | │ 54 │ │ except Except │ myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │ myaiabnkhoj-server-1 | │ 56 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │ myaiabnkhoj-server-1 | │ uting.py:756 in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 753 │ │ """ │ myaiabnkhoj-server-1 | │ 754 │ │ The main ent │ myaiabnkhoj-server-1 | │ 755 │ │ """ │ myaiabnkhoj-server-1 | │ ❱ 756 │ │ await self.m │ myaiabnkhoj-server-1 | │ 757 │ │ myaiabnkhoj-server-1 | │ 758 │ async def app(se │ myaiabnkhoj-server-1 | │ 759 │ │ assert scope │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │ myaiabnkhoj-server-1 | │ uting.py:776 in app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 773 │ │ │ match, c │ myaiabnkhoj-server-1 | │ 774 │ │ │ if match │ myaiabnkhoj-server-1 | │ 775 │ │ │ │ scop │ myaiabnkhoj-server-1 | │ ❱ 776 │ │ │ │ awai │ myaiabnkhoj-server-1 | │ 777 │ │ │ │ retu │ myaiabnkhoj-server-1 | │ 778 │ │ │ elif mat │ myaiabnkhoj-server-1 | │ 779 │ │ │ │ part │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │ myaiabnkhoj-server-1 | │ uting.py:373 in handle │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 370 │ │ return URLPa │ myaiabnkhoj-server-1 | │ 371 │ │ myaiabnkhoj-server-1 | │ 372 │ async def handle │ myaiabnkhoj-server-1 | │ ❱ 373 │ │ await self.a │ myaiabnkhoj-server-1 | │ 374 │ │ myaiabnkhoj-server-1 | │ 375 │ def __eq__(self, │ myaiabnkhoj-server-1 | │ 376 │ │ return ( │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │ myaiabnkhoj-server-1 | │ uting.py:96 in app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 93 │ │ async def ap │ myaiabnkhoj-server-1 | │ 94 │ │ │ await fu │ myaiabnkhoj-server-1 | │ 95 │ │ │ myaiabnkhoj-server-1 | │ ❱ 96 │ │ await wrap_a │ myaiabnkhoj-server-1 | │ 97 │ │ myaiabnkhoj-server-1 | │ 98 │ return app │ myaiabnkhoj-server-1 | │ 99 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │ myaiabnkhoj-server-1 | │ xception_handler.py:64 in │ myaiabnkhoj-server-1 | │ wrapped_app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │ myaiabnkhoj-server-1 | │ 62 │ │ │ │ myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │ myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │ myaiabnkhoj-server-1 | │ 65 │ │ │ │ myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │ myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │ myaiabnkhoj-server-1 | │ xception_handler.py:53 in │ myaiabnkhoj-server-1 | │ wrapped_app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │ myaiabnkhoj-server-1 | │ 51 │ │ │ myaiabnkhoj-server-1 | │ 52 │ │ try: │ myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │ myaiabnkhoj-server-1 | │ 54 │ │ except Except │ myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │ myaiabnkhoj-server-1 | │ 56 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │ myaiabnkhoj-server-1 | │ uting.py:94 in app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 91 │ │ session = We │ myaiabnkhoj-server-1 | │ 92 │ │ │ myaiabnkhoj-server-1 | │ 93 │ │ async def ap │ myaiabnkhoj-server-1 | │ ❱ 94 │ │ │ await fu │ myaiabnkhoj-server-1 | │ 95 │ │ │ myaiabnkhoj-server-1 | │ 96 │ │ await wrap_a │ myaiabnkhoj-server-1 | │ 97 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/fastapi/rout │ myaiabnkhoj-server-1 | │ ing.py:348 in app │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 345 │ │ │ if erro │ myaiabnkhoj-server-1 | │ 346 │ │ │ │ rai │ myaiabnkhoj-server-1 | │ 347 │ │ │ assert │ myaiabnkhoj-server-1 | │ ❱ 348 │ │ │ await d │ myaiabnkhoj-server-1 | │ 349 │ │ myaiabnkhoj-server-1 | │ 350 │ return app │ myaiabnkhoj-server-1 | │ 351 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /app/src/khoj/routers/api_ │ myaiabnkhoj-server-1 | │ chat.py:578 in │ myaiabnkhoj-server-1 | │ websocket_endpoint │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 575 │ │ is_automate │ myaiabnkhoj-server-1 | │ 576 │ │ │ myaiabnkhoj-server-1 | │ 577 │ │ if conversa │ myaiabnkhoj-server-1 | │ ❱ 578 │ │ │ convers │ myaiabnkhoj-server-1 | │ is_automated_task) │ myaiabnkhoj-server-1 | │ 579 │ │ │ convers │ myaiabnkhoj-server-1 | │ conversation_comman │ myaiabnkhoj-server-1 | │ 580 │ │ │ await s │ myaiabnkhoj-server-1 | │ {conversation_comma │ myaiabnkhoj-server-1 | │ 581 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │ myaiabnkhoj-server-1 | │ ers.py:250 in │ myaiabnkhoj-server-1 | │ aget_relevant_information_ │ myaiabnkhoj-server-1 | │ sources │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 247 │ ) │ myaiabnkhoj-server-1 | │ 248 │ │ myaiabnkhoj-server-1 | │ 249 │ with timer("Cha │ myaiabnkhoj-server-1 | │ ❱ 250 │ │ response = │ myaiabnkhoj-server-1 | │ response_type="json │ myaiabnkhoj-server-1 | │ 251 │ │ myaiabnkhoj-server-1 | │ 252 │ try: │ myaiabnkhoj-server-1 | │ 253 │ │ response = │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │ myaiabnkhoj-server-1 | │ ers.py:547 in │ myaiabnkhoj-server-1 | │ send_message_to_model_wrap │ myaiabnkhoj-server-1 | │ per │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 544 │ │ │ tokeniz │ myaiabnkhoj-server-1 | │ 545 │ │ ) │ myaiabnkhoj-server-1 | │ 546 │ │ │ myaiabnkhoj-server-1 | │ ❱ 547 │ │ openai_resp │ myaiabnkhoj-server-1 | │ 548 │ │ │ message │ myaiabnkhoj-server-1 | │ 549 │ │ │ api_key │ myaiabnkhoj-server-1 | │ 550 │ │ │ model=c │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │ myaiabnkhoj-server-1 | │ nversation/openai/gpt.py:9 │ myaiabnkhoj-server-1 | │ 8 in send_message_to_model │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 95 │ """ │ myaiabnkhoj-server-1 | │ 96 │ │ myaiabnkhoj-server-1 | │ 97 │ # Get Response f │ myaiabnkhoj-server-1 | │ ❱ 98 │ return completio │ myaiabnkhoj-server-1 | │ 99 │ │ messages=mes │ myaiabnkhoj-server-1 | │ 100 │ │ model=model, │ myaiabnkhoj-server-1 | │ 101 │ │ openai_api_k │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:330 in wrapped_f │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 327 │ │ │ f, funct │ myaiabnkhoj-server-1 | │ 328 │ │ ) │ myaiabnkhoj-server-1 | │ 329 │ │ def wrapped_ │ myaiabnkhoj-server-1 | │ ❱ 330 │ │ │ return s │ myaiabnkhoj-server-1 | │ 331 │ │ │ myaiabnkhoj-server-1 | │ 332 │ │ def retry_wi │ myaiabnkhoj-server-1 | │ 333 │ │ │ return s │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:467 in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 464 │ │ │ myaiabnkhoj-server-1 | │ 465 │ │ retry_state │ myaiabnkhoj-server-1 | │ 466 │ │ while True: │ myaiabnkhoj-server-1 | │ ❱ 467 │ │ │ do = sel │ myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │ myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │ myaiabnkhoj-server-1 | │ 470 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:368 in iter │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 365 │ │ self._begin_ │ myaiabnkhoj-server-1 | │ 366 │ │ result = Non │ myaiabnkhoj-server-1 | │ 367 │ │ for action i │ myaiabnkhoj-server-1 | │ ❱ 368 │ │ │ result = │ myaiabnkhoj-server-1 | │ 369 │ │ return resul │ myaiabnkhoj-server-1 | │ 370 │ │ myaiabnkhoj-server-1 | │ 371 │ def _begin_iter( │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:410 in exc_check │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 407 │ │ │ │ fut │ myaiabnkhoj-server-1 | │ 408 │ │ │ │ retr │ myaiabnkhoj-server-1 | │ 409 │ │ │ │ if s │ myaiabnkhoj-server-1 | │ ❱ 410 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 411 │ │ │ │ rais │ myaiabnkhoj-server-1 | │ 412 │ │ │ │ myaiabnkhoj-server-1 | │ 413 │ │ │ self._ad │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:183 in reraise │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 180 │ │ myaiabnkhoj-server-1 | │ 181 │ def reraise(self │ myaiabnkhoj-server-1 | │ 182 │ │ if self.last │ myaiabnkhoj-server-1 | │ ❱ 183 │ │ │ raise se │ myaiabnkhoj-server-1 | │ 184 │ │ raise self │ myaiabnkhoj-server-1 | │ 185 │ │ myaiabnkhoj-server-1 | │ 186 │ def __str__(self │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │ myaiabnkhoj-server-1 | │ rent/futures/_base.py:451 │ myaiabnkhoj-server-1 | │ in result │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 448 │ │ │ │ if s │ myaiabnkhoj-server-1 | │ 449 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 450 │ │ │ │ elif │ myaiabnkhoj-server-1 | │ ❱ 451 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 452 │ │ │ │ │ myaiabnkhoj-server-1 | │ 453 │ │ │ │ self │ myaiabnkhoj-server-1 | │ 454 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │ myaiabnkhoj-server-1 | │ rent/futures/_base.py:403 │ myaiabnkhoj-server-1 | │ in __get_result │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 400 │ def __get_result │ myaiabnkhoj-server-1 | │ 401 │ │ if self._exc │ myaiabnkhoj-server-1 | │ 402 │ │ │ try: │ myaiabnkhoj-server-1 | │ ❱ 403 │ │ │ │ rais │ myaiabnkhoj-server-1 | │ 404 │ │ │ finally: │ myaiabnkhoj-server-1 | │ 405 │ │ │ │ # Br │ myaiabnkhoj-server-1 | │ 406 │ │ │ │ self │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/tenacity/__i │ myaiabnkhoj-server-1 | │ nit__.py:470 in __call__ │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 467 │ │ │ do = sel │ myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │ myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │ myaiabnkhoj-server-1 | │ ❱ 470 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 471 │ │ │ │ exce │ myaiabnkhoj-server-1 | │ 472 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 473 │ │ │ │ else │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │ myaiabnkhoj-server-1 | │ nversation/openai/utils.py │ myaiabnkhoj-server-1 | │ :50 in │ myaiabnkhoj-server-1 | │ completion_with_backoff │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 47 │ │ myaiabnkhoj-server-1 | │ 48 │ formatted_messag │ myaiabnkhoj-server-1 | │ in messages] │ myaiabnkhoj-server-1 | │ 49 │ │ myaiabnkhoj-server-1 | │ ❱ 50 │ chat = client.ch │ myaiabnkhoj-server-1 | │ 51 │ │ stream=True, │ myaiabnkhoj-server-1 | │ 52 │ │ messages=for │ myaiabnkhoj-server-1 | │ 53 │ │ model=model, │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/openai/_util │ myaiabnkhoj-server-1 | │ s/_utils.py:277 in wrapper │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 274 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 275 │ │ │ │ │ │ myaiabnkhoj-server-1 | │ 276 │ │ │ │ rais │ myaiabnkhoj-server-1 | │ ❱ 277 │ │ │ return f │ myaiabnkhoj-server-1 | │ 278 │ │ │ myaiabnkhoj-server-1 | │ 279 │ │ return wrapp │ myaiabnkhoj-server-1 | │ 280 │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/openai/resou │ myaiabnkhoj-server-1 | │ rces/chat/completions.py:6 │ myaiabnkhoj-server-1 | │ 40 in create │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 637 │ │ extra_body: │ myaiabnkhoj-server-1 | │ 638 │ │ timeout: fl │ myaiabnkhoj-server-1 | │ 639 │ ) -> ChatComple │ myaiabnkhoj-server-1 | │ ❱ 640 │ │ return self │ myaiabnkhoj-server-1 | │ 641 │ │ │ "/chat/ │ myaiabnkhoj-server-1 | │ 642 │ │ │ body=ma │ myaiabnkhoj-server-1 | │ 643 │ │ │ │ { │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/openai/_base │ myaiabnkhoj-server-1 | │ _client.py:1250 in post │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 1247 │ │ opts = Fina │ myaiabnkhoj-server-1 | │ 1248 │ │ │ method= │ myaiabnkhoj-server-1 | │ **options │ myaiabnkhoj-server-1 | │ 1249 │ │ ) │ myaiabnkhoj-server-1 | │ ❱ 1250 │ │ return cast │ myaiabnkhoj-server-1 | │ stream_cls=stream_c │ myaiabnkhoj-server-1 | │ 1251 │ │ myaiabnkhoj-server-1 | │ 1252 │ def patch( │ myaiabnkhoj-server-1 | │ 1253 │ │ self, │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/openai/_base │ myaiabnkhoj-server-1 | │ _client.py:931 in request │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 928 │ │ stream: boo │ myaiabnkhoj-server-1 | │ 929 │ │ stream_cls: │ myaiabnkhoj-server-1 | │ 930 │ ) -> ResponseT │ myaiabnkhoj-server-1 | │ ❱ 931 │ │ return self │ myaiabnkhoj-server-1 | │ 932 │ │ │ cast_to │ myaiabnkhoj-server-1 | │ 933 │ │ │ options │ myaiabnkhoj-server-1 | │ 934 │ │ │ stream= │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │ myaiabnkhoj-server-1 | │ dist-packages/openai/_base │ myaiabnkhoj-server-1 | │ _client.py:1030 in │ myaiabnkhoj-server-1 | │ _request │ myaiabnkhoj-server-1 | │ │ myaiabnkhoj-server-1 | │ 1027 │ │ │ │ err │ myaiabnkhoj-server-1 | │ 1028 │ │ │ │ myaiabnkhoj-server-1 | │ 1029 │ │ │ log.deb │ myaiabnkhoj-server-1 | │ ❱ 1030 │ │ │ raise s │ myaiabnkhoj-server-1 | │ 1031 │ │ │ myaiabnkhoj-server-1 | │ 1032 │ │ return self │ myaiabnkhoj-server-1 | │ 1033 │ │ │ cast_to │ myaiabnkhoj-server-1 | ╰────────────────────────────╯ myaiabnkhoj-server-1 | BadRequestError: Error code: myaiabnkhoj-server-1 | 400 - {'error': {'message': myaiabnkhoj-server-1 | 'response_format does not
support streaming', 'type':
'invalid_request_error'}}
| [05:56:20.452685] DEBUG uvicorn.error: = connection is protocol.py:1227
CLOSING
| [05:56:20.453936] DEBUG uvicorn.error: > CLOSE 1000 (OK) [2 protocol.py:1178
bytes]
| [05:56:20.455200] DEBUG uvicorn.error: = connection is protocol.py:1497
CLOSED
| [05:56:20.457202] DEBUG uvicorn.error: ! failing connection protocol.py:1412
with code 1006
| [05:56:20.459085] ERROR uvicorn.error: closing handshake failed server.py:248
╭─ Traceback (most recent call last) ─╮
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/server.py:24 │
│ 4 in handler │
│ │
│ 241 │ │ │ │ raise │
│ 242 │ │ │ │
│ 243 │ │ │ try: │
│ ❱ 244 │ │ │ │ await self.c │
│ 245 │ │ │ except Connectio │
│ 246 │ │ │ │ raise │
│ 247 │ │ │ except Exception │
│ │
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/protocol.py: │
│ 770 in close │
│ │
│ 767 │ │ """ │
│ 768 │ │ try: │
│ 769 │ │ │ async with async │
│ ❱ 770 │ │ │ │ await self.w │
│ 771 │ │ except asyncio.Timeo │
│ 772 │ │ │ # If the close f │
│ 773 │ │ │ # are full, the │
│ │
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/protocol.py: │
│ 1236 in write_close_frame │
│ │
│ 1233 │ │ │ │ data = close │
│ 1234 │ │ │ │
│ 1235 │ │ │ # 7.1.2. Start t │
│ ❱ 1236 │ │ │ await self.write │
│ 1237 │ │
│ 1238 │ async def keepalive_ping │
│ 1239 │ │ """ │
│ │
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/protocol.py: │
│ 1209 in write_frame │
│ │
│ 1206 │ │ │ │ f"Cannot wri │
│ 1207 │ │ │ ) │
│ 1208 │ │ self.write_frame_syn │
│ ❱ 1209 │ │ await self.drain() │
│ 1210 │ │
│ 1211 │ async def write_close_fr │
│ 1212 │ │ self, close: Close, │
│ │
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/protocol.py: │
│ 1198 in drain │
│ │
│ 1195 │ │ │ self.fail_connec │
│ 1196 │ │ │ # Wait until the │
│ 1197 │ │ │ # with the corre │
│ ❱ 1198 │ │ │ await self.ensur │
│ 1199 │ │
│ 1200 │ async def write_frame( │
│ 1201 │ │ self, fin: bool, opc │
│ │
│ /usr/local/lib/python3.10/dist-pack │
│ ages/websockets/legacy/protocol.py: │
│ 939 in ensure_open │
│ │
│ 936 │ │ │ │ return │
│ 937 │ │ │
│ 938 │ │ if self.state is Sta │
│ ❱ 939 │ │ │ raise self.conne │
│ 940 │ │ │
│ 941 │ │ if self.state is Sta │
│ 942 │ │ │ # If we started │
╰─────────────────────────────────────╯
ConnectionClosedError: sent 1000 (OK);
no close frame received
| [05:56:20.763183] INFO uvicorn.error: connection closed server.py:264
| [05:56:20.764402] DEBUG uvicorn.error: x half-closing TCP protocol.py:1319
connection

`

sabaimran · 2024-06-21T06:04:07Z

Awesome, this is really helpful, thanks for including it. Will see if I can get it resolved today.

sabaimran · 2024-06-21T06:53:52Z

Thanks for the helpful report @stevennt ! I've pushed a fix. It'll be included in the next release (~1 week). If you want the fix sooner and you're using the docker image, you can build from the pre tag when the image finishes building (~20 minutes from now).

stevennt · 2024-06-21T07:23:45Z

I just pulled the master branch which has the below commit fix. The issue still persists. @sabaimran

d3c0111

sabaimran · 2024-06-21T07:26:13Z

How are you running the project? Straight from source? Or from the docker image?

You're seeing the exact same stack trace?

stevennt · 2024-06-21T08:05:55Z

What I did was to pull the source, run docker-compose build, docker-compose down -v, and docker-compose up.
Then I setup the configs at server admin.
First:

then:

Maybe something needs to be set at this tokenizer field?

Below is the stack trace:

docker-compose up [+] Running 2/0 ⠿ Container myaiabnkhoj-database-1 Created 0.0s ⠿ Container myaiabnkhoj-server-1 Created 0.0s Attaching to myaiabnkhoj-database-1, myaiabnkhoj-server-1 myaiabnkhoj-database-1 | myaiabnkhoj-database-1 | PostgreSQL Database directory appears to contain a database; Skipping initialization myaiabnkhoj-database-1 | myaiabnkhoj-database-1 | 2024-06-21 07:58:14.007 UTC [1] LOG: starting PostgreSQL 15.4 (Debian 15.4-2.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit myaiabnkhoj-database-1 | 2024-06-21 07:58:14.007 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 myaiabnkhoj-database-1 | 2024-06-21 07:58:14.007 UTC [1] LOG: listening on IPv6 address "::", port 5432 myaiabnkhoj-database-1 | 2024-06-21 07:58:14.173 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" myaiabnkhoj-database-1 | 2024-06-21 07:58:14.367 UTC [28] LOG: database system was shut down at 2024-06-21 07:58:05 UTC myaiabnkhoj-database-1 | 2024-06-21 07:58:14.487 UTC [1] LOG: database system is ready to accept connections myaiabnkhoj-server-1 | [07:58:49.489205] WARNING khoj.routers.email: RESEND_API_KEY not email.py:26 myaiabnkhoj-server-1 | set - email sending disabled myaiabnkhoj-server-1 | [07:58:49.529753] INFO khoj: 🚒 Initializing ABNCopilot v0.0.0 main.py:124 myaiabnkhoj-server-1 | [07:58:49.531214] INFO khoj: 📦 Initializing DB: main.py:125 myaiabnkhoj-server-1 | Operations to perform: myaiabnkhoj-server-1 | Apply all migrations: admin, auth, myaiabnkhoj-server-1 | contenttypes, database, myaiabnkhoj-server-1 | django_apscheduler, sessions myaiabnkhoj-server-1 | Running migrations: myaiabnkhoj-server-1 | No migrations to apply. myaiabnkhoj-server-1 | [07:58:49.533447] DEBUG khoj: 🌍 Initializing Web Client: main.py:126 myaiabnkhoj-server-1 | 0 static files copied to myaiabnkhoj-server-1 | '/app/src/khoj/static', 201 unmodified. myaiabnkhoj-server-1 | [07:58:49.539414] INFO khoj: 🌘 Starting Khoj main.py:138 myaiabnkhoj-server-1 | [07:58:49.829809] INFO khoj.configure: 🚨 Khoj is not configure.py:198 myaiabnkhoj-server-1 | configured. myaiabnkhoj-server-1 | Initializing it with a default myaiabnkhoj-server-1 | config. myaiabnkhoj-server-1 | /usr/local/lib/python3.10/dist-packages/huggingface_hub/file_download.py:1132: FutureWarning:resume_downloadis deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, useforce_download=True`.
myaiabnkhoj-server-1 | warnings.warn(

myaiabnkhoj-server-1 | [08:00:03.609363] INFO khoj.database.adapters: No default init.py:558
myaiabnkhoj-server-1 | conversation config found, skipping
myaiabnkhoj-server-1 | default agent creation
myaiabnkhoj-server-1 | [08:00:03.610718] INFO khoj.configure: 📡 Telemetry enabled configure.py:240
myaiabnkhoj-server-1 | [08:00:03.611839] INFO khoj: 🌖 ABN Copilot is ready to use main.py:190
myaiabnkhoj-server-1 | [08:00:03.624946] INFO uvicorn.error: Started server process server.py:75
myaiabnkhoj-server-1 | [1]
myaiabnkhoj-server-1 | [08:00:03.626195] INFO uvicorn.error: Waiting for application on.py:45
myaiabnkhoj-server-1 | startup.
myaiabnkhoj-server-1 | [08:00:03.627583] INFO uvicorn.error: Application startup complete. on.py:59
myaiabnkhoj-server-1 | [08:00:03.629082] INFO uvicorn.error: Uvicorn running on server.py:206
myaiabnkhoj-server-1 | http://0.0.0.0:42110 (Press CTRL+C to
myaiabnkhoj-server-1 | quit)

myaiabnkhoj-server-1 | [08:04:47.153646] DEBUG uvicorn.error: < TEXT 'What are the protocol.py:1172
myaiabnkhoj-server-1 | number of articles in Wikipedia?'
myaiabnkhoj-server-1 | [45 bytes]
myaiabnkhoj-server-1 | [08:04:47.160158] DEBUG uvicorn.error: > TEXT '{"type": protocol.py:1178
myaiabnkhoj-server-1 | "status", "message":
myaiabnkhoj-server-1 | "**Understanding ...e":
myaiabnkhoj-server-1 | "application/json"}' [139 bytes]
myaiabnkhoj-server-1 | [08:04:49.544382] INFO khoj.configure: 📡 Uploading configure.py:353
myaiabnkhoj-server-1 | telemetry to
myaiabnkhoj-server-1 | https://khoj.beta.haletic.com/v1/tel
myaiabnkhoj-server-1 | emetry...
myaiabnkhoj-server-1 | [08:04:49.550244] DEBUG khoj.configure: Telemetry state: configure.py:354
myaiabnkhoj-server-1 | [{'telemetry_type': 'api',
myaiabnkhoj-server-1 | 'server_version': '0.0.0', 'os':
myaiabnkhoj-server-1 | 'Linux', 'timestamp': '2024-06-21
myaiabnkhoj-server-1 | 08:03:59', 'client_host':
myaiabnkhoj-server-1 | '172.26.0.1', 'user_agent':
myaiabnkhoj-server-1 | 'unknown', 'referer': 'unknown',
myaiabnkhoj-server-1 | 'host': 'unknown', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'api':
myaiabnkhoj-server-1 | 'get_all_filenames'},
myaiabnkhoj-server-1 | {'telemetry_type': 'api',
myaiabnkhoj-server-1 | 'server_version': '0.0.0', 'os':
myaiabnkhoj-server-1 | 'Linux', 'timestamp': '2024-06-21
myaiabnkhoj-server-1 | 08:03:59', 'client_host':
myaiabnkhoj-server-1 | '172.26.0.1', 'user_agent':
myaiabnkhoj-server-1 | 'Mozilla/5.0 (X11; Linux x86_64)
myaiabnkhoj-server-1 | AppleWebKit/537.36 (KHTML, like
myaiabnkhoj-server-1 | Gecko) Chrome/123.0.0.0
myaiabnkhoj-server-1 | Safari/537.36', 'referer':
myaiabnkhoj-server-1 | 'http://localhost:42110/', 'host':
myaiabnkhoj-server-1 | 'localhost:42110', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'api': 'chat_options'},
myaiabnkhoj-server-1 | {'telemetry_type': 'api',
myaiabnkhoj-server-1 | 'server_version': '0.0.0', 'os':
myaiabnkhoj-server-1 | 'Linux', 'timestamp': '2024-06-21
myaiabnkhoj-server-1 | 08:03:59', 'client_host':
myaiabnkhoj-server-1 | '172.26.0.1', 'user_agent':
myaiabnkhoj-server-1 | 'Mozilla/5.0 (X11; Linux x86_64)
myaiabnkhoj-server-1 | AppleWebKit/537.36 (KHTML, like
myaiabnkhoj-server-1 | Gecko) Chrome/123.0.0.0
myaiabnkhoj-server-1 | Safari/537.36', 'referer':
myaiabnkhoj-server-1 | 'http://localhost:42110/', 'host':
myaiabnkhoj-server-1 | 'localhost:42110', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'api': 'chat_sessions'},
myaiabnkhoj-server-1 | {'telemetry_type': 'api',
myaiabnkhoj-server-1 | 'server_version': '0.0.0', 'os':
myaiabnkhoj-server-1 | 'Linux', 'timestamp': '2024-06-21
myaiabnkhoj-server-1 | 08:03:59', 'client_host':
myaiabnkhoj-server-1 | '172.26.0.1', 'user_agent':
myaiabnkhoj-server-1 | 'Mozilla/5.0 (X11; Linux x86_64)
myaiabnkhoj-server-1 | AppleWebKit/537.36 (KHTML, like
myaiabnkhoj-server-1 | Gecko) Chrome/123.0.0.0
myaiabnkhoj-server-1 | Safari/537.36', 'referer':
myaiabnkhoj-server-1 | 'http://localhost:42110/', 'host':
myaiabnkhoj-server-1 | 'localhost:42110', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'api': 'chat_history',
myaiabnkhoj-server-1 | 'client': 'web'}, {'telemetry_type':
myaiabnkhoj-server-1 | 'api', 'server_version': '0.0.0',
myaiabnkhoj-server-1 | 'os': 'Linux', 'timestamp':
myaiabnkhoj-server-1 | '2024-06-21 08:04:15',
myaiabnkhoj-server-1 | 'client_host': '172.26.0.1',
myaiabnkhoj-server-1 | 'user_agent': 'Mozilla/5.0 (X11;
myaiabnkhoj-server-1 | Linux x86_64) AppleWebKit/537.36
myaiabnkhoj-server-1 | (KHTML, like Gecko) Chrome/123.0.0.0
myaiabnkhoj-server-1 | Safari/537.36', 'referer':
myaiabnkhoj-server-1 | 'http://localhost:42110/', 'host':
myaiabnkhoj-server-1 | 'localhost:42110', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'num_org': 0,
myaiabnkhoj-server-1 | 'num_markdown': 0, 'num_pdf': 1,
myaiabnkhoj-server-1 | 'num_plaintext': 0, 'num_docx': 0,
myaiabnkhoj-server-1 | 'api': 'index/update', 'client':
myaiabnkhoj-server-1 | 'web'}, {'telemetry_type': 'api',
myaiabnkhoj-server-1 | 'server_version': '0.0.0', 'os':
myaiabnkhoj-server-1 | 'Linux', 'timestamp': '2024-06-21
myaiabnkhoj-server-1 | 08:04:15', 'client_host':
myaiabnkhoj-server-1 | '172.26.0.1', 'user_agent':
myaiabnkhoj-server-1 | 'unknown', 'referer': 'unknown',
myaiabnkhoj-server-1 | 'host': 'unknown', 'server_id':
myaiabnkhoj-server-1 | '26485c08-2711-4a10-9a5f-b96e41e1e2a
myaiabnkhoj-server-1 | d', 'subscription_type': 'standard',
myaiabnkhoj-server-1 | 'is_recurring': False, 'client_id':
myaiabnkhoj-server-1 | 'default', 'api':
myaiabnkhoj-server-1 | 'get_all_filenames'}]
myaiabnkhoj-server-1 | You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama_fast.LlamaTokenizerFast'>. This is expected, and simply means that the legacy (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set legacy=False. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in huggingface/transformers#24565
myaiabnkhoj-server-1 | [08:04:52.298187] WARNING khoj.processor.conversation.utils: utils.py:217
myaiabnkhoj-server-1 | Fallback to default chat model
myaiabnkhoj-server-1 | tokenizer: None.
myaiabnkhoj-server-1 | Configure tokenizer for unsupported
myaiabnkhoj-server-1 | model: llama3-70b-8192 in Khoj settings
myaiabnkhoj-server-1 | to improve context stuffing.
myaiabnkhoj-server-1 | [08:04:52.682359] DEBUG khoj.processor.conversation.openai before_sleep.py:65
myaiabnkhoj-server-1 | .utils: Retrying
myaiabnkhoj-server-1 | khoj.processor.conversation.openai
myaiabnkhoj-server-1 | .utils.completion_with_backoff in
myaiabnkhoj-server-1 | 0.6461066858467513 seconds as it
myaiabnkhoj-server-1 | raised BadRequestError: Error
myaiabnkhoj-server-1 | code: 400 - {'error': {'message':
myaiabnkhoj-server-1 | 'response_format does not support myaiabnkhoj-server-1 | streaming', 'type': myaiabnkhoj-server-1 | 'invalid_request_error'}}. myaiabnkhoj-server-1 | [08:04:53.536637] DEBUG khoj.processor.conversation.openai before_sleep.py:65 myaiabnkhoj-server-1 | .utils: Retrying myaiabnkhoj-server-1 | khoj.processor.conversation.openai myaiabnkhoj-server-1 | .utils.completion_with_backoff in myaiabnkhoj-server-1 | 1.0890849572597519 seconds as it myaiabnkhoj-server-1 | raised BadRequestError: Error myaiabnkhoj-server-1 | code: 400 - {'error': {'message': myaiabnkhoj-server-1 | 'response_format does not support
myaiabnkhoj-server-1 | streaming', 'type':
myaiabnkhoj-server-1 | 'invalid_request_error'}}.
myaiabnkhoj-server-1 | [08:04:54.883765] DEBUG khoj.routers.helpers: Chat actor: helpers.py:175
myaiabnkhoj-server-1 | Infer information sources to refer:
myaiabnkhoj-server-1 | 7.722 seconds
myaiabnkhoj-server-1 | [08:04:54.888881] ERROR uvicorn.error: Exception in websockets_impl.py:188
myaiabnkhoj-server-1 | ASGI application
myaiabnkhoj-server-1 |
myaiabnkhoj-server-1 | ╭─ Traceback (most recent ca─╮
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/uvicorn/prot │
myaiabnkhoj-server-1 | │ ocols/websockets/websocket │
myaiabnkhoj-server-1 | │ s_impl.py:184 in run_asgi │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 181 │ │ termination │
myaiabnkhoj-server-1 | │ 182 │ │ """ │
myaiabnkhoj-server-1 | │ 183 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 184 │ │ │ result = │
myaiabnkhoj-server-1 | │ 185 │ │ except BaseE │
myaiabnkhoj-server-1 | │ 186 │ │ │ self.clo │
myaiabnkhoj-server-1 | │ 187 │ │ │ msg = "E │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/uvicorn/midd │
myaiabnkhoj-server-1 | │ leware/proxy_headers.py:75 │
myaiabnkhoj-server-1 | │ in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 72 │ │ │ │ │ p │
myaiabnkhoj-server-1 | │ 73 │ │ │ │ │ s │
myaiabnkhoj-server-1 | │ 74 │ │ │
myaiabnkhoj-server-1 | │ ❱ 75 │ │ return await │
myaiabnkhoj-server-1 | │ 76 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/fastapi/appl │
myaiabnkhoj-server-1 | │ ications.py:1054 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1051 │ async def __cal │
myaiabnkhoj-server-1 | │ 1052 │ │ if self.roo │
myaiabnkhoj-server-1 | │ 1053 │ │ │ scope[" │
myaiabnkhoj-server-1 | │ ❱ 1054 │ │ await super │
myaiabnkhoj-server-1 | │ 1055 │ │
myaiabnkhoj-server-1 | │ 1056 │ def add_api_rou │
myaiabnkhoj-server-1 | │ 1057 │ │ self, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ap │
myaiabnkhoj-server-1 | │ plications.py:123 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 120 │ │ scope["app"] │
myaiabnkhoj-server-1 | │ 121 │ │ if self.midd │
myaiabnkhoj-server-1 | │ 122 │ │ │ self.mid │
myaiabnkhoj-server-1 | │ ❱ 123 │ │ await self.m │
myaiabnkhoj-server-1 | │ 124 │ │
myaiabnkhoj-server-1 | │ 125 │ def on_event(sel │
myaiabnkhoj-server-1 | │ 126 │ │ return self. │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/errors.py:151 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 148 │ │
myaiabnkhoj-server-1 | │ 149 │ async def __call │
myaiabnkhoj-server-1 | │ 150 │ │ if scope["ty │
myaiabnkhoj-server-1 | │ ❱ 151 │ │ │ await se │
myaiabnkhoj-server-1 | │ 152 │ │ │ return │
myaiabnkhoj-server-1 | │ 153 │ │ │
myaiabnkhoj-server-1 | │ 154 │ │ response_sta │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/sessions.py:85 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 82 │ │ │ │ │ h │
myaiabnkhoj-server-1 | │ 83 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 84 │ │ │
myaiabnkhoj-server-1 | │ ❱ 85 │ │ await self.ap │
myaiabnkhoj-server-1 | │ 86 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/authentication.py │
myaiabnkhoj-server-1 | │ :49 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 46 │ │ if auth_resul │
myaiabnkhoj-server-1 | │ 47 │ │ │ auth_resu │
myaiabnkhoj-server-1 | │ 48 │ │ scope["auth"] │
myaiabnkhoj-server-1 | │ ❱ 49 │ │ await self.ap │
myaiabnkhoj-server-1 | │ 50 │ │
myaiabnkhoj-server-1 | │ 51 │ @staticmethod │
myaiabnkhoj-server-1 | │ 52 │ def default_on_er │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/cors.py:77 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 74 │ │
myaiabnkhoj-server-1 | │ 75 │ async def __call │
myaiabnkhoj-server-1 | │ 76 │ │ if scope["ty │
myaiabnkhoj-server-1 | │ ❱ 77 │ │ │ await se │
myaiabnkhoj-server-1 | │ 78 │ │ │ return │
myaiabnkhoj-server-1 | │ 79 │ │ │
myaiabnkhoj-server-1 | │ 80 │ │ method = sco │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/exceptions.py:65 │
myaiabnkhoj-server-1 | │ in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 62 │ │ else: │
myaiabnkhoj-server-1 | │ 63 │ │ │ conn = We │
myaiabnkhoj-server-1 | │ 64 │ │ │
myaiabnkhoj-server-1 | │ ❱ 65 │ │ await wrap_ap │
myaiabnkhoj-server-1 | │ 66 │ │
myaiabnkhoj-server-1 | │ 67 │ def http_exceptio │
myaiabnkhoj-server-1 | │ 68 │ │ assert isinst │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │
myaiabnkhoj-server-1 | │ xception_handler.py:64 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │
myaiabnkhoj-server-1 | │ 62 │ │ │ │
myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │
myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 65 │ │ │ │
myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │
myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/e │
myaiabnkhoj-server-1 | │ xception_handler.py:53 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 51 │ │ │
myaiabnkhoj-server-1 | │ 52 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │
myaiabnkhoj-server-1 | │ 54 │ │ except Except │
myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │
myaiabnkhoj-server-1 | │ 56 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:756 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 753 │ │ """ │
myaiabnkhoj-server-1 | │ 754 │ │ The main ent │
myaiabnkhoj-server-1 | │ 755 │ │ """ │
myaiabnkhoj-server-1 | │ ❱ 756 │ │ await self.m │
myaiabnkhoj-server-1 | │ 757 │ │
myaiabnkhoj-server-1 | │ 758 │ async def app(se │
myaiabnkhoj-server-1 | │ 759 │ │ assert scope │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:776 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 773 │ │ │ match, c │
myaiabnkhoj-server-1 | │ 774 │ │ │ if match │
myaiabnkhoj-server-1 | │ 775 │ │ │ │ scop │
myaiabnkhoj-server-1 | │ ❱ 776 │ │ │ │ awai │
myaiabnkhoj-server-1 | │ 777 │ │ │ │ retu │
myaiabnkhoj-server-1 | │ 778 │ │ │ elif mat │
myaiabnkhoj-server-1 | │ 779 │ │ │ │ part │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:373 in handle │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 370 │ │ return URLPa │
myaiabnkhoj-server-1 | │ 371 │ │
myaiabnkhoj-server-1 | │ 372 │ async def handle │
myaiabnkhoj-server-1 | │ ❱ 373 │ │ await self.a │
myaiabnkhoj-server-1 | │ 374 │ │
myaiabnkhoj-server-1 | │ 375 │ def eq(self, │
myaiabnkhoj-server-1 | │ 376 │ │ return ( │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:96 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 93 │ │ async def ap │
myaiabnkhoj-server-1 | │ 94 │ │ │ await fu │
myaiabnkhoj-server-1 | │ 95 │ │ │
myaiabnkhoj-server-1 | │ ❱ 96 │ │ await wrap_a │
myaiabnkhoj-server-1 | │ 97 │ │
myaiabnkhoj-server-1 | │ 98 │ return app │
myaiabnkhoj-server-1 | │ 99 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/e │
myaiabnkhoj-server-1 | │ xception_handler.py:64 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │
myaiabnkhoj-server-1 | │ 62 │ │ │ │
myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │
myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 65 │ │ │ │
myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │
myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/e │
myaiabnkhoj-server-1 | │ xception_handler.py:53 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 51 │ │ │
myaiabnkhoj-server-1 | │ 52 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │
myaiabnkhoj-server-1 | │ 54 │ │ except Except │
myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │
myaiabnkhoj-server-1 | │ 56 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:94 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 91 │ │ session = We │
myaiabnkhoj-server-1 | │ 92 │ │ │
myaiabnkhoj-server-1 | │ 93 │ │ async def ap │
myaiabnkhoj-server-1 | │ ❱ 94 │ │ │ await fu │
myaiabnkhoj-server-1 | │ 95 │ │ │
myaiabnkhoj-server-1 | │ 96 │ │ await wrap_a │
myaiabnkhoj-server-1 | │ 97 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/fastapi/rout │
myaiabnkhoj-server-1 | │ ing.py:348 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 345 │ │ │ if erro │
myaiabnkhoj-server-1 | │ 346 │ │ │ │ rai │
myaiabnkhoj-server-1 | │ 347 │ │ │ assert │
myaiabnkhoj-server-1 | │ ❱ 348 │ │ │ await d │
myaiabnkhoj-server-1 | │ 349 │ │
myaiabnkhoj-server-1 | │ 350 │ return app │
myaiabnkhoj-server-1 | │ 351 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/api │
myaiabnkhoj-server-1 | │ chat.py:593 in │
myaiabnkhoj-server-1 | │ websocket_endpoint │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 590 │ │ is_automate │
myaiabnkhoj-server-1 | │ 591 │ │ │
myaiabnkhoj-server-1 | │ 592 │ │ if conversa │
myaiabnkhoj-server-1 | │ ❱ 593 │ │ │ convers │
myaiabnkhoj-server-1 | │ is_automated_task) │
myaiabnkhoj-server-1 | │ 594 │ │ │ convers │
myaiabnkhoj-server-1 | │ conversation_comman │
myaiabnkhoj-server-1 | │ 595 │ │ │ await s │
myaiabnkhoj-server-1 | │ {conversation_comma │
myaiabnkhoj-server-1 | │ 596 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │
myaiabnkhoj-server-1 | │ ers.py:250 in │
myaiabnkhoj-server-1 | │ aget_relevant_information │
myaiabnkhoj-server-1 | │ sources │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 247 │ ) │
myaiabnkhoj-server-1 | │ 248 │ │
myaiabnkhoj-server-1 | │ 249 │ with timer("Cha │
myaiabnkhoj-server-1 | │ ❱ 250 │ │ response = │
myaiabnkhoj-server-1 | │ response_type="json │
myaiabnkhoj-server-1 | │ 251 │ │
myaiabnkhoj-server-1 | │ 252 │ try: │
myaiabnkhoj-server-1 | │ 253 │ │ response = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │
myaiabnkhoj-server-1 | │ ers.py:547 in │
myaiabnkhoj-server-1 | │ send_message_to_model_wrap │
myaiabnkhoj-server-1 | │ per │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 544 │ │ │ tokeniz │
myaiabnkhoj-server-1 | │ 545 │ │ ) │
myaiabnkhoj-server-1 | │ 546 │ │ │
myaiabnkhoj-server-1 | │ ❱ 547 │ │ openai_resp │
myaiabnkhoj-server-1 | │ 548 │ │ │ message │
myaiabnkhoj-server-1 | │ 549 │ │ │ api_key │
myaiabnkhoj-server-1 | │ 550 │ │ │ model=c │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │
myaiabnkhoj-server-1 | │ nversation/openai/gpt.py:9 │
myaiabnkhoj-server-1 | │ 8 in send_message_to_model │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 95 │ """ │
myaiabnkhoj-server-1 | │ 96 │ │
myaiabnkhoj-server-1 | │ 97 │ # Get Response f │
myaiabnkhoj-server-1 | │ ❱ 98 │ return completio │
myaiabnkhoj-server-1 | │ 99 │ │ messages=mes │
myaiabnkhoj-server-1 | │ 100 │ │ model=model, │
myaiabnkhoj-server-1 | │ 101 │ │ openai_api_k │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:330 in wrapped_f │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 327 │ │ │ f, funct │
myaiabnkhoj-server-1 | │ 328 │ │ ) │
myaiabnkhoj-server-1 | │ 329 │ │ def wrapped │
myaiabnkhoj-server-1 | │ ❱ 330 │ │ │ return s │
myaiabnkhoj-server-1 | │ 331 │ │ │
myaiabnkhoj-server-1 | │ 332 │ │ def retry_wi │
myaiabnkhoj-server-1 | │ 333 │ │ │ return s │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:467 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 464 │ │ │
myaiabnkhoj-server-1 | │ 465 │ │ retry_state │
myaiabnkhoj-server-1 | │ 466 │ │ while True: │
myaiabnkhoj-server-1 | │ ❱ 467 │ │ │ do = sel │
myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │
myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │
myaiabnkhoj-server-1 | │ 470 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:368 in iter │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 365 │ │ self.begin │
myaiabnkhoj-server-1 | │ 366 │ │ result = Non │
myaiabnkhoj-server-1 | │ 367 │ │ for action i │
myaiabnkhoj-server-1 | │ ❱ 368 │ │ │ result = │
myaiabnkhoj-server-1 | │ 369 │ │ return resul │
myaiabnkhoj-server-1 | │ 370 │ │
myaiabnkhoj-server-1 | │ 371 │ def _begin_iter( │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:410 in exc_check │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 407 │ │ │ │ fut │
myaiabnkhoj-server-1 | │ 408 │ │ │ │ retr │
myaiabnkhoj-server-1 | │ 409 │ │ │ │ if s │
myaiabnkhoj-server-1 | │ ❱ 410 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 411 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ 412 │ │ │ │
myaiabnkhoj-server-1 | │ 413 │ │ │ self._ad │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:183 in reraise │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 180 │ │
myaiabnkhoj-server-1 | │ 181 │ def reraise(self │
myaiabnkhoj-server-1 | │ 182 │ │ if self.last │
myaiabnkhoj-server-1 | │ ❱ 183 │ │ │ raise se │
myaiabnkhoj-server-1 | │ 184 │ │ raise self │
myaiabnkhoj-server-1 | │ 185 │ │
myaiabnkhoj-server-1 | │ 186 │ def str(self │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │
myaiabnkhoj-server-1 | │ rent/futures/_base.py:451 │
myaiabnkhoj-server-1 | │ in result │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 448 │ │ │ │ if s │
myaiabnkhoj-server-1 | │ 449 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 450 │ │ │ │ elif │
myaiabnkhoj-server-1 | │ ❱ 451 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 452 │ │ │ │ │
myaiabnkhoj-server-1 | │ 453 │ │ │ │ self │
myaiabnkhoj-server-1 | │ 454 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │
myaiabnkhoj-server-1 | │ rent/futures/_base.py:403 │
myaiabnkhoj-server-1 | │ in __get_result │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 400 │ def __get_result │
myaiabnkhoj-server-1 | │ 401 │ │ if self._exc │
myaiabnkhoj-server-1 | │ 402 │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 403 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ 404 │ │ │ finally: │
myaiabnkhoj-server-1 | │ 405 │ │ │ │ # Br │
myaiabnkhoj-server-1 | │ 406 │ │ │ │ self │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:470 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 467 │ │ │ do = sel │
myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │
myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 470 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 471 │ │ │ │ exce │
myaiabnkhoj-server-1 | │ 472 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 473 │ │ │ │ else │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │
myaiabnkhoj-server-1 | │ nversation/openai/utils.py │
myaiabnkhoj-server-1 | │ :50 in │
myaiabnkhoj-server-1 | │ completion_with_backoff │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 47 │ │
myaiabnkhoj-server-1 | │ 48 │ formatted_messag │
myaiabnkhoj-server-1 | │ in messages] │
myaiabnkhoj-server-1 | │ 49 │ │
myaiabnkhoj-server-1 | │ ❱ 50 │ chat = client.ch │
myaiabnkhoj-server-1 | │ 51 │ │ stream=True, │
myaiabnkhoj-server-1 | │ 52 │ │ messages=for │
myaiabnkhoj-server-1 | │ 53 │ │ model=model, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_util │
myaiabnkhoj-server-1 | │ s/_utils.py:277 in wrapper │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 274 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 275 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 276 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ ❱ 277 │ │ │ return f │
myaiabnkhoj-server-1 | │ 278 │ │ │
myaiabnkhoj-server-1 | │ 279 │ │ return wrapp │
myaiabnkhoj-server-1 | │ 280 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/resou │
myaiabnkhoj-server-1 | │ rces/chat/completions.py:6 │
myaiabnkhoj-server-1 | │ 40 in create │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 637 │ │ extra_body: │
myaiabnkhoj-server-1 | │ 638 │ │ timeout: fl │
myaiabnkhoj-server-1 | │ 639 │ ) -> ChatComple │
myaiabnkhoj-server-1 | │ ❱ 640 │ │ return self │
myaiabnkhoj-server-1 | │ 641 │ │ │ "/chat/ │
myaiabnkhoj-server-1 | │ 642 │ │ │ body=ma │
myaiabnkhoj-server-1 | │ 643 │ │ │ │ { │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:1250 in post │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1247 │ │ opts = Fina │
myaiabnkhoj-server-1 | │ 1248 │ │ │ method= │
myaiabnkhoj-server-1 | │ **options │
myaiabnkhoj-server-1 | │ 1249 │ │ ) │
myaiabnkhoj-server-1 | │ ❱ 1250 │ │ return cast │
myaiabnkhoj-server-1 | │ stream_cls=stream_c │
myaiabnkhoj-server-1 | │ 1251 │ │
myaiabnkhoj-server-1 | │ 1252 │ def patch( │
myaiabnkhoj-server-1 | │ 1253 │ │ self, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:931 in request │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 928 │ │ stream: boo │
myaiabnkhoj-server-1 | │ 929 │ │ stream_cls: │
myaiabnkhoj-server-1 | │ 930 │ ) -> ResponseT │
myaiabnkhoj-server-1 | │ ❱ 931 │ │ return self │
myaiabnkhoj-server-1 | │ 932 │ │ │ cast_to │
myaiabnkhoj-server-1 | │ 933 │ │ │ options │
myaiabnkhoj-server-1 | │ 934 │ │ │ stream= │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:1030 in │
myaiabnkhoj-server-1 | │ _request │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1027 │ │ │ │ err │
myaiabnkhoj-server-1 | │ 1028 │ │ │ │
myaiabnkhoj-server-1 | │ 1029 │ │ │ log.deb │
myaiabnkhoj-server-1 | │ ❱ 1030 │ │ │ raise s │
myaiabnkhoj-server-1 | │ 1031 │ │ │
myaiabnkhoj-server-1 | │ 1032 │ │ return self │
myaiabnkhoj-server-1 | │ 1033 │ │ │ cast_to │
myaiabnkhoj-server-1 | ╰────────────────────────────╯
myaiabnkhoj-server-1 | BadRequestError: Error code:
myaiabnkhoj-server-1 | 400 - {'error': {'message':
myaiabnkhoj-server-1 | 'response_format` does not
myaiabnkhoj-server-1 | support streaming', 'type':
myaiabnkhoj-server-1 | 'invalid_request_error'}}
myaiabnkhoj-server-1 | [08:04:55.971712] DEBUG uvicorn.error: = connection is protocol.py:1227
myaiabnkhoj-server-1 | CLOSING
myaiabnkhoj-server-1 | [08:04:55.972998] DEBUG uvicorn.error: > CLOSE 1000 (OK) [2 protocol.py:1178
myaiabnkhoj-server-1 | bytes]
myaiabnkhoj-server-1 | [08:04:55.974311] DEBUG uvicorn.error: = connection is protocol.py:1497
myaiabnkhoj-server-1 | CLOSED
myaiabnkhoj-server-1 | [08:04:55.975695] DEBUG uvicorn.error: ! failing connection protocol.py:1412
myaiabnkhoj-server-1 | with code 1006
myaiabnkhoj-server-1 | [08:04:55.977672] ERROR uvicorn.error: closing handshake failed server.py:248
myaiabnkhoj-server-1 | ╭─ Traceback (most recent call last) ─╮
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/server.py:24 │
myaiabnkhoj-server-1 | │ 4 in handler │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 241 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 242 │ │ │ │
myaiabnkhoj-server-1 | │ 243 │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 244 │ │ │ │ await self.c │
myaiabnkhoj-server-1 | │ 245 │ │ │ except Connectio │
myaiabnkhoj-server-1 | │ 246 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 247 │ │ │ except Exception │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 770 in close │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 767 │ │ """ │
myaiabnkhoj-server-1 | │ 768 │ │ try: │
myaiabnkhoj-server-1 | │ 769 │ │ │ async with async │
myaiabnkhoj-server-1 | │ ❱ 770 │ │ │ │ await self.w │
myaiabnkhoj-server-1 | │ 771 │ │ except asyncio.Timeo │
myaiabnkhoj-server-1 | │ 772 │ │ │ # If the close f │
myaiabnkhoj-server-1 | │ 773 │ │ │ # are full, the │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1236 in write_close_frame │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1233 │ │ │ │ data = close │
myaiabnkhoj-server-1 | │ 1234 │ │ │ │
myaiabnkhoj-server-1 | │ 1235 │ │ │ # 7.1.2. Start t │
myaiabnkhoj-server-1 | │ ❱ 1236 │ │ │ await self.write │
myaiabnkhoj-server-1 | │ 1237 │ │
myaiabnkhoj-server-1 | │ 1238 │ async def keepalive_ping │
myaiabnkhoj-server-1 | │ 1239 │ │ """ │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1209 in write_frame │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1206 │ │ │ │ f"Cannot wri │
myaiabnkhoj-server-1 | │ 1207 │ │ │ ) │
myaiabnkhoj-server-1 | │ 1208 │ │ self.write_frame_syn │
myaiabnkhoj-server-1 | │ ❱ 1209 │ │ await self.drain() │
myaiabnkhoj-server-1 | │ 1210 │ │
myaiabnkhoj-server-1 | │ 1211 │ async def write_close_fr │
myaiabnkhoj-server-1 | │ 1212 │ │ self, close: Close, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1198 in drain │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1195 │ │ │ self.fail_connec │
myaiabnkhoj-server-1 | │ 1196 │ │ │ # Wait until the │
myaiabnkhoj-server-1 | │ 1197 │ │ │ # with the corre │
myaiabnkhoj-server-1 | │ ❱ 1198 │ │ │ await self.ensur │
myaiabnkhoj-server-1 | │ 1199 │ │
myaiabnkhoj-server-1 | │ 1200 │ async def write_frame( │
myaiabnkhoj-server-1 | │ 1201 │ │ self, fin: bool, opc │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 939 in ensure_open │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 936 │ │ │ │ return │
myaiabnkhoj-server-1 | │ 937 │ │ │
myaiabnkhoj-server-1 | │ 938 │ │ if self.state is Sta │
myaiabnkhoj-server-1 | │ ❱ 939 │ │ │ raise self.conne │
myaiabnkhoj-server-1 | │ 940 │ │ │
myaiabnkhoj-server-1 | │ 941 │ │ if self.state is Sta │
myaiabnkhoj-server-1 | │ 942 │ │ │ # If we started │
myaiabnkhoj-server-1 | ╰─────────────────────────────────────╯
myaiabnkhoj-server-1 | ConnectionClosedError: sent 1000 (OK);
myaiabnkhoj-server-1 | no close frame received
myaiabnkhoj-server-1 | [08:04:56.261760] INFO uvicorn.error: connection closed server.py:264
myaiabnkhoj-server-1 | [08:04:56.262973] DEBUG uvicorn.error: x half-closing TCP protocol.py:1319
myaiabnkhoj-server-1 | connection

`

stevennt · 2024-06-21T08:10:47Z

This doesn't work: What are the number of articles in Wikipedia?

This works: /general What are the number of articles in Wikipedia?

@sabaimran

stevennt · 2024-06-21T11:14:09Z

I also saw this in a console log:

BadRequestError: Error code:
400 - {'error': {'message':
'response_format` does not
support streaming', 'type':
'invalid_request_error'}}

sabaimran · 2024-06-21T23:18:58Z

Hmm interesting. You should run docker compose up --build when you're restarting the container after pulling from the latest changes.
But you're right, this is likely a different issue. I'd assumed you were using ollama / llama3 and tested on that. The tokenizer shouldn't matter here.
/general is still working, correct?

stevennt · 2024-06-22T01:45:04Z

API: I'm not using Ollama. I'm using a vendor-hosted LLAMA3 API. With that API: Yes /general works faultlessly but the moment I say /notes or /default it breaks (regardless of whether or not I have uploaded any document).

At the same time, OpenAI API works faultlessly with both /general and /notes.

Yes I do run docker-compose build with the revised code. Other test with text changes do show up, meaning the build process was ok.

@sabaimran

stevennt · 2024-06-27T01:10:45Z

Wrapping the API in LiteLLM:

myaiabnkhoj-server-1 | [01:05:12.918899] DEBUG uvicorn.error: < TEXT 'How can I protocol.py:1172
myaiabnkhoj-server-1 | talk to an LLM' [24 bytes]
myaiabnkhoj-server-1 | [01:05:12.928024] DEBUG uvicorn.error: > TEXT '{"type": protocol.py:1178
myaiabnkhoj-server-1 | "status", "message":
myaiabnkhoj-server-1 | "**Understanding ...e":
myaiabnkhoj-server-1 | "application/json"}' [118 bytes]
myaiabnkhoj-server-1 | [01:05:12.941103] WARNING khoj.processor.conversation.utils: utils.py:217
myaiabnkhoj-server-1 | Fallback to default chat model
myaiabnkhoj-server-1 | tokenizer: None.
myaiabnkhoj-server-1 | Configure tokenizer for unsupported
myaiabnkhoj-server-1 | model: llama3-70b-8192 in Khoj settings
myaiabnkhoj-server-1 | to improve context stuffing.
myaiabnkhoj-server-1 | [01:05:14.901754] DEBUG khoj.processor.conversation.openai before_sleep.py:65
myaiabnkhoj-server-1 | .utils: Retrying
myaiabnkhoj-server-1 | khoj.processor.conversation.openai
myaiabnkhoj-server-1 | .utils.completion_with_backoff in
myaiabnkhoj-server-1 | 0.8713305851005899 seconds as it
myaiabnkhoj-server-1 | raised BadRequestError: Error
myaiabnkhoj-server-1 | code: 400 - {'error': {'message':
myaiabnkhoj-server-1 | "litellm.BadRequestError:
myaiabnkhoj-server-1 | BadRequestError: GroqException -
myaiabnkhoj-server-1 | Error code: 400 - {'error':
myaiabnkhoj-server-1 | {'message': 'response_format does myaiabnkhoj-server-1 | not support streaming', 'type': myaiabnkhoj-server-1 | 'invalid_request_error'}}", myaiabnkhoj-server-1 | 'type': None, 'param': None, myaiabnkhoj-server-1 | 'code': 400}}. myaiabnkhoj-server-1 | [01:05:16.811876] DEBUG khoj.processor.conversation.openai before_sleep.py:65 myaiabnkhoj-server-1 | .utils: Retrying myaiabnkhoj-server-1 | khoj.processor.conversation.openai myaiabnkhoj-server-1 | .utils.completion_with_backoff in myaiabnkhoj-server-1 | 0.04274018957486159 seconds as it myaiabnkhoj-server-1 | raised BadRequestError: Error myaiabnkhoj-server-1 | code: 400 - {'error': {'message': myaiabnkhoj-server-1 | "litellm.BadRequestError: myaiabnkhoj-server-1 | BadRequestError: GroqException - myaiabnkhoj-server-1 | Error code: 400 - {'error': myaiabnkhoj-server-1 | {'message': 'response_format does
myaiabnkhoj-server-1 | not support streaming', 'type':
myaiabnkhoj-server-1 | 'invalid_request_error'}}",
myaiabnkhoj-server-1 | 'type': None, 'param': None,
myaiabnkhoj-server-1 | 'code': 400}}.
myaiabnkhoj-server-1 | [01:05:17.897260] DEBUG khoj.routers.helpers: Chat actor: helpers.py:175
myaiabnkhoj-server-1 | Infer information sources to refer:
myaiabnkhoj-server-1 | 4.964 seconds
myaiabnkhoj-server-1 | [01:05:17.901853] ERROR uvicorn.error: Exception in websockets_impl.py:188
myaiabnkhoj-server-1 | ASGI application
myaiabnkhoj-server-1 |
myaiabnkhoj-server-1 | ╭─ Traceback (most recent ca─╮
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/uvicorn/prot │
myaiabnkhoj-server-1 | │ ocols/websockets/websocket │
myaiabnkhoj-server-1 | │ s_impl.py:184 in run_asgi │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 181 │ │ termination │
myaiabnkhoj-server-1 | │ 182 │ │ """ │
myaiabnkhoj-server-1 | │ 183 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 184 │ │ │ result = │
myaiabnkhoj-server-1 | │ 185 │ │ except BaseE │
myaiabnkhoj-server-1 | │ 186 │ │ │ self.clo │
myaiabnkhoj-server-1 | │ 187 │ │ │ msg = "E │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/uvicorn/midd │
myaiabnkhoj-server-1 | │ leware/proxy_headers.py:75 │
myaiabnkhoj-server-1 | │ in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 72 │ │ │ │ │ p │
myaiabnkhoj-server-1 | │ 73 │ │ │ │ │ s │
myaiabnkhoj-server-1 | │ 74 │ │ │
myaiabnkhoj-server-1 | │ ❱ 75 │ │ return await │
myaiabnkhoj-server-1 | │ 76 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/fastapi/appl │
myaiabnkhoj-server-1 | │ ications.py:1054 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1051 │ async def __cal │
myaiabnkhoj-server-1 | │ 1052 │ │ if self.roo │
myaiabnkhoj-server-1 | │ 1053 │ │ │ scope[" │
myaiabnkhoj-server-1 | │ ❱ 1054 │ │ await super │
myaiabnkhoj-server-1 | │ 1055 │ │
myaiabnkhoj-server-1 | │ 1056 │ def add_api_rou │
myaiabnkhoj-server-1 | │ 1057 │ │ self, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ap │
myaiabnkhoj-server-1 | │ plications.py:123 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 120 │ │ scope["app"] │
myaiabnkhoj-server-1 | │ 121 │ │ if self.midd │
myaiabnkhoj-server-1 | │ 122 │ │ │ self.mid │
myaiabnkhoj-server-1 | │ ❱ 123 │ │ await self.m │
myaiabnkhoj-server-1 | │ 124 │ │
myaiabnkhoj-server-1 | │ 125 │ def on_event(sel │
myaiabnkhoj-server-1 | │ 126 │ │ return self. │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/errors.py:151 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 148 │ │
myaiabnkhoj-server-1 | │ 149 │ async def __call │
myaiabnkhoj-server-1 | │ 150 │ │ if scope["ty │
myaiabnkhoj-server-1 | │ ❱ 151 │ │ │ await se │
myaiabnkhoj-server-1 | │ 152 │ │ │ return │
myaiabnkhoj-server-1 | │ 153 │ │ │
myaiabnkhoj-server-1 | │ 154 │ │ response_sta │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/sessions.py:85 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 82 │ │ │ │ │ h │
myaiabnkhoj-server-1 | │ 83 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 84 │ │ │
myaiabnkhoj-server-1 | │ ❱ 85 │ │ await self.ap │
myaiabnkhoj-server-1 | │ 86 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/configure.py │
myaiabnkhoj-server-1 | │ :315 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 312 │ │ async def __ │
myaiabnkhoj-server-1 | │ 313 │ │ │ if scope │
myaiabnkhoj-server-1 | │ 314 │ │ │ │ scop │
myaiabnkhoj-server-1 | │ ❱ 315 │ │ │ await se │
myaiabnkhoj-server-1 | │ 316 │ │ │
myaiabnkhoj-server-1 | │ 317 │ │ def init │
myaiabnkhoj-server-1 | │ 318 │ │ │ super(). │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/authentication.py │
myaiabnkhoj-server-1 | │ :49 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 46 │ │ if auth_resul │
myaiabnkhoj-server-1 | │ 47 │ │ │ auth_resu │
myaiabnkhoj-server-1 | │ 48 │ │ scope["auth"] │
myaiabnkhoj-server-1 | │ ❱ 49 │ │ await self.ap │
myaiabnkhoj-server-1 | │ 50 │ │
myaiabnkhoj-server-1 | │ 51 │ @staticmethod │
myaiabnkhoj-server-1 | │ 52 │ def default_on_er │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/cors.py:77 in │
myaiabnkhoj-server-1 | │ call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 74 │ │
myaiabnkhoj-server-1 | │ 75 │ async def __call │
myaiabnkhoj-server-1 | │ 76 │ │ if scope["ty │
myaiabnkhoj-server-1 | │ ❱ 77 │ │ │ await se │
myaiabnkhoj-server-1 | │ 78 │ │ │ return │
myaiabnkhoj-server-1 | │ 79 │ │ │
myaiabnkhoj-server-1 | │ 80 │ │ method = sco │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/mi │
myaiabnkhoj-server-1 | │ ddleware/exceptions.py:65 │
myaiabnkhoj-server-1 | │ in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 62 │ │ else: │
myaiabnkhoj-server-1 | │ 63 │ │ │ conn = We │
myaiabnkhoj-server-1 | │ 64 │ │ │
myaiabnkhoj-server-1 | │ ❱ 65 │ │ await wrap_ap │
myaiabnkhoj-server-1 | │ 66 │ │
myaiabnkhoj-server-1 | │ 67 │ def http_exceptio │
myaiabnkhoj-server-1 | │ 68 │ │ assert isinst │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/_e │
myaiabnkhoj-server-1 | │ xception_handler.py:64 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │
myaiabnkhoj-server-1 | │ 62 │ │ │ │
myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │
myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 65 │ │ │ │
myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │
myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/e │
myaiabnkhoj-server-1 | │ xception_handler.py:53 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 51 │ │ │
myaiabnkhoj-server-1 | │ 52 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │
myaiabnkhoj-server-1 | │ 54 │ │ except Except │
myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │
myaiabnkhoj-server-1 | │ 56 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:756 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 753 │ │ """ │
myaiabnkhoj-server-1 | │ 754 │ │ The main ent │
myaiabnkhoj-server-1 | │ 755 │ │ """ │
myaiabnkhoj-server-1 | │ ❱ 756 │ │ await self.m │
myaiabnkhoj-server-1 | │ 757 │ │
myaiabnkhoj-server-1 | │ 758 │ async def app(se │
myaiabnkhoj-server-1 | │ 759 │ │ assert scope │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:776 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 773 │ │ │ match, c │
myaiabnkhoj-server-1 | │ 774 │ │ │ if match │
myaiabnkhoj-server-1 | │ 775 │ │ │ │ scop │
myaiabnkhoj-server-1 | │ ❱ 776 │ │ │ │ awai │
myaiabnkhoj-server-1 | │ 777 │ │ │ │ retu │
myaiabnkhoj-server-1 | │ 778 │ │ │ elif mat │
myaiabnkhoj-server-1 | │ 779 │ │ │ │ part │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:373 in handle │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 370 │ │ return URLPa │
myaiabnkhoj-server-1 | │ 371 │ │
myaiabnkhoj-server-1 | │ 372 │ async def handle │
myaiabnkhoj-server-1 | │ ❱ 373 │ │ await self.a │
myaiabnkhoj-server-1 | │ 374 │ │
myaiabnkhoj-server-1 | │ 375 │ def eq(self, │
myaiabnkhoj-server-1 | │ 376 │ │ return ( │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:96 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 93 │ │ async def ap │
myaiabnkhoj-server-1 | │ 94 │ │ │ await fu │
myaiabnkhoj-server-1 | │ 95 │ │ │
myaiabnkhoj-server-1 | │ ❱ 96 │ │ await wrap_a │
myaiabnkhoj-server-1 | │ 97 │ │
myaiabnkhoj-server-1 | │ 98 │ return app │
myaiabnkhoj-server-1 | │ 99 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/e │
myaiabnkhoj-server-1 | │ xception_handler.py:64 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 61 │ │ │ │ handl │
myaiabnkhoj-server-1 | │ 62 │ │ │ │
myaiabnkhoj-server-1 | │ 63 │ │ │ if handle │
myaiabnkhoj-server-1 | │ ❱ 64 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 65 │ │ │ │
myaiabnkhoj-server-1 | │ 66 │ │ │ if respon │
myaiabnkhoj-server-1 | │ 67 │ │ │ │ msg = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/e │
myaiabnkhoj-server-1 | │ xception_handler.py:53 in │
myaiabnkhoj-server-1 | │ wrapped_app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 50 │ │ │ await sen │
myaiabnkhoj-server-1 | │ 51 │ │ │
myaiabnkhoj-server-1 | │ 52 │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 53 │ │ │ await app │
myaiabnkhoj-server-1 | │ 54 │ │ except Except │
myaiabnkhoj-server-1 | │ 55 │ │ │ handler = │
myaiabnkhoj-server-1 | │ 56 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/starlette/ro │
myaiabnkhoj-server-1 | │ uting.py:94 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 91 │ │ session = We │
myaiabnkhoj-server-1 | │ 92 │ │ │
myaiabnkhoj-server-1 | │ 93 │ │ async def ap │
myaiabnkhoj-server-1 | │ ❱ 94 │ │ │ await fu │
myaiabnkhoj-server-1 | │ 95 │ │ │
myaiabnkhoj-server-1 | │ 96 │ │ await wrap_a │
myaiabnkhoj-server-1 | │ 97 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/fastapi/rout │
myaiabnkhoj-server-1 | │ ing.py:348 in app │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 345 │ │ │ if erro │
myaiabnkhoj-server-1 | │ 346 │ │ │ │ rai │
myaiabnkhoj-server-1 | │ 347 │ │ │ assert │
myaiabnkhoj-server-1 | │ ❱ 348 │ │ │ await d │
myaiabnkhoj-server-1 | │ 349 │ │
myaiabnkhoj-server-1 | │ 350 │ return app │
myaiabnkhoj-server-1 | │ 351 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/api │
myaiabnkhoj-server-1 | │ chat.py:593 in │
myaiabnkhoj-server-1 | │ websocket_endpoint │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 590 │ │ is_automate │
myaiabnkhoj-server-1 | │ 591 │ │ │
myaiabnkhoj-server-1 | │ 592 │ │ if conversa │
myaiabnkhoj-server-1 | │ ❱ 593 │ │ │ convers │
myaiabnkhoj-server-1 | │ is_automated_task) │
myaiabnkhoj-server-1 | │ 594 │ │ │ convers │
myaiabnkhoj-server-1 | │ conversation_comman │
myaiabnkhoj-server-1 | │ 595 │ │ │ await s │
myaiabnkhoj-server-1 | │ {conversation_comma │
myaiabnkhoj-server-1 | │ 596 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │
myaiabnkhoj-server-1 | │ ers.py:250 in │
myaiabnkhoj-server-1 | │ aget_relevant_information │
myaiabnkhoj-server-1 | │ sources │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 247 │ ) │
myaiabnkhoj-server-1 | │ 248 │ │
myaiabnkhoj-server-1 | │ 249 │ with timer("Cha │
myaiabnkhoj-server-1 | │ ❱ 250 │ │ response = │
myaiabnkhoj-server-1 | │ response_type="json │
myaiabnkhoj-server-1 | │ 251 │ │
myaiabnkhoj-server-1 | │ 252 │ try: │
myaiabnkhoj-server-1 | │ 253 │ │ response = │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/routers/help │
myaiabnkhoj-server-1 | │ ers.py:547 in │
myaiabnkhoj-server-1 | │ send_message_to_model_wrap │
myaiabnkhoj-server-1 | │ per │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 544 │ │ │ tokeniz │
myaiabnkhoj-server-1 | │ 545 │ │ ) │
myaiabnkhoj-server-1 | │ 546 │ │ │
myaiabnkhoj-server-1 | │ ❱ 547 │ │ openai_resp │
myaiabnkhoj-server-1 | │ 548 │ │ │ message │
myaiabnkhoj-server-1 | │ 549 │ │ │ api_key │
myaiabnkhoj-server-1 | │ 550 │ │ │ model=c │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │
myaiabnkhoj-server-1 | │ nversation/openai/gpt.py:9 │
myaiabnkhoj-server-1 | │ 8 in send_message_to_model │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 95 │ """ │
myaiabnkhoj-server-1 | │ 96 │ │
myaiabnkhoj-server-1 | │ 97 │ # Get Response f │
myaiabnkhoj-server-1 | │ ❱ 98 │ return completio │
myaiabnkhoj-server-1 | │ 99 │ │ messages=mes │
myaiabnkhoj-server-1 | │ 100 │ │ model=model, │
myaiabnkhoj-server-1 | │ 101 │ │ openai_api_k │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:330 in wrapped_f │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 327 │ │ │ f, funct │
myaiabnkhoj-server-1 | │ 328 │ │ ) │
myaiabnkhoj-server-1 | │ 329 │ │ def wrapped │
myaiabnkhoj-server-1 | │ ❱ 330 │ │ │ return s │
myaiabnkhoj-server-1 | │ 331 │ │ │
myaiabnkhoj-server-1 | │ 332 │ │ def retry_wi │
myaiabnkhoj-server-1 | │ 333 │ │ │ return s │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:467 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 464 │ │ │
myaiabnkhoj-server-1 | │ 465 │ │ retry_state │
myaiabnkhoj-server-1 | │ 466 │ │ while True: │
myaiabnkhoj-server-1 | │ ❱ 467 │ │ │ do = sel │
myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │
myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │
myaiabnkhoj-server-1 | │ 470 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:368 in iter │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 365 │ │ self.begin │
myaiabnkhoj-server-1 | │ 366 │ │ result = Non │
myaiabnkhoj-server-1 | │ 367 │ │ for action i │
myaiabnkhoj-server-1 | │ ❱ 368 │ │ │ result = │
myaiabnkhoj-server-1 | │ 369 │ │ return resul │
myaiabnkhoj-server-1 | │ 370 │ │
myaiabnkhoj-server-1 | │ 371 │ def _begin_iter( │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:410 in exc_check │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 407 │ │ │ │ fut │
myaiabnkhoj-server-1 | │ 408 │ │ │ │ retr │
myaiabnkhoj-server-1 | │ 409 │ │ │ │ if s │
myaiabnkhoj-server-1 | │ ❱ 410 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 411 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ 412 │ │ │ │
myaiabnkhoj-server-1 | │ 413 │ │ │ self._ad │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:183 in reraise │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 180 │ │
myaiabnkhoj-server-1 | │ 181 │ def reraise(self │
myaiabnkhoj-server-1 | │ 182 │ │ if self.last │
myaiabnkhoj-server-1 | │ ❱ 183 │ │ │ raise se │
myaiabnkhoj-server-1 | │ 184 │ │ raise self │
myaiabnkhoj-server-1 | │ 185 │ │
myaiabnkhoj-server-1 | │ 186 │ def str(self │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │
myaiabnkhoj-server-1 | │ rent/futures/_base.py:451 │
myaiabnkhoj-server-1 | │ in result │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 448 │ │ │ │ if s │
myaiabnkhoj-server-1 | │ 449 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 450 │ │ │ │ elif │
myaiabnkhoj-server-1 | │ ❱ 451 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 452 │ │ │ │ │
myaiabnkhoj-server-1 | │ 453 │ │ │ │ self │
myaiabnkhoj-server-1 | │ 454 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/lib/python3.10/concur │
myaiabnkhoj-server-1 | │ rent/futures/_base.py:403 │
myaiabnkhoj-server-1 | │ in __get_result │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 400 │ def __get_result │
myaiabnkhoj-server-1 | │ 401 │ │ if self._exc │
myaiabnkhoj-server-1 | │ 402 │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 403 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ 404 │ │ │ finally: │
myaiabnkhoj-server-1 | │ 405 │ │ │ │ # Br │
myaiabnkhoj-server-1 | │ 406 │ │ │ │ self │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/tenacity/i │
myaiabnkhoj-server-1 | │ nit.py:470 in call │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 467 │ │ │ do = sel │
myaiabnkhoj-server-1 | │ 468 │ │ │ if isins │
myaiabnkhoj-server-1 | │ 469 │ │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 470 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 471 │ │ │ │ exce │
myaiabnkhoj-server-1 | │ 472 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 473 │ │ │ │ else │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /app/src/khoj/processor/co │
myaiabnkhoj-server-1 | │ nversation/openai/utils.py │
myaiabnkhoj-server-1 | │ :50 in │
myaiabnkhoj-server-1 | │ completion_with_backoff │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 47 │ │
myaiabnkhoj-server-1 | │ 48 │ formatted_messag │
myaiabnkhoj-server-1 | │ in messages] │
myaiabnkhoj-server-1 | │ 49 │ │
myaiabnkhoj-server-1 | │ ❱ 50 │ chat = client.ch │
myaiabnkhoj-server-1 | │ 51 │ │ stream=True, │
myaiabnkhoj-server-1 | │ 52 │ │ messages=for │
myaiabnkhoj-server-1 | │ 53 │ │ model=model, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_util │
myaiabnkhoj-server-1 | │ s/_utils.py:277 in wrapper │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 274 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 275 │ │ │ │ │ │
myaiabnkhoj-server-1 | │ 276 │ │ │ │ rais │
myaiabnkhoj-server-1 | │ ❱ 277 │ │ │ return f │
myaiabnkhoj-server-1 | │ 278 │ │ │
myaiabnkhoj-server-1 | │ 279 │ │ return wrapp │
myaiabnkhoj-server-1 | │ 280 │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/resou │
myaiabnkhoj-server-1 | │ rces/chat/completions.py:6 │
myaiabnkhoj-server-1 | │ 40 in create │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 637 │ │ extra_body: │
myaiabnkhoj-server-1 | │ 638 │ │ timeout: fl │
myaiabnkhoj-server-1 | │ 639 │ ) -> ChatComple │
myaiabnkhoj-server-1 | │ ❱ 640 │ │ return self │
myaiabnkhoj-server-1 | │ 641 │ │ │ "/chat/ │
myaiabnkhoj-server-1 | │ 642 │ │ │ body=ma │
myaiabnkhoj-server-1 | │ 643 │ │ │ │ { │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:1250 in post │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1247 │ │ opts = Fina │
myaiabnkhoj-server-1 | │ 1248 │ │ │ method= │
myaiabnkhoj-server-1 | │ **options │
myaiabnkhoj-server-1 | │ 1249 │ │ ) │
myaiabnkhoj-server-1 | │ ❱ 1250 │ │ return cast │
myaiabnkhoj-server-1 | │ stream_cls=stream_c │
myaiabnkhoj-server-1 | │ 1251 │ │
myaiabnkhoj-server-1 | │ 1252 │ def patch( │
myaiabnkhoj-server-1 | │ 1253 │ │ self, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:931 in request │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 928 │ │ stream: boo │
myaiabnkhoj-server-1 | │ 929 │ │ stream_cls: │
myaiabnkhoj-server-1 | │ 930 │ ) -> ResponseT │
myaiabnkhoj-server-1 | │ ❱ 931 │ │ return self │
myaiabnkhoj-server-1 | │ 932 │ │ │ cast_to │
myaiabnkhoj-server-1 | │ 933 │ │ │ options │
myaiabnkhoj-server-1 | │ 934 │ │ │ stream= │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/ │
myaiabnkhoj-server-1 | │ dist-packages/openai/_base │
myaiabnkhoj-server-1 | │ _client.py:1030 in │
myaiabnkhoj-server-1 | │ _request │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1027 │ │ │ │ err │
myaiabnkhoj-server-1 | │ 1028 │ │ │ │
myaiabnkhoj-server-1 | │ 1029 │ │ │ log.deb │
myaiabnkhoj-server-1 | │ ❱ 1030 │ │ │ raise s │
myaiabnkhoj-server-1 | │ 1031 │ │ │
myaiabnkhoj-server-1 | │ 1032 │ │ return self │
myaiabnkhoj-server-1 | │ 1033 │ │ │ cast_to │
myaiabnkhoj-server-1 | ╰────────────────────────────╯
myaiabnkhoj-server-1 | BadRequestError: Error code:
myaiabnkhoj-server-1 | 400 - {'error': {'message':
myaiabnkhoj-server-1 | "litellm.BadRequestError:
myaiabnkhoj-server-1 | BadRequestError: GroqException
myaiabnkhoj-server-1 | - Error code: 400 - {'error':
myaiabnkhoj-server-1 | {'message': 'response_format`
myaiabnkhoj-server-1 | does not support streaming',
myaiabnkhoj-server-1 | 'type':
myaiabnkhoj-server-1 | 'invalid_request_error'}}",
myaiabnkhoj-server-1 | 'type': None, 'param': None,
myaiabnkhoj-server-1 | 'code': 400}}
myaiabnkhoj-server-1 | [01:05:19.184406] DEBUG uvicorn.error: = connection is protocol.py:1227
myaiabnkhoj-server-1 | CLOSING
myaiabnkhoj-server-1 | [01:05:19.185674] DEBUG uvicorn.error: > CLOSE 1000 (OK) [2 protocol.py:1178
myaiabnkhoj-server-1 | bytes]
myaiabnkhoj-server-1 | [01:05:19.186874] DEBUG uvicorn.error: = connection is protocol.py:1497
myaiabnkhoj-server-1 | CLOSED
myaiabnkhoj-server-1 | [01:05:19.188183] DEBUG uvicorn.error: % sending keepalive protocol.py:1262
myaiabnkhoj-server-1 | ping
myaiabnkhoj-server-1 | [01:05:19.189827] DEBUG uvicorn.error: ! failing connection protocol.py:1412
myaiabnkhoj-server-1 | with code 1006
myaiabnkhoj-server-1 | [01:05:19.191496] ERROR uvicorn.error: closing handshake failed server.py:248
myaiabnkhoj-server-1 | ╭─ Traceback (most recent call last) ─╮
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/server.py:24 │
myaiabnkhoj-server-1 | │ 4 in handler │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 241 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 242 │ │ │ │
myaiabnkhoj-server-1 | │ 243 │ │ │ try: │
myaiabnkhoj-server-1 | │ ❱ 244 │ │ │ │ await self.c │
myaiabnkhoj-server-1 | │ 245 │ │ │ except Connectio │
myaiabnkhoj-server-1 | │ 246 │ │ │ │ raise │
myaiabnkhoj-server-1 | │ 247 │ │ │ except Exception │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 770 in close │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 767 │ │ """ │
myaiabnkhoj-server-1 | │ 768 │ │ try: │
myaiabnkhoj-server-1 | │ 769 │ │ │ async with async │
myaiabnkhoj-server-1 | │ ❱ 770 │ │ │ │ await self.w │
myaiabnkhoj-server-1 | │ 771 │ │ except asyncio.Timeo │
myaiabnkhoj-server-1 | │ 772 │ │ │ # If the close f │
myaiabnkhoj-server-1 | │ 773 │ │ │ # are full, the │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1236 in write_close_frame │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1233 │ │ │ │ data = close │
myaiabnkhoj-server-1 | │ 1234 │ │ │ │
myaiabnkhoj-server-1 | │ 1235 │ │ │ # 7.1.2. Start t │
myaiabnkhoj-server-1 | │ ❱ 1236 │ │ │ await self.write │
myaiabnkhoj-server-1 | │ 1237 │ │
myaiabnkhoj-server-1 | │ 1238 │ async def keepalive_ping │
myaiabnkhoj-server-1 | │ 1239 │ │ """ │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1209 in write_frame │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1206 │ │ │ │ f"Cannot wri │
myaiabnkhoj-server-1 | │ 1207 │ │ │ ) │
myaiabnkhoj-server-1 | │ 1208 │ │ self.write_frame_syn │
myaiabnkhoj-server-1 | │ ❱ 1209 │ │ await self.drain() │
myaiabnkhoj-server-1 | │ 1210 │ │
myaiabnkhoj-server-1 | │ 1211 │ async def write_close_fr │
myaiabnkhoj-server-1 | │ 1212 │ │ self, close: Close, │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 1198 in drain │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 1195 │ │ │ self.fail_connec │
myaiabnkhoj-server-1 | │ 1196 │ │ │ # Wait until the │
myaiabnkhoj-server-1 | │ 1197 │ │ │ # with the corre │
myaiabnkhoj-server-1 | │ ❱ 1198 │ │ │ await self.ensur │
myaiabnkhoj-server-1 | │ 1199 │ │
myaiabnkhoj-server-1 | │ 1200 │ async def write_frame( │
myaiabnkhoj-server-1 | │ 1201 │ │ self, fin: bool, opc │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ /usr/local/lib/python3.10/dist-pack │
myaiabnkhoj-server-1 | │ ages/websockets/legacy/protocol.py: │
myaiabnkhoj-server-1 | │ 939 in ensure_open │
myaiabnkhoj-server-1 | │ │
myaiabnkhoj-server-1 | │ 936 │ │ │ │ return │
myaiabnkhoj-server-1 | │ 937 │ │ │
myaiabnkhoj-server-1 | │ 938 │ │ if self.state is Sta │
myaiabnkhoj-server-1 | │ ❱ 939 │ │ │ raise self.conne │
myaiabnkhoj-server-1 | │ 940 │ │ │
myaiabnkhoj-server-1 | │ 941 │ │ if self.state is Sta │
myaiabnkhoj-server-1 | │ 942 │ │ │ # If we started │
myaiabnkhoj-server-1 | ╰─────────────────────────────────────╯
myaiabnkhoj-server-1 | ConnectionClosedError: sent 1000 (OK);
myaiabnkhoj-server-1 | no close frame received
myaiabnkhoj-server-1 | [01:05:19.537140] INFO uvicorn.error: connection closed server.py:264
myaiabnkhoj-server-1 | [01:05:19.538533] DEBUG uvicorn.error: x half-closing TCP protocol.py:1319
myaiabnkhoj-server-1 | connection
myaiabnkhoj-database-1 | 2024-06-27 01:05:25.934 UTC [28] LOG: checkpoint starting: time
myaiabnkhoj-database-1 | 2024-06-27 01:05:27.270 UTC [28] LOG: checkpoint complete: wrote 13 buffers (0.1%); 0 WAL file(s) added, 0 removed, 0 recycled; write=1.038 s, sync=0.145 s, total=1.336 s; sync files=12, longest=0.113 s, average=0.012 s; distance=15 kB, estimate=15 kB

Ok I tried to wrap that LLAMA3 API (via GROQ) behind LiteLLM as well to ensure it follows the OpenAi format. The error is the same. @sabaimran

stevennt · 2024-06-27T01:13:19Z

and here is the log from litellm backend:
litellm-1 | 01:05:17 - LiteLLM Router:ERROR: router.py:2313 - An exception occurred - litellm.BadRequestError: BadRequestError: GroqException - Error code: 400 - {'error': {'message': 'response_format does not support streaming', 'type': 'invalid_request_error'}} LiteLLM Retried: 1 times, LiteLLM Max Retries: 2 litellm-1 | 01:05:17 - LiteLLM Proxy:ERROR: proxy_server.py:3141 - litellm.proxy.proxy_server.chat_completion(): Exception occured - litellm.BadRequestError: BadRequestError: GroqException - Error code: 400 - {'error': {'message': 'response_format does not support streaming', 'type': 'invalid_request_error'}} LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
litellm-1 | Traceback (most recent call last):
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/llms/openai.py", line 968, in async_streaming
litellm-1 | response = await openai_aclient.chat.completions.create(
litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 1181, in create
litellm-1 | return await self._post(
litellm-1 | ^^^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/openai/_base_client.py", line 1790, in post
litellm-1 | return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/openai/_base_client.py", line 1493, in request
litellm-1 | return await self._request(
litellm-1 | ^^^^^^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/openai/_base_client.py", line 1584, in _request
litellm-1 | raise self._make_status_error_from_response(err.response) from None
litellm-1 | openai.BadRequestError: Error code: 400 - {'error': {'message': 'response_format does not support streaming', 'type': 'invalid_request_error'}} litellm-1 | litellm-1 | During handling of the above exception, another exception occurred: litellm-1 | litellm-1 | Traceback (most recent call last): litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/main.py", line 379, in acompletion litellm-1 | response = await init_response litellm-1 | ^^^^^^^^^^^^^^^^^^^ litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/llms/openai.py", line 991, in async_streaming litellm-1 | raise OpenAIError(status_code=e.status_code, message=str(e)) litellm-1 | litellm.llms.openai.OpenAIError: Error code: 400 - {'error': {'message': 'response_format does not support streaming', 'type': 'invalid_request_error'}}
litellm-1 |
litellm-1 | During handling of the above exception, another exception occurred:
litellm-1 |
litellm-1 | Traceback (most recent call last):
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/proxy/proxy_server.py", line 3043, in chat_completion
litellm-1 | responses = await llm_responses
litellm-1 | ^^^^^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 643, in acompletion
litellm-1 | raise e
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 631, in acompletion
litellm-1 | response = await self.async_function_with_fallbacks(**kwargs)
litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2315, in async_function_with_fallbacks
litellm-1 | raise original_exception
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2177, in async_function_with_fallbacks
litellm-1 | raise e
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2160, in async_function_with_fallbacks
litellm-1 | response = await self.async_function_with_retries(*args, **kwargs)
litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2415, in async_function_with_retries
litellm-1 | raise original_exception
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 2337, in async_function_with_retries
litellm-1 | response = await original_function(*args, **kwargs)
litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 773, in _acompletion
litellm-1 | raise e
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/router.py", line 745, in _acompletion
litellm-1 | response = await _response
litellm-1 | ^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 1442, in wrapper_async
litellm-1 | raise e
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 1270, in wrapper_async
litellm-1 | result = await original_function(*args, **kwargs)
litellm-1 | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/main.py", line 407, in acompletion
litellm-1 | raise exception_type(
litellm-1 | ^^^^^^^^^^^^^^^
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 7503, in exception_type
litellm-1 | raise e
litellm-1 | File "/usr/local/lib/python3.11/site-packages/litellm/utils.py", line 5818, in exception_type
litellm-1 | raise BadRequestError(
litellm-1 | litellm.exceptions.BadRequestError: litellm.BadRequestError: BadRequestError: GroqException - Error code: 400 - {'error': {'message': 'response_format` does not support streaming', 'type': 'invalid_request_error'}} LiteLLM Retried: 1 times, LiteLLM Max Retries: 2
litellm-1 |
litellm-1 | 01:05:17 - LiteLLM Proxy:ERROR: _common.py:120 - Giving up chat_completion(...) after 1 tries (litellm.proxy._types.ProxyException)
litellm-1 | INFO: 172.19.0.11:60098 - "POST /v1/chat/completions HTTP/1.1" 400 Bad Request

@sabaimran

stevennt · 2024-06-27T01:20:48Z

Could it be something with the streaming settings under the /notes mode?

stevennt · 2024-07-08T00:56:27Z

@sabaimran @debanjum Not sure if this is just for me or any crucial step I'm missing, but so far: Only OpenAPI endpoint works when talking to notes. All other OpenAI-compatible endpoints fail when talking to notes and only work in /general mode.

I have tried:

GROQ API (llama3-70b-8192 model)
LLAMA3 served from LiteLLM
NousResearch/Hermes-2-Pro-Mistral-7B-GGUF

So far, this part looks the most suspicious in the log:

Btw, the backend has a myriad of configuration options (good job to Khoj for making it so configurable), but the backend documentation is not catching up with that :). Would be helpful to add them!

sabaimran · 2024-09-16T21:00:50Z

@stevennt , sorry for the long gap in responding to this! I see why this is happening. It's because the OpenAI API Compatible server you're using (in this case, via LiteLLM or Groq) fails when it sees a parameter it doesn't recognize. Specifically, it seems to be getting triggered by response_format, which allows is to enforce the JSON scheme with OpenAI.

I don't seem to be running into this error with any of the OpenAI API compatible servers we're using. For LiteLLM, I think you should be able to configure the setting for drop params to change the behavior of this failure.

stevennt · 2024-09-18T07:03:26Z

Hey @sabaimran excellent. It was indeed that drop_params that solved the problem. Khoj seems to work now with the LiteLLM & Groq combination.
Thanks.

stevennt added the fix Fix something that isn't working as expected label Jun 21, 2024

stevennt changed the title ~~[FIX]~~ [FIX] The chat fails when querying with /notes while running LLAMA3 API Jun 21, 2024

sabaimran closed this as completed in d3c0111 Jun 21, 2024

sabaimran reopened this Jun 21, 2024

stevennt closed this as completed Sep 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[FIX] The chat fails when querying with /notes while running LLAMA3 API #831

[FIX] The chat fails when querying with /notes while running LLAMA3 API #831

stevennt commented Jun 21, 2024

stevennt commented Jun 21, 2024

sabaimran commented Jun 21, 2024

stevennt commented Jun 21, 2024

sabaimran commented Jun 21, 2024

sabaimran commented Jun 21, 2024 •

edited

Loading

stevennt commented Jun 21, 2024

sabaimran commented Jun 21, 2024 •

edited

Loading

stevennt commented Jun 21, 2024

stevennt commented Jun 21, 2024

stevennt commented Jun 21, 2024

sabaimran commented Jun 21, 2024

stevennt commented Jun 22, 2024 •

edited

Loading

stevennt commented Jun 27, 2024

stevennt commented Jun 27, 2024

stevennt commented Jun 27, 2024

stevennt commented Jul 8, 2024 •

edited

Loading

sabaimran commented Sep 16, 2024

stevennt commented Sep 18, 2024

[FIX] The chat fails when querying with /notes while running LLAMA3 API #831

[FIX] The chat fails when querying with /notes while running LLAMA3 API #831

Comments

stevennt commented Jun 21, 2024

Describe the bug

To Reproduce

Screenshots

Platform

If self-hosted

Additional context

stevennt commented Jun 21, 2024

sabaimran commented Jun 21, 2024

stevennt commented Jun 21, 2024

sabaimran commented Jun 21, 2024

sabaimran commented Jun 21, 2024 • edited Loading

stevennt commented Jun 21, 2024

sabaimran commented Jun 21, 2024 • edited Loading

stevennt commented Jun 21, 2024

stevennt commented Jun 21, 2024

stevennt commented Jun 21, 2024

sabaimran commented Jun 21, 2024

stevennt commented Jun 22, 2024 • edited Loading

stevennt commented Jun 27, 2024

stevennt commented Jun 27, 2024

stevennt commented Jun 27, 2024

stevennt commented Jul 8, 2024 • edited Loading

sabaimran commented Sep 16, 2024

stevennt commented Sep 18, 2024

sabaimran commented Jun 21, 2024 •

edited

Loading

sabaimran commented Jun 21, 2024 •

edited

Loading

stevennt commented Jun 22, 2024 •

edited

Loading

stevennt commented Jul 8, 2024 •

edited

Loading