Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix: Refactor AsyncExitStack usage to resolve lock handling errors #1862

Conversation

sergey21000
Copy link

Fixes #1861

This PR replaces the manual creation of exit_stack = contextlib.AsyncExitStack() with its use as a context manager in the create_completion and create_chat_completion functions.

The original implementation manually managed exit_stack and explicitly called exit_stack.aclose() (in get_event_publisher(on_complete=exit_stack.aclose())), which resulted in the following error when sending a request to llama_cpp.server with "stream": true parameter:

| Traceback (most recent call last):
|   File "/tmp/tmp.mimYBwBzDw/.venv/lib64/python3.11/site-packages/llama_cpp/server/app.py", line 185, in get_event_publisher
|     await on_complete()
|   File "/usr/lib64/python3.11/contextlib.py", line 687, in aclose
|     await self.__aexit__(None, None, None)
|   File "/usr/lib64/python3.11/contextlib.py", line 745, in __aexit__
|     raise exc_details[1]
|   File "/usr/lib64/python3.11/contextlib.py", line 728, in __aexit__
|     cb_suppress = await cb(*exc_details)
|                   ^^^^^^^^^^^^^^^^^^^^^^
|   File "/usr/lib64/python3.11/contextlib.py", line 217, in __aexit__
|     await anext(self.gen)
|   File "/tmp/tmp.mimYBwBzDw/.venv/lib64/python3.11/site-packages/llama_cpp/server/app.py", line 86, in get_llama_proxy
|     llama_inner_lock.release()
|   File "/tmp/tmp.mimYBwBzDw/.venv/lib64/python3.11/site-packages/anyio/_core/_synchronization.py", line 241, in release
|     self._lock.release()
|   File "/tmp/tmp.mimYBwBzDw/.venv/lib64/python3.11/site-packages/anyio/_backends/_asyncio.py", line 1874, in release
|     raise RuntimeError("The current task is not holding this lock")
| RuntimeError: The current task is not holding this lock

This error apparently occurred because an attempt was made to release a lock in an execution context different from the one in which it was acquired.

The changes:

  • Replace manual AsyncExitStack management with the async with contextlib.AsyncExitStack() pattern.
  • Removed explicit calls to exit_stack.aclose() and the passing of on_complete=exit_stack.aclose, as resource cleanup is now handled automatically by the context manager.

@sergey21000 sergey21000 changed the title Refactor AsyncExitStack usage to use context manager and resolve lock handling errors Fix: Refactor AsyncExitStack usage to resolve lock handling errors Dec 12, 2024
@sergey21000 sergey21000 deleted the fix/async-context-manager-handling branch December 16, 2024 21:09
@sergey21000
Copy link
Author

Superseded by #1871.

@gjpower
Copy link
Contributor

gjpower commented Dec 19, 2024

what was the issue in the branch that the pull request was closed?

@sergey21000
Copy link
Author

what was the issue in the branch that the pull request was closed?

When sending a second request while the first one was still running, the server would freeze and crash.

@gjpower
Copy link
Contributor

gjpower commented Dec 23, 2024

I fixed this in #1879 it required opening the exit_stack for streaming request inside the streaming response handler. This prevents the attempting at trying to close it from a different task in which it was opened.

I took inspiration from this branch on the use of with async with contextlib.AsyncExitStack()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Crash due to "The current task is not holding this lock" when {"stream": true}
2 participants