Don't try to embed (recently) deleted documents #2618
Labels
chat
gpt4all-chat issues
chat-ui-ux
Issues related to the look and feel of GPT4All Chat.
enhancement
New feature or request
local-docs
Right now, the localdocs database queues chunks to be embedded via Qt signals to the embedding thread. There is no upper limit to the number of chunks that can be queued, and since the text is included, no check is done of whether the document still exists by the time it is finally popped from the queue.
Try adding a large file to a LocalDocs collection and then removing it shortly after. You will see a lot of CPU usage and LocalDocs showing partial progress for some time, despite no useful work being done.
It would not be hard to add a check on the receiving side of the queue to verify that the chunks are still in the database, so we do not perform unnecessary work.
The text was updated successfully, but these errors were encountered: