cleanDB() can block the database thread for too long

[cebtenzzre]

Scanning all collections for deleted files, and removing embeddings and documents where necessary, can be a slow operation. This is done on startup and whenever a folder is updated, and is not interruptible or done in chunks/incrementally. This can mean that requesting an embedding for LocalDocs query lookup in a chat, or even just closing the UI, can take significantly longer than expected. When a chat is waiting on an embedding, this will also block the LLM thread, which can make the UI feel really stuck.

We should find a way to make cleanDB(), which performs this operation, block the database thread for less time, e.g. by making it incremental like scanQueue().

[manyoso]

You said there was a simple fix that would help the worst part of this bug without fixing it totally... or so I recall... can you elaborate?