Retrieval Service efficiency optimization

[charli117]

…taset fetching

Summary

Fix #13553

There are two main points:

1. multithreading. Instead of using ThreadPoolExecutor (ThreadPoolExecutor) to start a new Thread each time, can reuse threads and reduce the cost of thread creation and destruction.
2. Database query optimization. When querying document and paragraph information in batches, load_only has been used to reduce data transmission and reduce the number of single queries.

Tip

Close issue syntax: Fixes #<issue number> or Resolves #<issue number>, see documentation for more details.

Screenshots

After optimization, the speed of hit test is increased significantly

Checklist

Important

Please review the checklist below before submitting your pull request.

• This change requires a documentation update, included: Dify Document
• I understand that this PR may be closed in case there was no previous discussion or issues. (This doesn't apply to typos!)
• I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
• I've updated the documentation accordingly.
• I ran dev/reformat(backend) and cd web && npx lint-staged(frontend) to appease the lint gods

[JohnJyong]

pls fix the mypy check, thanks @charli117

[charli117]

fix this problem #13553

[bowenliang123]

LGTM with minor comments.

[charli117]

LGTM with minor comments.

@bowenliang123 Very good optimization suggestion, already arranged