Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

20,539% speed up in collections overview #1680

Merged
merged 1 commit into from
Dec 10, 2024
Merged

Conversation

NolanTrem
Copy link
Collaborator

@NolanTrem NolanTrem commented Dec 10, 2024

Important

Optimized get_collections_overview() in collection.py for performance, added timing logs, and updated database schema with new columns.

  • Performance Optimization:
    • get_collections_overview() in collection.py optimized by removing unnecessary joins and simplifying query structure.
    • Significant speed improvement in fetching collections overview.
  • Logging:
    • Added timing logs in get_collections_overview() and list_collections() in collections_router.py to measure execution time.
  • Database Schema:
    • Added user_count and document_count columns to collections table in collection.py.

This description was created by Ellipsis for 0785a21. It will automatically update as commits are pushed.

@emrgnt-cmplxty emrgnt-cmplxty marked this pull request as ready for review December 10, 2024 21:20
@emrgnt-cmplxty emrgnt-cmplxty merged commit a702d3a into main Dec 10, 2024
10 of 30 checks passed
Copy link
Contributor

@ellipsis-dev ellipsis-dev bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❌ Changes requested. Reviewed everything up to 0785a21 in 52 seconds

More details
  • Looked at 165 lines of code in 2 files
  • Skipped 0 files when reviewing.
  • Skipped posting 3 drafted comments based on config settings.
1. py/core/main/api/v3/collections_router.py:1
  • Draft comment:
    The import of the 'time' module is unnecessary in the final code and should be removed.
  • Reason this comment was not posted:
    Decided after close inspection that this draft comment was likely wrong and/or not actionable:
    The 'time' module is used in the code to measure execution time, which suggests that its import is necessary. The comment seems to overlook the fact that the 'time' module is actively used in the code. Therefore, the comment is likely incorrect.
    I might be missing the context of whether the timing code is meant to be temporary or permanent. If it's temporary, the comment could be valid. However, without explicit indication of this, I should assume the code is intentional.
    The presence of the timing code in the diff suggests it is intentional, and without explicit indication that it is temporary, I should assume it is necessary.
    The comment should be deleted because the 'time' module is used in the code, making its import necessary.
2. py/core/providers/database/collection.py:1
  • Draft comment:
    The import of the 'time' module is unnecessary in the final code and should be removed.
  • Reason this comment was not posted:
    Marked as duplicate.
3. py/core/providers/database/collection.py:355
  • Draft comment:
    Replace print statements with proper logging for production use.
  • Reason this comment was not posted:
    Marked as duplicate.

Workflow ID: wflow_bBmOjCSgACjfgP06


Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

@@ -228,6 +230,12 @@ async def list_collections(
limit=limit,
)

t1 = time.time()

print(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Replace print statements with proper logging for production use.

@NolanTrem NolanTrem deleted the Nolan/Collections branch December 10, 2024 21:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants