Feat/add_search_dataset_endpoint #16
Conversation
There was a problem hiding this comment.
As it is right now, I think we need to adjust the AARS to accomodate for the pagination issue.
Ideally, we'd want to get the total amount of pages or objects in the PaginationResponse object, this way we can avoid calling .all(). Think about all datatypes, that have a potentially unlimited amount of records, like Datasets being constantly added, etc.
Implementing search is always difficult in these scenarios, but should be possible using the filter method (check out *__in or other modifiers to the search params in the docs/code). If necessary, let's implement more modifiers to have a better search.
| page: int = 1, | ||
| page_size: int = 20, | ||
| ) -> List[DatasetResponse]: | ||
| ) -> dict: |
There was a problem hiding this comment.
Should be a Pydantic type for the API docs
| if by: | ||
| dataset_resp = Dataset.filter(owner=by) | ||
| else: | ||
| dataset_resp = Dataset.fetch_objects() | ||
| datasets = await dataset_resp.page(page=page, page_size=page_size) | ||
|
|
||
| all_datasets = await dataset_resp.all() |
There was a problem hiding this comment.
This can be a very costly operation, only use it if you cannot avoid it
|
|
||
| start = (page - 1) * page_size | ||
| end = start + page_size | ||
| datasets = all_datasets[start:end] |
There was a problem hiding this comment.
Use the .page operation instead of fetching all datasets
Added local search functionality using the search query parameter to filter datasets by name.
Implemented local pagination by calculating start and end indices based on page and page_size parameters.