Aw/pagination #58
Aw/pagination #58
Conversation
…-prototype into aw/pagination
There was a problem hiding this comment.
Do we definitely, always, return results in a consistent and deterministic order/sequence?
If not, I suggest using an offset may not make sense.
I mean, can we point to what, exactly, is stopping results coming back in a different order each time we submit a query? For example, in theory, could this result in page 1 being B, D, A, F and page 2 being D, E, G, H (noting the duplicated D)?
(if I have missed a SortBy somewhere, I apologise! 😁)
| /// <summary> | ||
| /// The value used to define how many records are skipped in the search response (if any). | ||
| /// </summary> | ||
| public int Offset { get; } |
There was a problem hiding this comment.
Should this have an explicit default value?
e.g., if there is no offset given, the int will default to 0 but this is easy to gloss over.
There was a problem hiding this comment.
Change made. On the point above regarding ordering, this has already been noted and the pagination has been built with the expectation that the ticket to handle ordering will follow which such ensure such anomalies don't occur 👍
There was a problem hiding this comment.
... cog search attaches a significance to each result and returns in that order though, so it should be consistent? https://learn.microsoft.com/en-us/azure/search/search-pagination-page-layout#paging-results
There was a problem hiding this comment.
@CathLass -- from that page (emphasis mine):
The results of paginated queries aren't guaranteed to be stable if the underlying index is changing. Paging changes the value of $skip for each page, but each query is independent and operates on the current view of the data as it exists in the index at query time (in other words, there's no caching or snapshot of results, such as those found in a general purpose database).
If the data changes, or additions/deletions occur, then it looks like results on e.g. page 20 might not be consistent?
Similarly, if we have many documents with the same score how will they be sorted? To illustrate, here's an adapted example from the docs -- for records with a rating of 3, are we sure we won't get ids 2,5,6 on one request/page load and 6,5,2 on another?
{ "id": "1", "rating": 5 }
{ "id": "2", "rating": 3 }
{ "id": "5", "rating": 3 }
{ "id": "6", "rating": 3 }
{ "id": "3", "rating": 2 }
{ "id": "4", "rating": 1 }
In terms of what a user would potentially see is different search results for each refresh of e.g., page 2 (or they'd potentially be in a different order).
Equally, it could be a complete non-issue if the ratings are to, say, fifty decimal places and it's massively unlikely that ratings will ever duplicate? I do not know cog search well enough to know what is normal here, sorry!
Dfe.Data.SearchPrototype/Infrastructure/Builders/ISearchOptionsBuilder.cs
Outdated
Show resolved
Hide resolved
| /// The Total Count returned from Establishment search gives us a total | ||
| /// of all available records which correlates with the given search criteria. | ||
| /// </summary> | ||
| public long? TotalNumberOfEstablishments { get; } |
There was a problem hiding this comment.
Does TotalNumberOfEstablishments belong here? I thought this class was here to give us a readonly list of establishments. The encapsulating class SearchResults pulls all the elements of the search response together, so it might sit there better? And that would remove the need to pass that long into the mapper, which is what I object to most :)
....SearchPrototype/Infrastructure/Mappers/PageableSearchResultsToEstablishmentResultsMapper.cs
Outdated
Show resolved
Hide resolved
| @@ -29,6 +29,22 @@ public void Build_WithSize_SearchOptionsWithCorrectSize() | |||
| searchOptions.Size.Should().Be(100); | |||
| } | |||
|
|
|||
| [Fact] | |||
| public void Build_WithOfset_SearchOptionsWithCorrectOffset() | |||
No description provided.