[FEATURE] searchApi should support search_after

[mstykow]

Is your feature request related to a problem?

The OpenSearch API supports pagination using the search_after parameter: https://opensearch.org/docs/2.4/opensearch/search/paginate/#the-search_after-parameter

However, it seems that opensearch-js's searchApi does not support this parameter: https://opensearch-project.github.io/opensearch-js/2.1/API-Search.html

What solution would you like?

searchApi allows pagination using search_after.

What alternatives have you considered?

Using pagination with size and from, but this has serious limitations.

[mstykow]

I found this Stackoverflow answer for elasticsearch-js saying that search_after should be part of the body. If that's true, perhaps some documentation would be helpful.

https://stackoverflow.com/questions/62716360/elasticsearch-search-after-with-nodejs

[nhtruong]

search_after is not a URL parameter of the search endpoint. It's actually a body parameter:

# GET shakespeare/_search
{
  "size": 10,
  "query": {
    "match": {
      "play_name": "Hamlet"
    }
  },
  "search_after": [ 1, "32635"],
  "sort": [
    { "speech_number": "asc" },
    { "_id": "asc" } 
  ]
}

I see that you mean though. It would be a lot more convenient if you can just pass search_after as a param of the client.search method instead of wedging it into the body. Unfortunately, we have a pattern where our API methods are consistent with the API endpoints they represent. Breaking away from that pattern is a change that will introduce edge cases, though it might be worth it? That's up for debate.

Anyhow, we're actually have a series of Guides in the work that includes sample code for search, that will hopeful help you avoid this pitfall in the future :)

[mstykow]

Gotcha. Looking at your example, it looks like search_after and sort should behave the same way which is adding to the confusion of why search_after cannot be passed as a param.

[nhtruong]

I think this request is more appropriate for the OS cluster repo. The client is just following the API defined by the cluster.