[bug]: search client is permanently broken if initialized without network connectivity

[giantslogik]

Description

If the search client is initialized when there is no network connectivity. It does not recover when connectivity is restored.

const searchClient = algoliasearch('algoliaAppId' , ‘algoliaApiKey’);

const sgAlgoliaIndex = searchClient.initIndex('indexname');

sgAlgoliaIndex.browseObjects…..

We re-execute the above code when connectivity is restored , however it continues to fail. We continue to get the same log output after connectivity is restored.

We get this error without a connection and then after internet connectivity is restored.

Client

Search

Version

^4.23.3

Relevant log output

{"name":"RetryError","message":"Unreachable hosts - your application id may be incorrect. If the error persists, contact support@algolia.com.","transporterStackTrace":[{"request":{"data":"{\"query\":\"\",\"filters\":\"REMOVED",\"attributesToRetrieve\":[\"REMOVED\"]}","headers":{"x-algolia-api-key":"*****","x-algolia-application-id":"REMOVED","content-type":"application/x-www-form-urlencoded"},"method":"POST","url":"https://REMOVED-dsn.algolia.net/1/indexes/superGroups/browse?x-algolia-agent=Algolia%20for%20JavaScript%20(4.23.3)%3B%20Browser","connectTimeout":1,"responseTimeout":2},"response":{"content":"The Internet connection appears to be offline.","status":0,"isTimedOut":false},"host":{"protocol":"https","url":"REMOVED-dsn.algolia.net","accept":1},"triesLeft":3},{"request":{"data":"{\"query\":\"\",\"filters\":\"REMOVED",\"attributesToRetrieve\":[\"REMOVED\"]}","headers":{"x-algolia-api-key":"*****","x-algolia-application-id":"REMOVED","content-type":"application/x-www-form-urlencoded"},"method":"POST","url":"https://REMOVED-3.algolianet.com/1/indexes/superGroups/browse?x-algolia-agent=Algolia%20for%20JavaScript%20(4.23.3)%3B%20Browser","connectTimeout":1,"responseTimeout":2},"response":{"content":"The Internet connection appears to be offline.","status":0,"isTimedOut":false},"host":{"protocol":"https","url":"REMOVED-3.algolianet.com","accept":3},"triesLeft":2},{"request":{"data":"{\"query\":\"\",\"filters\":\"REMOVED\",\"attributesToRetrieve\":[\"REMOVED\"]}","headers":{"x-algolia-api-key":"*****","x-algolia-application-id":"REMOVED","content-type":"application/x-www-form-urlencoded"},"method":"POST","url":"https://REMOVED-1.algolianet.com/1/indexes/superGroups/browse?x-algolia-agent=Algolia%20for%20JavaScript%20(4.23.3)%3B%20Browser","connectTimeout":1,"responseTimeout":2},"response":{"content":"The Internet connection appears to be offline.","status":0,"isTimedOut":false},"host":{"protocol":"https","url":"REMOVED-1.algolianet.com","accept":3},"triesLeft":1},{"request":{"data":"{\"query\":\"\",\"filters\":\"type.type:Public AND NOT monetized:true AND summary.totalPosts>0\",\"attributesToRetrieve\":[\"publicId\"]}","headers":{"x-algolia-api-key":"*****","x-algolia-application-id":"REMOVED","content-type":"application/x-www-form-urlencoded"},"method":"POST","url":"https://REMOVED-2.algolianet.com/1/indexes/superGroups/browse?x-algolia-agent=Algolia%20for%20JavaScript%20(4.23.3)%3B%20Browser","connectTimeout":1,"responseTimeout":2},"response":{"content":"The Internet connection appears to be offline.","status":0,"isTimedOut":false},"host":{"protocol":"https","url":"REMOVED-2.algolianet.com","accept":3},"triesLeft":0}]}

[shortcuts]

Hey, does this also happen with v5? I assume the requests are cached and answers automatically

[giantslogik]

Hey, does this also happen with v5? I assume the requests are cached and answers automatically
@shortcuts

Stopped using v5 due to issues we ran into. #1554
I can try latest v5 if it would work.

Why would failures be cached? I tried to reset the transporter & cache/ recreate the serchclient and initIndex again as well

sgAlgoliaIndex
        .browseObjects(..........)
        .then(() => {
       .......
        })
        .catch(e => {
          console.log('catch'  + JSON.stringify(e));
          searchClient.clearCache();
          searchClient.transporter.hostsCache.clear();
          searchClient.transporter.requestsCache.clear();
          searchClient.transporter.responsesCache.clear();
          tok = setTimeout(() => {
            ./*REtry the request*/
          }, 5000);
        });

[Fluf22]

Hey @giantslogik 👋🏻
It looks like from the logs you pasted in your question that the timeouts are too low:
...owser","connectTimeout":1,"responseTimeout":2},"resp...

I guess you're customising the timeout values somewhere else in the code? The values are in milliseconds, so could you try again after increasing them by a thousand factor, please?

[giantslogik]

@Fluf22

We are not setting a connection Timeout in the code. Based on your comment i modified the code to be:

searchClient = algoliasearch('algoliaAppId', 'algoliaApiKey', {
       timeouts: { connect: 10000, read: 10000, write: 10000 }
     });

The log remains the same:
"connectTimeout":1,"responseTimeout":2

[giantslogik]

@Fluf22 @shortcuts

The code creates multiple search clients with calls to searchClient = algoliasearch('algoliaAppId', 'algoliaApiKey'); .

Based on observation i suspect it returns the same immutable (and in this case broken) singleton instance each time.
Can you confirm that its not actually possible to create multiple clients with the library (4.23.3)

[Fluf22]

@giantslogik I'm really sorry for my mistake. It seems it's our serializer that reports a bad value for the timeouts... You can get rid of it.
Could you try again after setting cacheable: false in the requestOptions, please?

[giantslogik]

@Fluf22
cacheable: false did not change anything.

I also navigated to other parts of our (React Native) Application which use Algolia either with InstantSearch or directly with index.search. They too are broken and don't recover from the initial failure.

[Fluf22]

Could you please try disabling the cache: https://www.algolia.com/doc/guides/building-search-ui/going-further/improve-performance/js/#disable-the-cache ?
I'm trying to rule out if it's a cache issue, or if there is something else going on

[giantslogik]

@Fluf22 disabling the cache did not help. (same error)

However note that our application calls searchClient = algoliasearch('algoliaAppId', 'algoliaApiKey') at multiple locations in the code. If the search client internally uses a singleton like mechanism, its possible the AlgoliaSearchOptions arent being set correctly. i.e. i disabled cache for only the relavent search client, other searchclient instances were initialized without the options.

[giantslogik]

@shortcuts @Fluf22 I updated algoliasearch to the latest v5.12.0 . Same issue.

const searchClienti = algoliasearch('algoliaAppId') ,'algoliaApiKey');

 searchClienti
        .browseObjects({
          indexName: 'name',
          browseParams: {
            query: '',
            filters: 'REMOVED',
            attributesToRetrieve: ['REMOVED']
          },
          aggregator: batch => {
            hits.push(...batch.hits);
          }
        })
        .then(() => {
          /*REMOVED*/
        })
        .catch(e => {
          console.log('catch' + JSON.stringify(e));
          tok = setTimeout(() => {
            /*RETRY*/
          }, 5000);
        });

The error looks the same only the library version is different:

{"name":"RetryError","stackTrace":[{"request":{"data":"{\"query\":\"\",\"filters\":\"REMOVED\",\"attributesToRetrieve\":[\"REMOVED\"]}","headers":{"accept":"application/json","content-type":"text/plain"},"method":"POST","url":"https://REMOVED-dsn.algolia.net/1/indexes/superGroups/browse?x-algolia-agent=Algolia%20for%20JavaScript%20(5.12.0)%3B%20Search%20(5.12.0)%3B%20Browser&x-algolia-api-key=***&x-algolia-application-id=REMOVED","connectTimeout":1000,"responseTimeout":2000},"response":{"content":"The Internet connection appears to be offline.","status":0,"isTimedOut":false},"host":{"url":"REMOVED-dsn.algolia.net","accept":"read","protocol":"https","status":"up","lastUpdate":1730886699846},"triesLeft":3},{"request":{"data":"{\"query\":\"\",\"filters\":\"REMOVED\",\"attributesToRetrieve\":[\"REMOVED\"]}","headers":{"accept":"application/json","content-type":"text/plain"},"method":"POST","url":"https://REMOVED-2.algolianet.com/1/indexes/superGroups/browse?x-algolia-agent=Algolia%20for%20JavaScript%20(5.12.0)%3B%20Search%20(5.12.0)%3B%20Browser&x-algolia-api-key=***&x-algolia-application-id=REMOVED","connectTimeout":1000,"responseTimeout":2000},"response":{"content":"The Internet connection appears to be offline.","status":0,"isTimedOut":false},"host":{"url":"REMOVED-2.algolianet.com","accept":"readWrite","protocol":"https","status":"up","lastUpdate":1730886699846},"triesLeft":2},{"request":{"data":"{\"query\":\"\",\"filters\":\"REMOVED\",\"attributesToRetrieve\":[\"REMOVED\"]}","headers":{"accept":"application/json","content-type":"text/plain"},"method":"POST","url":"https://REMOVED-1.algolianet.com/1/indexes/superGroups/browse?x-algolia-agent=Algolia%20for%20JavaScript%20(5.12.0)%3B%20Search%20(5.12.0)%3B%20Browser&x-algolia-api-key=***&x-algolia-application-id=REMOVED","connectTimeout":1000,"responseTimeout":2000},"response":{"content":"The Internet connection appears to be offline.","status":0,"isTimedOut":false},"host":{"url":"REMOVED-1.algolianet.com","accept":"readWrite","protocol":"https","status":"up","lastUpdate":1730886699846},"triesLeft":1},{"request":{"data":"{\"query\":\"\",\"filters\":\"REMOVED\",\"attributesToRetrieve\":[\"REMOVED\"]}","headers":{"accept":"application/json","content-type":"text/plain"},"method":"POST","url":"https://REMOVED-3.algolianet.com/1/indexes/superGroups/browse?x-algolia-agent=Algolia%20for%20JavaScript%20(5.12.0)%3B%20Search%20(5.12.0)%3B%20Browser&x-algolia-api-key=***&x-algolia-application-id=REMOVED","connectTimeout":1000,"responseTimeout":2000},"response":{"content":"The Internet connection appears to be offline.","status":0,"isTimedOut":false},"host":{"url":"REMOVED-3.algolianet.com","accept":"readWrite","protocol":"https","status":"up","lastUpdate":1730886699846},"triesLeft":0}]}

[Fluf22]

I edited your comment and deleted the first version in the history because there was still some api keys visible in it.
Just saying, if you want to rotate them for security reasons.

Let me setup a quick repro and I'll get back to you

[Fluf22]

@giantslogik I think this issue is not related to the API client...
Here is a quick example I setup to try and reproduce your issue: https://stackblitz.com/edit/sb1-edtzzj

When I cut the network, I do get similar logs than those you shared above:

catch{"name":"RetryError","stackTrace":[{"request":{"data":"{\"query\":\"\"}","headers":{"accept":"application/json","content-type":"text/plain","x-algolia-api-key":"*****","x-algolia-application-id":"latency"},"method":"POST","url":"https://latency-dsn.algolia.net/1/indexes/algoflix/query?x-algolia-agent=Algolia%20for%20JavaScript%20(5.12.0)%3B%20Search%20(5.12.0)%3B%20Node.js%20(18.20.3)","connectTimeout":2000,"responseTimeout":5000},"response":{"status":0,"content":"socket hang up","isTimedOut":false},"host":{"url":"latency-dsn.algolia.net","accept":"read","protocol":"https","status":"up","lastUpdate":1730890748299},"triesLeft":3},{"request":{"data":"{\"query\":\"\"}","headers":{"accept":"application/json","content-type":"text/plain","x-algolia-api-key":"*****","x-algolia-application-id":"latency"},"method":"POST","url":"https://latency-1.algolianet.com/1/indexes/algoflix/query?x-algolia-agent=Algolia%20for%20JavaScript%20(5.12.0)%3B%20Search%20(5.12.0)%3B%20Node.js%20(18.20.3)","connectTimeout":2000,"responseTimeout":5000},"response":{"status":0,"content":"socket hang up","isTimedOut":false},"host":{"url":"latency-1.algolianet.com","accept":"readWrite","protocol":"https","status":"up","lastUpdate":1730890748299},"triesLeft":2},{"request":{"data":"{\"query\":\"\"}","headers":{"accept":"application/json","content-type":"text/plain","x-algolia-api-key":"*****","x-algolia-application-id":"latency"},"method":"POST","url":"https://latency-3.algolianet.com/1/indexes/algoflix/query?x-algolia-agent=Algolia%20for%20JavaScript%20(5.12.0)%3B%20Search%20(5.12.0)%3B%20Node.js%20(18.20.3)","connectTimeout":2000,"responseTimeout":5000},"response":{"status":0,"content":"socket hang up","isTimedOut":false},"host":{"url":"latency-3.algolianet.com","accept":"readWrite","protocol":"https","status":"up","lastUpdate":1730890748299},"triesLeft":1},{"request":{"data":"{\"query\":\"\"}","headers":{"accept":"application/json","content-type":"text/plain","x-algolia-api-key":"*****","x-algolia-application-id":"latency"},"method":"POST","url":"https://latency-2.algolianet.com/1/indexes/algoflix/query?x-algolia-agent=Algolia%20for%20JavaScript%20(5.12.0)%3B%20Search%20(5.12.0)%3B%20Node.js%20(18.20.3)","connectTimeout":2000,"responseTimeout":5000},"response":{"status":0,"content":"socket hang up","isTimedOut":false},"host":{"url":"latency-2.algolianet.com","accept":"readWrite","protocol":"https","status":"up","lastUpdate":1730890748299},"triesLeft":0}]}

Then, I quickly enable back the network, before the second call in the catch block triggers, and I get the expected response.

Maybe your issue is specific to the react-native setup? Could you try to setup a minimal code example that repro the issue, please?

[giantslogik]

@Fluf22 I've confirmed that the network is working and other (non-algolia) APIs we use work while Algoia client continues to return "The Internet connection appears to be offline." . While it may be React native related, its related to whatever networking / caching the client uses under the hood.

I will try and modify the stackblitz you shared.

[Haroenv]

I wonder if this is related to the hostsCache? Maybe all hosts are marked as down and nothing is retried? Have you tried passing nullCache to the hostsCache option?

[giantslogik]

@Haroenv can confirm its not related to hostsCache.

I'm leaning towards this being a react native + ios simulator issue. The algolia client recovers when testing using the mac os network link conditioner. It does not recover when disconnecting/reconnecting the mac os wifi of the machine running the simulator.

Will update this issue after further testing.