Looking for a proper way to handle connection errors.

[barca-reddit]

Hello folks, thanks for developing and maintaining this library. I have a question, or two questions actually if you don't mind.

Right now I am running a bot that is connected to a Twitter stream, which listens for tweets from an array of users, and then I do some fancy stuff with it. The problem is, every once in a while, Twitter throws a 429 error at me, maybe once every 24-48 hrs, and as a result of that I get an unhandledRejection error, and then the Stream no longer works without manually restarting the bot.

app[worker.1]: Error: Request failed with code 429
app[worker.1]: at RequestHandlerHelper.createResponseError (/app/node_modules/twitter-api-v2/dist/client-mixins/request-handler.helper.js:73:16)
app[worker.1]: at IncomingMessage.<anonymous> (/app/node_modules/twitter-api-v2/dist/client-mixins/request-handler.helper.js:114:33)
app[worker.1]: at IncomingMessage.emit (node:events:402:35)
app[worker.1]: at endReadableNT (node:internal/streams/readable:1343:12)
app[worker.1]: at processTicksAndRejections (node:internal/process/task_queues:83:21)

Here is a simplified version of my code:

class _Twitter {
    constructor({ streamName }) {
        this.streamName = streamName;
    }

    async initSources() {
        const sourceList = [
            'Barca_Buzz', 'barcacentre', 'BarcaTimes', 'BarcaUniversal', 'FCBarcelona',
            'FCBarcelonaFl', 'GSpanishFN', 'infosfcb', 'LaSenyera', 'ReshadRahman_',
        ];

        const streamRules = await this.client.v2.streamRules();

        // Cleanup all existing rules upon initiating
        if (streamRules?.data?.length > 0) {
            await this.client.v2.updateStreamRules({
                delete: {
                    ids: streamRules.data.map(rule => rule.id),
                },
            });
        }

        await this.client.v2.updateStreamRules({
            add: sourceList.map(source => ({
                value: `(from:${source}) -is:retweet -is:reply`,
                tag: source.name
            }))
        });
    }

    async stream() {
        try {
            this.client = new TwitterApi(TwitterBearerToken);
            await this.initSources();

            this.stream = await this.client.v2.searchStream(
                {
                    "tweet.fields": ['id', 'text', 'created_at', 'entities'],
                    "user.fields": ['username', 'name', 'profile_image_url'],
                    'media.fields': ['preview_image_url', 'url', 'width', 'height', 'type'],
                    'expansions': ['author_id', 'attachments.media_keys']
                }
            );

            this.stream.autoReconnect = true;
            this.stream.autoReconnectRetries = 999;

            // Listen for all possible events (for debugging purposes);

            this.stream.on(ETwitterStreamEvent.Data, async (data) => {
                console.log(`Event: Data (${data.includes.users[0].username})\n---`);
                // do something with the tweet here
            });

            this.stream.on(ETwitterStreamEvent.ConnectionClosed, async (data) => {
                console.log(`Event: ConnectionClosed${data ? `\n${JSON.stringify(data, null, 4)}\n` : '\n'}---`);
            });
            this.stream.on(ETwitterStreamEvent.ConnectionError, async (data) => {
                console.log(`Event: ConnectionError${data ? `\n${JSON.stringify(data, null, 4)}\n` : '\n'}---`);
            });
            this.stream.on(ETwitterStreamEvent.ConnectionLost, async (data) => {
                console.log(`Event: ConnectionLost${data ? `\n${JSON.stringify(data, null, 4)}\n` : '\n'}---`);
            });
            this.stream.on(ETwitterStreamEvent.DataError, async (data) => {
                console.log(`Event: DataError${data ? `\n${JSON.stringify(data, null, 4)}\n` : '\n'}---`);
            });
            this.stream.on(ETwitterStreamEvent.DataKeepAlive, async (data) => {
                console.log(`Event: DataKeepAlive${data ? `\n${JSON.stringify(data, null, 4)}\n` : '\n'}---`);
            });
            this.stream.on(ETwitterStreamEvent.Error, async (data) => {
                console.log(`Event: Error${data ? `\n${JSON.stringify(data, null, 4)}\n` : '\n'}---`);
            });
            this.stream.on(ETwitterStreamEvent.ReconnectAttempt, async (data) => {
                console.log(`Event: ReconnectAttempt${data ? `\n${JSON.stringify(data, null, 4)}\n` : '\n'}---`);
            });
            this.stream.on(ETwitterStreamEvent.ReconnectLimitExceeded, async (data) => {
                console.log(`Event: ReconnectLimitExceeded${data ? `\n${JSON.stringify(data, null, 4)}\n` : '\n'}---`);
            });
            this.stream.on(ETwitterStreamEvent.Reconnected, async (data) => {
                console.log(`Event: Reconnected${data ? `\n${JSON.stringify(data, null, 4)}\n` : '\n'}---`);
            });
            this.stream.on(ETwitterStreamEvent.TweetParseError, async (data) => {
                console.log(`Event: TweetParseError${data ? `\n${JSON.stringify(data, null, 4)}\n` : '\n'}---`);
            });

        } catch (error) {
            console.log(this.streamName, 'error');
            console.log(error);

            if (error.code = 429) {
                await this.stream.reconnect();
            }
        }
    }
}

const Twitter1 = new _Twitter({ streamName: 'Stream 1' });

(async () => {
    await Twitter1.stream();
})();

So as you can see, I am creating a _Twitter class, and then creating an instance of that class to connect to the Streaming API. This all works fine, except that only the Data and DataKeepAlive events are emitting properly, and the rest goes into my catch block.

It also appears that autoReconnect and autoReconnectRetries are not a properties of this.stream, and reconnect() is not a valid method either. I think the problem is that I have to use and implement the TweetStream into my code, but I am not sure how to exactly, and I can't find documentation for this either.

In any case, I don't want to make this too long, so I would just like to ask if you can point me in the right direction, such as providing a very quick example if that is possible.

Thank you for your time.

EDIT: It appears that this.stream is in fact a TweetStream object, there was just an error with my test script.

I would still like to ask, is this the proper way to implement autoReconnect? The actual code on my server does not have this try catch block, because I just assumed it would reconnect automatically on any kind of connection error. I'm not sure how to test this properly because Twitter only allows for 1 concurrent stream.

[alkihis]

Hi!
Thank you for your detailed example.
Your example is correctly written, except for autoReconnectRetries that you can set to Infinity if you don't want a limit.

Auto-reconnect should work in your case, but it does not handle the first initial connection, that is a classic request.
It's why the TweetStream is obtained through a Promise: it await for request acception by Twitter before resolving.

You already handle possible errors (thrown by awaits), that can only occur before the TweetStream is created (after this.stream affectation, there is no await, and errors thrown in event handlers aren't catch by try-catch block).

So in your catch block, this.stream is always undefined.
You can simply re-start your method inside the catch block (after a few seconds?):

if (error.code === 429) {
  // restart stream
  this.stream();
}

[barca-reddit]

Auto-reconnect should work in your case, but it does not handle the first initial connection, that is a classic request. It's why the TweetStream is obtained through a Promise: it await for request acception by Twitter before resolving.

Allow me to clarify something. I became aware of the fact that my example above is failing, and in addition to that I am also now aware that await this.client.v2.searchStream(...) returns a promise (TweetStream), that needs to connect successfully at least once (upon start), and because Twitter only allows for one concurrent stream, my demo was failing immediately, thus I was getting undefined inside the catch block and as such .reconnect() was not working. All of that is clear to me now, apologies for the poor code.

With all of that in mind, before I wrote this reply here, I had production code with autoReconnect enabled on my server, and one which successfully connects to a Twitter stream. But as mentioned, it was failing to reconnect when an error occasionally occurs and it required a manual restart. Here is the full code without any edits:

const { ETwitterStreamEvent, TweetStream, TwitterApi, ETwitterApiError } = require('twitter-api-v2');
const { MessageActionRow, MessageButton } = require('discord.js');
const TwitterSources = require('../models/twitter-sources');
const Discord = require('../util/discord');

class Twitter {
    stripAt(handle) {
        return handle.replace(/^@/, '');
    }

    stripShorteners(text) {
        return text.replace(/\shttps:\/\/t\.co\/[\S]{5,}$/gmi, '');
    }

    async getSourceRuleId(handle) {
        const sourceList = await this.client.v2.streamRules();
        console.log(sourceList);
        return sourceList.data.find(source => source.tag.toLowerCase() === handle.toLowerCase()).id;
    }

    async listSources() {
        const sourceList = await this.client.v2.streamRules();
        if (sourceList.data) {
            return sourceList.data
                .map(source => `\`@${source.tag}\``)
                .sort((a, b) => a.localeCompare(b))
                .join(', ');
        }
        else {
            return 'No entries found.';
        }
    }

    async addSource(handle) {
        handle = this.stripAt(handle);

        await TwitterSources.findOneAndUpdate(
            { name: new RegExp(handle, 'i') },
            { name: handle },
            { upsert: true, runValidators: true }
        );
        await this.client.v2.updateStreamRules({
            add: [
                {
                    value: `(from:${handle}) -is:retweet -is:reply`,
                    tag: handle
                }
            ]
        });

        return true;
    }

    async removeSource(handle) {
        handle = this.stripAt(handle);

        await TwitterSources.findOneAndDelete({ name: new RegExp(handle, 'i') });
        await this.client.v2.updateStreamRules({
            delete: { ids: [await this.getSourceRuleId(handle)] }
        });

        return true;
    }

    async initSources() {
        const sourceList = await TwitterSources.find({}).lean();
        const streamRules = await this.client.v2.streamRules();

        if (streamRules?.data?.length > 0) {
            await this.client.v2.updateStreamRules({
                delete: {
                    ids: streamRules.data.map(rule => rule.id),
                },
            });
        }

        await this.client.v2.updateStreamRules({
            add: sourceList.map(source => ({
                value: `(from:${source.name}) -is:retweet -is:reply`,
                tag: source.name
            }))
        });
    }

    embedTweet(data) {
        return ({
            description: data.text,
            url: data.url,
            color: 15329769,
            timestamp: data.time,
            footer: {
                text: data.authorName,
                icon_url: data.authorPhoto
            },
            ...data.thumbnail && {
                thumbnail: {
                    url: data.thumbnail
                }
            }
        })
    }

    embedActions(data) {
        return new MessageActionRow()
            .addComponents(
                new MessageButton({
                    label: 'Source',
                    style: 'LINK',
                    url: data.url,
                }),
                new MessageButton({
                    customId: 'POST_ON_REDDIT',
                    label: 'Post on Reddit',
                    style: 'SECONDARY',
                    emoji: '776887363901063189'
                })
            );
    }

    async stream() {
        this.client = new TwitterApi(process.env.TWITTER_BEARER_TOKEN);
        await this.initSources();

        this.stream = await this.client.v2.searchStream(
            {
                "tweet.fields": ['id', 'text', 'created_at', 'entities'],
                "user.fields": ['username', 'name', 'profile_image_url'],
                'media.fields': ['preview_image_url', 'url', 'width', 'height', 'type'],
                'expansions': ['author_id', 'attachments.media_keys']
            }
        );
        this.stream.autoReconnect = true;
        this.stream.autoReconnectRetries = 999;

        this.stream.on(ETwitterStreamEvent.Data, async (tweet) => {
            const author = tweet.includes.users.find(user => user.id === tweet.data.author_id);
            const tweetData = {
                id: tweet.data.id,
                url: `https://twitter.com/${author.username}/status/${tweet.data.id}`,
                authorName: author.username,
                authorPhoto: author.profile_image_url,
                text: this.stripShorteners(tweet.data.text),
                time: new Date(tweet.data.created_at).getTime(),
                ...tweet?.includes?.media?.[0]?.url && {
                    thumbnail: tweet.includes.media[0].url
                }
            }

            await Discord.channels.cache
                .get(process.env.TWITTER_CHANNEL_ID)
                .send({
                    embeds: [this.embedTweet(tweetData)],
                    components: [this.embedActions(tweetData)]
                });
        });
    }
}

const twitter = new Twitter();

module.exports = twitter;

Provided that there is no try catch block here, what is the purpose of autoReconnect if you have to intercept the error and use the reconnect() method anyway?

In any case, please don't consider responding to me a priority as I don't want to take too much of your time, I have working code now anyway so not a big deal.

[alkihis]

Hi,

NOTE: If the given script is a copy-paste, be careful of the naming of your variables. You name a method stream then you create a TweetStream instance, and store it into the stream property that overwrite the method on its first call!

try-catch block is only useful for .searchStream call.
You can re-write your stream (rename it to startStream?) method as it follows:

this.client = new TwitterApi(process.env.TWITTER_BEARER_TOKEN);
await this.initSources();

try {
  this.stream = await this.client.v2.searchStream(
            {
                "tweet.fields": ['id', 'text', 'created_at', 'entities'],
                "user.fields": ['username', 'name', 'profile_image_url'],
                'media.fields': ['preview_image_url', 'url', 'width', 'height', 'type'],
                'expansions': ['author_id', 'attachments.media_keys']
            }
        );
} catch (e) {
  // await and reconnect?
  this.startStream();
  return;
}

this.stream.autoReconnect = true;
this.stream.autoReconnectRetries = Infinity;

// then attach event handlers...

For handling connection and stream errors after the first call, you must use the following events:

• ReconnectAttempt when a reconnection is triggered (f.e. after a response error or request timeout) - this is sent before reconnection request is sent
• ReconnectError when a reconnection has failed
• ConnectionClosed when connection is closed (f.e. when reconnect retries has been exceeded, should not occur with Infinity)

Reconnections are fully automatic. If those events aren't triggered, then there is a problem elsewhere.

How do you know in production when there is a connection error? At which step of your script does these errors occur?

[barca-reddit]

Hello once again @alkihis, thank you for your reply.

NOTE: If the given script is a copy-paste, be careful of the naming of your variables. You name a method stream then you create a TweetStream instance, and store it into the stream property that overwrite the method on its first call!

This was indeed a copy-paste from the code running on the server. And you have correctly pointed out that the class method of stream() was a duplicate of the property this.stream. Huge oversight on my part, I have no idea how I managed to miss something so simple.

For handling connection and stream errors after the first call, you must use the following events:

* `ReconnectAttempt` when a reconnection is triggered (f.e. after a response error or request timeout) - this is sent **before** reconnection request is sent

* `ReconnectError` when a reconnection has failed

* `ConnectionClosed` when connection is closed (f.e. when reconnect retries has been exceeded, should not occur with Infinity)

Reconnections are fully automatic. If those events aren't triggered, then there is a problem elsewhere.

So as mentioned, because Twitter does not allow you to run two or more concurrent connections (stream), I temporarily turned off my bot, then ran a modified version of the code on my local machine, and I can confirm that everything works as intended.

Upon launching the script and then turning off my network connection to artificially simulate a disconnect, I've got the following output:

Event: ConnectionLost
---
Event: ReconnectAttempt
---
Event: Error
{
    "type": "reconnect error",
    "error": {}
}
---
Event: ReconnectAttempt 1
---
Event: Error
{
    "type": "reconnect error",
    "error": {}
}
---
Event: ReconnectAttempt 2
---
Event: Error
{
    "type": "reconnect error",
    "error": {}
}
---
Event: ReconnectAttempt 3
---
Event: Error
{
    "type": "reconnect error",
    "error": {}
}
---
Event: ReconnectAttempt 4
---
Event: Reconnected

The event handlers are emitting properly, and the connection automatically recovered upon turning on my network again.

---

Thanks so much once again for helping me, I appreciate it!

[barca-reddit]

Hello again, apologies for reopening this issue but I've ran into some unexpected problems, and I was hoping you can help me figure it out.

Basically I have made the suggested adjustments to my code since we last talked, and pushed them to my production server for the Discord bot that I am running. This morning I found out that the bot has crashed because it ran out of memory.

It appears that I've lost connection to the Twitter stream, and ETwitterStreamEvent.ReconnectAttempt and ETwitterStreamEvent.Error were continuously emitting in an extremely short interval of time until it the crash:

2021-10-14T19:39:41.636845+00:00 app[worker.1]: Twitter Event:ReconnectAttempt: null
2021-10-14T19:39:41.727128+00:00 app[worker.1]: Twitter Event:Error: {"type":"reconnect error","error":{}}
2021-10-14T19:39:41.728673+00:00 app[worker.1]: Twitter Event:ReconnectAttempt: null
2021-10-14T19:39:41.819645+00:00 app[worker.1]: Twitter Event:Error: {"type":"reconnect error","error":{}}
2021-10-14T19:39:41.821180+00:00 app[worker.1]: Twitter Event:ReconnectAttempt: null
2021-10-14T19:39:41.896057+00:00 app[worker.1]: Twitter Event:Error: {"type":"reconnect error","error":{}}
2021-10-14T19:39:41.897486+00:00 app[worker.1]: Twitter Event:ReconnectAttempt: null
2021-10-14T19:39:42.148890+00:00 app[worker.1]: Twitter Event:Error: {"type":"reconnect error","error":{}}

As you can see, these events were emitted mere milliseconds between each other, and I had thousands of those entries in my log files. Since then I have turned off the bot and I was able to reproduce this same behavior on my local machine. Here is a full sample that you can also run:

const { TwitterApi, ETwitterStreamEvent } = require('twitter-api-v2');

class Twitter {
    async startStream() {
        try {
            this.client = new TwitterApi(TWITTER_BEARER_TOKEN);
            this.stream = await this.client.v2.searchStream(
                {
                    "tweet.fields": ['id', 'text', 'created_at', 'entities'],
                    "user.fields": ['username', 'name', 'profile_image_url'],
                    'media.fields': ['preview_image_url', 'url', 'width', 'height', 'type'],
                    'expansions': ['author_id', 'attachments.media_keys']
                }
            );

            console.log('Connected to the Twitter stream');

            this.stream.autoReconnect = true;
            this.stream.autoReconnectRetries = Infinity;

            this.stream.on(ETwitterStreamEvent.Data, async (tweet) => {
                console.log(tweet);
            });
            this.stream.on(ETwitterStreamEvent.Error, async (error) => {
                console.log(`Twitter Event:Error: ${JSON.stringify(error)}`);
            });
            this.stream.on(ETwitterStreamEvent.ReconnectAttempt, async () => {
                console.log(`Twitter Event:ReconnectAttempt`);
            });
            this.stream.on(ETwitterStreamEvent.Reconnected, async () => {
                console.log(`Twitter Event:Reconnected`);
            });
            this.stream.on(ETwitterStreamEvent.DataKeepAlive, async () => {
                console.log(`Twitter Event:DataKeepAlive`);
            });
        } catch (error) {
            console.log(error);
        }
    }
}

const twitter = new Twitter();

(async () => {
    await twitter.startStream();
})();

To reproduce the disconnect and the subsequent flood of emits, I forcefully shut down the connections to 104.244.42.x which is the IP range of twitter.com and api.twitter.com, usually it's 104.244.42.140 but it may vary.

On Linux, after I initially and successfully connect to the stream I run this command to find out which port it's running on - sudo ss -t | grep 104.244 and then I used a tool called tcpkill to shut down the connection - sudo tcpkill -i XXX port YYY, where YYY is the name of my network, usually eth0 or eno1 and YYY is the port that is connected to the twitter stream.

After a few seconds, it kills the connection and the events start emitting like crazy.

Connected to the Twitter stream
Twitter Event:DataKeepAlive
Twitter Event:DataKeepAlive
Twitter Event:DataKeepAlive

#after I run tcpkill command:

Twitter Event:Error: {"type":"connection error","error":{"errno":-104,"code":"ECONNRESET","syscall":"read"}}
Twitter Event:ReconnectAttempt
Twitter Event:Error: {"type":"reconnect error","error":{}}
Twitter Event:ReconnectAttempt
Twitter Event:Error: {"type":"reconnect error","error":{}}
Twitter Event:ReconnectAttempt
Twitter Event:Error: {"type":"reconnect error","error":{}}
Twitter Event:ReconnectAttempt
Twitter Event:Error: {"type":"reconnect error","error":{}}
Twitter Event:ReconnectAttempt
Twitter Event:Error: {"type":"reconnect error","error":{}}
Twitter Event:ReconnectAttempt

I am not exactly sure why that's happening, because a few days ago I ran some tests before pushing the code on my server, and the disconnects were handled exactly as documented, or in particular:

.autoReconnectRetries: number / default 5 / If autoReconnect is true, maximum tries made until give up. Each try is spaced by min((attempt ** 2) * 1000, 25000) milliseconds.

But in the example I provided the retries occurred milliseconds apart from one another, and as far as I can tell, the stream didn't even manage to recover. Do you have any idea why this might be happening?

[alkihis]

Hi again,
Many thanks for the detailed explanation of what's happening.

I don't really know what's happening and why the reconnect timeout isn't awaited.
I will look into it as quick as possible following your specific test case and I'll come back here with more infos.

EDIT: After a quick review, I know what's happening and it's all my fault 😅
I'd tell you that you can use Infinity as autoReconnectRetries setting, but.. it causes a bug that hasn't fixed in master branch yet.
Using Infinity causes the try occurence count resolved as NaN ; as reconnect timeout is computed using current try occurence number, setTimeout is called with NaN as timeout milliseconds parameter (that must be resolved as 0, so delayed to next event loop turn).
This bug will be quickly resolved, but for now you can just re-switch to a value for autoReconnectRetries like 9999.

Very sorry for the inconvenience.

I'll update this issue when bugfix is available on npm.

[barca-reddit]

No inconvenience was caused at all, and if the report helped fix a bug than that makes me happy, so thank you very much for looking into it.

I have also tested this on my end using the same tcpkill method and can confirm that using an integer instead of Infinity fixes the issue. Will update my code accordingly and perhaps switch to Infinity again whenever an update is available.

I am not sure if I should close the issue or edit the title and leave it open, so whatever you suggest.

Anyway, thanks once again for helping out!

[alkihis]

Hello again,

1.6.1 update that contains the fix is now available on npm :)
It comes with another tiny new features that has been raised before: you can now create a stream instance that isn't auto-connected to Twitter immediately. Please consider this code sample, inspired by your example:

async startStream() {
  this.client = new TwitterApi(TWITTER_BEARER_TOKEN);
  this.stream = this.client.v2.searchStream(
    {
      "tweet.fields": ['id', 'text', 'created_at', 'entities'],
      "user.fields": ['username', 'name', 'profile_image_url'],
      'media.fields': ['preview_image_url', 'url', 'width', 'height', 'type'],
      'expansions': ['author_id', 'attachments.media_keys'],
      'autoConnect': false, // Note the autoConnect: false
    }
  );

  this.stream.on(ETwitterStreamEvent.Data, async (tweet) => {
    console.log(tweet);
  });
  this.stream.on(ETwitterStreamEvent.Error, async (error) => {
    console.log(`Twitter Event:Error: ${JSON.stringify(error)}`);
  });
  this.stream.on(ETwitterStreamEvent.ReconnectAttempt, async () => {
    console.log(`Twitter Event:ReconnectAttempt`);
  });
  this.stream.on(ETwitterStreamEvent.Reconnected, async () => {
    console.log(`Twitter Event:Reconnected`);
  });
  this.stream.on(ETwitterStreamEvent.DataKeepAlive, async () => {
    console.log(`Twitter Event:DataKeepAlive`);
  });
  this.stream.on(ETwitterStreamEvent.Connected, async () => {
    console.log('Connected to the Twitter stream');
  });

  try {
    // Options are directly given in .connect() method
    await this.stream.connect({ autoReconnect: true, autoReconnectRetries: Infinity });
  } catch (error) {
    console.log('Unable to establish the first connection. Auto-reconnect will be fired soon.');
    console.log(error);
  }
}

If you want to configure the retry timeout computed between each reconnect attempt, you can now overwrite nextRetryTimeout property of the TweetStream instance.
By default, this is set to tryOccurence => Math.min((tryOccurence ** 2) * 1000, 25000).

Keep me updated if the new version meets your expectations. :)

[barca-reddit]

Hey @alkihis how are you doing.

Really appreciate all these updates (1.16.1), and as a matter of fact, I have already been using and testing them for the last 36-48 hours because I ran into some more connection issues, although they might and probably are not directly related to your library these new tools and abilities came extremely handy to me. I might my issues in another comment, but for now I just wanted to let you know that I have tested this functionality you described above, and in one specific case autoReconnect: true doesn't work properly.

That specific case is an error 429:

ApiResponseError: Request failed with code 429
    at RequestHandlerHelper.createResponseError (/home/user/projects/barca_discord_bot/node_modules/twitter-api-v2/dist/client-mixins/request-handler.helper.js:70:16)
    at RequestHandlerHelper.onResponseEndHandler (/home/user/projects/barca_discord_bot/node_modules/twitter-api-v2/dist/client-mixins/request-handler.helper.js:110:25)
    at IncomingMessage.emit (node:events:406:35)
    at endReadableNT (node:internal/streams/readable:1343:12)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  error: true,
  type: 'response',
  code: 429,
  headers: {
   // some private header data removed here
    date: 'Wed, 20 Oct 2021 22:01:38 UTC',
    server: 'tsa_o',
    'content-type': 'application/json; charset=utf-8',
    'cache-control': 'no-cache, no-store, max-age=0',
    'content-length': '213',
    'x-access-level': 'read',
    'x-frame-options': 'SAMEORIGIN',
    'x-xss-protection': '0',
    'x-rate-limit-limit': '50',
    'x-rate-limit-reset': '1634767472',
    'content-disposition': 'attachment; filename=json.json',
    'x-content-type-options': 'nosniff',
    'x-rate-limit-remaining': '16',
    'strict-transport-security': 'max-age=631138519',
    connection: 'close'
  },
  rateLimit: { limit: 50, remaining: 16, reset: 1634767472 },
  data: {
    title: 'ConnectionException',
    detail: 'This stream is currently at the maximum allowed connection limit.',
    connection_issue: 'TooManyConnections',
    type: 'https://api.twitter.com/2/problems/streaming-connection'
  }
}

Here is a full reproducible script:

const { ETwitterStreamEvent, TwitterApi } = require('twitter-api-v2');
const TWITTER_BEARER_TOKEN = 'YOUR_TOKEN';

/*
    This is just a little trick to simiulate server enviornment,
    and keep the process running forever, otherwise the process exits
    after the script is executed.
*/

const server = require('http').createServer();
server.listen(12345);

class Twitter {
    async startStream() {
        try {
            if (!this.client) {
                this.client = new TwitterApi(TWITTER_BEARER_TOKEN);
            }

            this.stream = this.client.v2.searchStream(
                {
                    "tweet.fields": ['id', 'text', 'created_at', 'entities'],
                    "user.fields": ['username', 'name', 'profile_image_url'],
                    'media.fields': ['preview_image_url', 'url', 'width', 'height', 'type'],
                    'expansions': ['author_id', 'attachments.media_keys'],
                    'autoConnect': false
                }
            );

            this.stream.on(ETwitterStreamEvent.Data, async (data) => { console.log(data); });
            this.stream.on(ETwitterStreamEvent.Error, async (data) => {
                console.log('Event:Error', data);
            });
            this.stream.on(ETwitterStreamEvent.Connected, async () => { console.log('Event:Connected'); });
            this.stream.on(ETwitterStreamEvent.ConnectionLost, async () => { console.log('Event:ConnectionLost'); });
            this.stream.on(ETwitterStreamEvent.ConnectionError, async (data) => {
                console.log('Event:ConnectionError', data);
            });
            this.stream.on(ETwitterStreamEvent.TweetParseError, async (data) => {
                console.log('Event:TweetParseError', data);
            });
            this.stream.on(ETwitterStreamEvent.ConnectionClosed, async () => { console.log('Event:ConnectionClosed'); });
            this.stream.on(ETwitterStreamEvent.ReconnectAttempt, async (data) => {
                console.log('Event:ReconnectAttempt', data);
            });
            this.stream.on(ETwitterStreamEvent.ReconnectError, async (data) => {
                console.log('Event:ReconnectError', data);
            });
            this.stream.on(ETwitterStreamEvent.ReconnectLimitExceeded, async () => { console.log('Event:ReconnectLimitExceeded'); });
            this.stream.on(ETwitterStreamEvent.Reconnected, async () => { console.log('Event:Reconnected'); });
            this.stream.on(ETwitterStreamEvent.DataKeepAlive, async () => { console.log('Event:DataKeepAlive'); });

            try {
                await this.stream.connect({
                    autoReconnect: true,
                    autoReconnectRetries: Infinity,
                });
            } catch (error) {
                console.log('Unable to establish the first connection. Auto-reconnect will be fired soon.');
                console.log(error);
            }
        } catch (error) {
            console.log(error);
        }
    }
}

const twitter = new Twitter();

(async () => {
    await twitter.startStream();
})();

Is this intended behavior, a mistake on my part or possibly a bug? If it's a bug, you can force the TooManyConnections error by running this script in two separate terminals.

I have kinda managed to overcome this problem by implementing my own reconnect() method, called forceReconnect(), although I am not 100% certain this is the right way to do it.

class Twitter {
    async forceReconnect() {
        this.stream.destroy();
        await new Promise(resolve => setTimeout(() => resolve(), 30000)); // wait 30 seconds
        await this.startStream();
    }
    async startStream() {
    // ...
    try {
        await this.stream.connect({
            autoReconnect: true,
            autoReconnectRetries: Infinity,
        });
    } catch (error) {
        console.log('Unable to establish the first connection. Auto-reconnect will be fired soon.');
        console.log(error);
        await this.forceReconnect();
    }
}

By the way, the new nextRetryTimeout functionality is very useful. I thought of suggesting something like that but you're thinking ahead of me :) I will test it out once I manage to deal with these random Twitter disconnects.

Cheers!

[alkihis]

Hello again!

autoReconnect previously doesn't work with the initial connection, because of some tweaks.
With the new 1.6.3 update (yes, a new update again, sorry!), autoReconnect behaviour is now applied even for the initial connection. It means that you don't need to manually reconnect after a failed connect attempt.

It also means that .connect is a Promise that will never fail anymore on autoReconnect: true, because a possible error is catched and given to the makeAutoReconnectRetry stream chain, that will fire a new .ConnectError event (only when the initial connection fails), and .ReconnectError (as before) afterwards.

class Twitter {
    async startStream() {
    // await only the first connection to be failed or succeeded
    // you must log .on(.ConnectError) to know if the connection fails
    await this.stream.connect({
        autoReconnect: true,
        autoReconnectRetries: Infinity,
    });
}

[barca-reddit]

This is great, I just updated to the latest version, and to no surprise it does exactly as you described 👍 I will update my production server and start using this directly.

If I am not overstaying my welcome, I was hoping I could ask you about a connection error that I'm having with Twitter, but one which is probably not going to be directly related to node-twitter-api-v2, not caused by the library. All of that once I manage to collect enough data from the logs in a few days time (when it crashes again), and in case I am unable to figure it out on my own of course.

But in any case, thanks for all the updates and the effort you put into this!

[barca-reddit]

Hey again.

So something really strange happened over the course of the past 24 or so hours on the production server. Basically I am experiencing those ECONNRESET errors every so often, but this time one of the connections managed to "clone". In other words, I had one stream working properly and outputting tweets to a discord channel (which is what my bot does), while another stream was emitting errors and trying to reconnect simultaneously, only to be rejected by a TooManyConnections by Twitter. That went on since forever, until I woke up and restarted my bot.

I have started diligently logging every event and also noticed that at one point the Reconnected event emitted twice, and those events were all milliseconds apart from one another.

At the point I'm getting totally lost and kinda disheartened in what to do. But in any case, in case you decide to test this on your end, this is my production code, the relevant part of it at least.

const { ETwitterStreamEvent, TwitterApi } = require('twitter-api-v2');

class Twitter extends Commands {
    async initSources() {
        const sourceList = await TwitterSources.find({}).lean();
        const streamRules = await this.client.v2.streamRules();

        if (streamRules?.data?.length > 0) {
            await this.client.v2.updateStreamRules({
                delete: {
                    ids: streamRules.data.map(rule => rule.id),
                },
            });
        }

        await this.client.v2.updateStreamRules({
            add: sourceList.map(source => ({
                value: `(from:${source.name}) -is:retweet -is:reply`,
                tag: source.name
            }))
        });
    }

    async logToDb({ type, value, path, description }) {
        await Errors.create({
            ...value && { value: JSON.stringify(value, null, 4) },
            type: type,
            path: path,
            ...description && { description: description },
            clientState: this.client
        });

        // value: this.client.v2.getLastRateLimitStatus('https://api.twitter.com/2/tweets/search/stream')
    }

    async startStream() {
        try {
            if (!this.client) {
                this.client = TwitterClient;
            }
            await this.initSources();

            this.stream = this.client.v2.searchStream(
                {
                    "tweet.fields": ['id', 'text', 'created_at', 'entities'],
                    "user.fields": ['username', 'name', 'profile_image_url'],
                    'media.fields': ['preview_image_url', 'url', 'width', 'height', 'type'],
                    'expansions': ['author_id', 'attachments.media_keys'],
                    'autoConnect': false
                }
            );

            this.stream.on(ETwitterStreamEvent.Data, async (data) => {
                await this.handleTweet(data);
            });

            this.stream.on(ETwitterStreamEvent.DataError, async (data) => {
                await this.logToChannel({ msg: '**Twitter**:Event:❌ DataError' });
                await this.logToDb({
                    type: 'error',
                    value: data,
                    path: 'ETwitterStreamEvent.DataError',
                });
            });
            this.stream.on(ETwitterStreamEvent.Error, async (data) => {
                await this.logToChannel({ msg: '**Twitter**:Event:❌ Error' });
                await this.logToDb({
                    type: 'error',
                    value: data,
                    path: 'ETwitterStreamEvent.Error',
                });

                if (data?.error?.errno === -104 || data?.error?.code === 'ECONNRESET') {
                    await this.logToChannel({ msg: '**Twitter**:Event:❌❌ Error (ECONNRESET)' });
                    await this.logToDb({
                        type: 'error',
                        value: data,
                        path: 'ETwitterStreamEvent.Error',
                        description: 'ECONNRESET'
                    });
                }
            });
            this.stream.on(ETwitterStreamEvent.Connected, async () => {
                await this.logToChannel({ msg: '**Twitter**:Event:✅ Connected' });
                await this.logToDb({
                    type: 'log',
                    path: 'ETwitterStreamEvent.Connected',
                });
            });
            this.stream.on(ETwitterStreamEvent.ConnectError, async (data) => {
                await this.logToChannel({ msg: '**Twitter**:Event:❌ ConnectError' });
                await this.logToDb({
                    type: 'error',
                    ...data && { value: data },
                    path: 'ETwitterStreamEvent.ConnectError',
                });
            });
            this.stream.on(ETwitterStreamEvent.ConnectionLost, async () => {
                await this.logToChannel({ msg: '**Twitter**:Event:❌ ConnectionLost' });
                await this.logToDb({
                    type: 'log',
                    path: 'ETwitterStreamEvent.ConnectionLost',
                });
            });
            this.stream.on(ETwitterStreamEvent.ConnectionError, async (data) => {
                await this.logToChannel({ msg: '**Twitter**:Event:❌ ConnectionError' });
                await this.logToDb({
                    type: 'error',
                    value: data,
                    path: 'ETwitterStreamEvent.ConnectionError',
                });
            });
            this.stream.on(ETwitterStreamEvent.TweetParseError, async (data) => {
                await this.logToChannel({ msg: '**Twitter**:Event:❌ TweetParseError' });
                await this.logToDb({
                    type: 'error',
                    value: data,
                    path: 'ETwitterStreamEvent.TweetParseError',
                });
            });
            this.stream.on(ETwitterStreamEvent.ConnectionClosed, async () => {
                await this.logToChannel({ msg: '**Twitter**:Event:❌ ConnectionClosed' });
                await this.logToDb({
                    type: 'error',
                    path: 'ETwitterStreamEvent.ConnectionClosed',
                });
            });
            this.stream.on(ETwitterStreamEvent.ReconnectAttempt, async (data) => {
                await this.logToChannel({ msg: `**Twitter**:Event:❕ ReconnectAttempt (${data})` });
                await this.logToDb({
                    type: 'error',
                    value: data,
                    path: 'ETwitterStreamEvent.ReconnectAttempt',
                    description: 'retries'
                });
            });
            this.stream.on(ETwitterStreamEvent.ReconnectError, async (data) => {
                await this.logToChannel({ msg: `**Twitter**:Event:❌ ReconnectError (${data})` });
                await this.logToDb({
                    type: 'error',
                    value: data,
                    path: 'ETwitterStreamEvent.ReconnectError',
                    description: 'retries'
                });
            });
            this.stream.on(ETwitterStreamEvent.ReconnectLimitExceeded, async () => {
                await this.logToChannel({ msg: '**Twitter**:Event:❌ ReconnectLimitExceeded' });
                await this.logToDb({
                    type: 'error',
                    path: 'ETwitterStreamEvent.ReconnectLimitExceeded',
                });
            });
            this.stream.on(ETwitterStreamEvent.Reconnected, async () => {
                await this.logToChannel({ msg: '**Twitter**:✅ Event:Reconnected' });
                await this.logToDb({
                    type: 'log',
                    path: 'ETwitterStreamEvent.Reconnected',
                });
            });
            // this.stream.on(ETwitterStreamEvent.DataKeepAlive, async () => {
            //     await this.logToChannel({ msg: '**Twitter**:Event:DataKeepAlive' });
            // });

            try {
                await this.stream.connect({
                    autoReconnect: true,
                    autoReconnectRetries: Infinity,
                    nextRetryTimeout: () => 30000,
                });
            } catch (error) {
                console.log(error);
                await this.logToChannel({ msg: `**Twitter**:Event:❌ Twitter.stream.connect() Error` });
                await this.logToDb({
                    type: 'error',
                    value: error,
                    path: 'Twitter.stream.connect()',
                    description: 'inner block error'
                });

                // await this.recoverStream();
            }
        } catch (error) {
            console.log(error);
            await this.logToChannel({ msg: `**Twitter**:Event:❌ Twitter.startStream() Error` });
            await this.logToDb({
                type: 'error',
                value: error,
                path: 'Twitter.startStream()',
                description: 'outer block error'
            });
        }
    }
}

const twitter = new Twitter();

module.exports = twitter;

Overall I am not really sure what to think of this. Using version 1.6.1 I was in the process of dealing with a similar problem, where if an ECONNRESET occurs (on production server) then after exactly 42 replies, which is around 15 minutes (Twitter rate limits) the connection would recover and in the meantime it would keep throwing TooManyConnections error at you, but eventually recover.

Example error:

{
    "type": "reconnect error",
    "error": {
        "error": true,
        "type": "response",
        "code": 429,
        "headers": {
            // some private header data removed here
            "date": "Fri, 22 Oct 2021 12:46:37 UTC",
            "server": "tsa_b",
            "content-type": "application/json; charset=utf-8",
            "cache-control": "no-cache, no-store, max-age=0",
            "content-length": "213",
            "x-access-level": "read",
            "x-frame-options": "SAMEORIGIN",
            "x-xss-protection": "0",
            "x-rate-limit-limit": "50",
            "x-rate-limit-reset": "1634907396",
            "content-disposition": "attachment; filename=json.json",
            "x-content-type-options": "nosniff",
            "x-rate-limit-remaining": "39",
            "strict-transport-security": "max-age=631138519",
            "connection": "close"
        },
        "rateLimit": {
            "limit": 50,
            "remaining": 39,
            "reset": 1634907396
        },
        "data": {
            "title": "ConnectionException",
            "detail": "This stream is currently at the maximum allowed connection limit.",
            "connection_issue": "TooManyConnections",
            "type": "https://api.twitter.com/2/problems/streaming-connection"
        }
    },
    "message": "Reconnect error - 939 attempts made yet."
}

But I actually started logging the state of this.client every time the logToDb() method is called as you can see. And in there nothing indicates that I am reaching the these rate limits. Even less so immediately after an ECONNRESET occurs. So most likely I suspect what is happening here is that the previous connection is not getting properly destroyed before another one is being initiated.

I saw in your source code that you are using the req.destroy() method, but I don't have much experience with it to be able to tell if this has any relevance or significance.

Not sure what else to add, but in case you're interested in this or it would be of any help to you I can for example give you access to the database where the logs are contained. Not for the purpose of asking you to fix my problem, only in case this is a bug with the library and it will be helpful for you to see this log data, since this error seems to be random and hard to reproduce. In the meantime I will probably try to connect to the Twitter API without a library and see what error I can get and debug that way, if I manage to find my way around.

Cheers!

[alkihis]

Hi again,

I'm so sorry for all the encountered problems.
Unfortunately, I haven't the infrastructure available to test during a long time streaming connection, so it's quite difficult for me to debug.
Thanks a lot for your patience, and all the reported logging/code samples.

I really don't think that your code is causing the issue.

I'm currently trying to "lock" the instance from making two reconnection requests concurrently somehow.
It's really strange that the process that triggers a reconnection is only driven by three "hotpoints":

• When there's an error during the request: error event of this.req, error event of this.res, and close event of this.res
• When timeout is triggered, usually after two minutes
• When a reconnection attempt fails, after the timeout

This process is handled by .onConnectionError of TweetStream. It starts by unbinding registered timeouts, preventing keep alive timeout AND reconnection timeout to be triggered during a reconnection.

Then, it closes the request, first by unbinding all events on this.req and this.res (so, prevent another hypothetical error event to start reconnection process) then by calling this.req.destroy().

I'll include a new private property on TweetStream, connectionProcessRunning, that will prevent .reconnect() to be called simultaneously multiple times, but it is barely even a band-aid. This fix will be pushed soon on npm (as 1.6.4 I suppose).

If you have suggestions, I'll be happy to get some help on this :)

[barca-reddit]

I'm so sorry for all the encountered problems.

No issue whatsoever, you are contributing your free time to develop this, I should be thankful more than anything. It's a free product with no warranty.

Unfortunately, I haven't the infrastructure available to test during a long time streaming connection, so it's quite difficult for me to debug.

There is a free tier on Heroku which you can run 24/7 and the setup is quite trivial. They only require a valid credit card, but no payments necessary. This is in fact the same server I am using for this community bot.

As for the rest of your comment, I really don't feel competent enough to comment about it because I have limited experience with Node's req and I haven't really studied the source code here, was just taking a look to see what is going on.

In any case, this same error occurred again just around 30 min ago and I was around and being able to observe it. Not exactly sure if this is of any help to you, but here are all the events as logged out in my database by the logToDb() method. I have cleaned up the output a little bit for clarity, and removed the clientState (this.client) key because it didn't seem to contain any useful information.

The events are sorted first to last, or in other words the first event in the list below is the first error that occurred.

[
    {
        type: "error",
        value: '{ "code": "ECONNRESET" }',
        path: "ETwitterStreamEvent.ConnectionError",
        date: "2021-10-22T23:16:32.416+03:00",
    },
    {
        type: "error",
        value: '{ "type": "connection error", "error": { "code": "ECONNRESET" }, "message": "Connection lost or closed by Twitter." }',
        path: "ETwitterStreamEvent.Error",
        date: "2021-10-22T23:16:32.504+03:00",
    },
    {
        description: "retries",
        type: "error",
        path: "ETwitterStreamEvent.ReconnectAttempt",
        date: "2021-10-22T23:16:32.622+03:00"
    },
    {
        type: "error",
        value: "{}",
        path: "ETwitterStreamEvent.ConnectionError",
        date: "2021-10-22T23:16:32.755+03:00"
    },
    {
        type: "error",
        value: '{ "type": "connection error", "error": {}, "message": "Connection lost or closed by Twitter." }',
        path: "ETwitterStreamEvent.Error",
        date: "2021-10-22T23:16:32.885+03:00",
    },
    {
        description: "retries",
        type: "error",
        path: "ETwitterStreamEvent.ReconnectAttempt",
        date: "2021-10-22T23:16:37.906+03:00"
    },
    {
        type: "log",
        path: "ETwitterStreamEvent.Reconnected",
        date: "2021-10-22T23:16:38.086+03:00"
    },
    {
        type: "log",
        path: "ETwitterStreamEvent.Reconnected",
        date: "2021-10-22T23:16:38.198+03:00"
    },
    {
        description: "ECONNRESET",
        type: "error",
        value: '{ "type": "connection error", "error": { "code": "ECONNRESET" }, "message": "Connection lost or closed by Twitter." }',
        path: "ETwitterStreamEvent.Error",
        date: "2021-10-22T23:16:38.374+03:00",
    },
    {
        type: "error",
        value: '{ "errors": [ { "title": "ConnectionException", "detail": "This stream is currently at the maximum allowed connection limit.", "connection_issue": "TooManyConnections", "type": "https://api.twitter.com/2/problems/streaming-connection" } ] }',
        path: "ETwitterStreamEvent.DataError",
        date: "2021-10-22T23:16:52.635+03:00",
    },
    {
        type: "error",
        value: '{ "type": "data twitter error", "error": { "errors": [ { "title": "ConnectionException", "detail": "This stream is currently at the maximum allowed connection limit.", "connection_issue": "TooManyConnections", "type": "https://api.twitter.com/2/problems/streaming-connection" } ] }, "message": "Twitter sent a payload that is detected as an error payload." }',
        path: "ETwitterStreamEvent.Error",
        date: "2021-10-22T23:16:52.772+03:00",
    },
    {
        type: "error",
        value: "{}",
        path: "ETwitterStreamEvent.ConnectionError",
        date: "2021-10-22T23:16:52.927+03:00"
    },
    {
        type: "error",
        value: '{ "type": "connection error", "error": {}, "message": "Connection lost or closed by Twitter." }',
        path: "ETwitterStreamEvent.Error",
        date: "2021-10-22T23:16:53.038+03:00",
    },
    {
        description: "retries",
        type: "error",
        path: "ETwitterStreamEvent.ReconnectAttempt",
        date: "2021-10-22T23:16:53.571+03:00"
    },
    {
        description: "retries",
        type: "error",
        path: "ETwitterStreamEvent.ReconnectError",
        date: "2021-10-22T23:16:58.594+03:00"
    },
    {
        type: "error",
        value: '{ "type": "reconnect error", "error": { "error": true, "type": "response", "code": 429, "headers": { "date": "Fri, 22 Oct 2021 20:16:52 UTC", "server": "tsa_b", "content-type": "application/json; charset=utf-8", "cache-control": "no-cache, no-store, max-age=0", "content-length": "213", "x-access-level": "read", "x-frame-options": "SAMEORIGIN", "x-xss-protection": "0", "x-rate-limit-limit": "50", "x-rate-limit-reset": "1634934692", "content-disposition": "attachment; filename=json.json", "x-content-type-options": "nosniff", "x-rate-limit-remaining": "47", "strict-transport-security": "max-age=631138519", "x-response-time": "87", "connection": "close" }, "rateLimit": { "limit": 50, "remaining": 47, "reset": 1634934692 }, "data": { "title": "ConnectionException", "detail": "This stream is currently at the maximum allowed connection limit.", "connection_issue": "TooManyConnections", "type": "https://api.twitter.com/2/problems/streaming-connection" } }, "message": "Reconnect error - 1 attempts made yet." }',
        path: "ETwitterStreamEvent.Error",
        date: "2021-10-22T23:16:58.796+03:00",
    },
    {
        description: "retries",
        type: "error",
        value: "1",
        path: "ETwitterStreamEvent.ReconnectAttempt",
        date: "2021-10-22T23:17:22.739+03:00",
    },
    {
        description: "retries",
        type: "error",
        value: "1",
        path: "ETwitterStreamEvent.ReconnectError",
        date: "2021-10-22T23:17:22.847+03:00",
    },
    {
        type: "error",
        value: '{ "type": "reconnect error", "error": { "error": true, "type": "response", "code": 429, "headers": { "date": "Fri, 22 Oct 2021 20:17:22 UTC", "server": "tsa_b", "content-type": "application/json; charset=utf-8", "cache-control": "no-cache, no-store, max-age=0", "content-length": "213", "x-access-level": "read", "x-frame-options": "SAMEORIGIN", "x-xss-protection": "0", "x-rate-limit-limit": "50", "x-rate-limit-reset": "1634934692", "content-disposition": "attachment; filename=json.json", "x-content-type-options": "nosniff", "x-rate-limit-remaining": "46", "strict-transport-security": "max-age=631138519", "x-response-time": "86", "connection": "close" }, "rateLimit": { "limit": 50, "remaining": 46, "reset": 1634934692 }, "data": { "title": "ConnectionException", "detail": "This stream is currently at the maximum allowed connection limit.", "connection_issue": "TooManyConnections", "type": "https://api.twitter.com/2/problems/streaming-connection" } }, "message": "Reconnect error - 2 attempts made yet." }',
        path: "ETwitterStreamEvent.Error",
        date: "2021-10-22T23:17:22.983+03:00",
    },
    {
        description: "retries",
        type: "error",
        value: "2",
        path: "ETwitterStreamEvent.ReconnectAttempt",
        date: "2021-10-22T23:17:52.891+03:00",
    },
    {
        description: "retries",
        type: "error",
        value: "2",
        path: "ETwitterStreamEvent.ReconnectError",
        date: "2021-10-22T23:17:53.003+03:00",
    },
    {
        type: "error",
        value: '{ "type": "reconnect error", "error": { "error": true, "type": "response", "code": 429, "headers": { "date": "Fri, 22 Oct 2021 20:17:52 UTC", "server": "tsa_b", "content-type": "application/json; charset=utf-8", "cache-control": "no-cache, no-store, max-age=0", "content-length": "213", "x-access-level": "read", "x-frame-options": "SAMEORIGIN", "x-xss-protection": "0", "x-rate-limit-limit": "50", "x-rate-limit-reset": "1634934692", "content-disposition": "attachment; filename=json.json", "x-content-type-options": "nosniff", "x-rate-limit-remaining": "45", "strict-transport-security": "max-age=631138519", "x-response-time": "85", "connection": "close" }, "rateLimit": { "limit": 50, "remaining": 45, "reset": 1634934692 }, "data": { "title": "ConnectionException", "detail": "This stream is currently at the maximum allowed connection limit.", "connection_issue": "TooManyConnections", "type": "https://api.twitter.com/2/problems/streaming-connection" } }, "message": "Reconnect error - 3 attempts made yet." }',
        path: "ETwitterStreamEvent.Error",
        date: "2021-10-22T23:17:53.101+03:00",
    },
    {
        description: "retries",
        type: "error",
        value: "3",
        path: "ETwitterStreamEvent.ReconnectAttempt",
        date: "2021-10-22T23:18:23.106+03:00",
    },
    {
        description: "retries",
        type: "error",
        value: "3",
        path: "ETwitterStreamEvent.ReconnectError",
        date: "2021-10-22T23:18:23.218+03:00",
    },
    {
        type: "error",
        value: '{ "type": "reconnect error", "error": { "error": true, "type": "response", "code": 429, "headers": { "cache-control": "no-cache, no-store, max-age=0", "connection": "close", "content-disposition": "attachment; filename=json.json", "content-length": "213", "content-type": "application/json; charset=utf-8", "date": "Fri, 22 Oct 2021 20:18:23 GMT", "server": "tsa_b", "strict-transport-security": "max-age=631138519", "x-access-level": "read", "x-content-type-options": "nosniff", "x-frame-options": "SAMEORIGIN", "x-rate-limit-limit": "50", "x-rate-limit-remaining": "44", "x-rate-limit-reset": "1634934692", "x-response-time": "100", "x-xss-protection": "0" }, "rateLimit": { "limit": 50, "remaining": 44, "reset": 1634934692 }, "data": { "title": "ConnectionException", "detail": "This stream is currently at the maximum allowed connection limit.", "connection_issue": "TooManyConnections", "type": "https://api.twitter.com/2/problems/streaming-connection" } }, "message": "Reconnect error - 4 attempts made yet." }',
        path: "ETwitterStreamEvent.Error",
        date: "2021-10-22T23:18:23.366+03:00",
    },
    {
        description: "retries",
        type: "error",
        value: "4",
        path: "ETwitterStreamEvent.ReconnectAttempt",
        date: "2021-10-22T23:18:53.307+03:00",
    },
    {
        description: "retries",
        type: "error",
        value: "4",
        path: "ETwitterStreamEvent.ReconnectError",
        date: "2021-10-22T23:18:53.486+03:00",
    },
];

From here on Error, ReconnectAttempt and ReconnectError keep emitting forever.

[alkihis]

Hello again,
I'll test on my side during some days, then I'll come back to you.

[barca-reddit]

Hey @alkihis I just wanted to let you know that ever since I updated to 1.6.4 a few days ago (on Saturday if I recall correctly), I have stopped getting this ECONNRESET completely. This is my entire events log since then (sorted new -> old):

As you will notice, there is a single disconnect since then, but the stream managed to recover quickly. It was an OperationalDisconnect which I suppose is pretty normal, and it was also happening in the past.

The only unusual thing I've noticed here is that there was a seemingly random ETwitterStreamEvent.Connected emitted (the last log entry) with no errors or other events occurring or being emitted prior to it. I have checked if the bot was operating normally at the time of the event, and as far as I can tell everything seemed in order.

But anyway, I am not really sure what to make out of all of this and what exactly fixed the problem, but if it works then it works and that makes me happy. I thought all of this was worth mentioning, since in one of my comments here I suggested you could try reproducing the same error on Heroku, but at the moment, if you use the same code and same Heroku server I am not really sure that is going to yield any useful results (in the form of errors and disconnects).

Nevertheless, I have no intentions of stopping the bot, so it will keep on running it and if I run into the same problem, or a different problem then I shall let you know.

[Zedifuu]

Hey @alkihis,

I was actually about to reopen my previous issue again, but skimmed this thread and related my issue.
I'm also having continued issues with reconnect, unfortunately. I sat during work today puzzling over what the issue could be so I'm just gonna drop my thoughts and hopefully it helps brainstorm what the issue could be.

I haven't looked into the package source yet, but how is it the reconnect function works to actually restore the connection? Does it try and retrieve the previous connection that it had to Twitter or does it create & reauthenticate a brand new connection based off of the exact same tweet stream parameters in the object?

My thoughts are for this because I've observed in my production env logs, recovering from a connection issue when it was an operational disconnect which is just a short blip from twitter disconnecting, that just requires requires recovering that connection (probably from Twitter doing work on their cluster requiring the connection to close. That's what I understand operational disconnect as anyway)
But i've seen in my log, if its a different kind like an actual network outage, if the network isn't restored in time say... 20 seconds, the script gets stuck in a reconnect loop where even though all other hosted services the script runs have restored connections with the network, the Twitter package just keeps retrying to connect to a previous stream until the retry limit is exhausted.

Now I wonder if this is what's actually happening because Twitter destroys the previous connection on their end after 20 seconds from losing it, we're trying to connect to that previous stream and are unable to because twitter has destroyed it? Where after 20 seconds we should be authenticating again & creating a brand new connection?

I am a complete noob regarding how the requests actually work and have no knowledge on how the reconnect functions currently, so everything I've just said could be logical nonsense.

Before I version bumped to the release adding missing endpoints, I listened to the disconnect/connection close Emitter and had it fire .connect() 21 seconds after the connection closed (before the autoConnect Boolean released on the repo changing .connect() returning a promise) It was rough but worked for most things....
but if I have assumed how it works correctly, and the reconnect function just does the exact same as my work around and just calls .connect() I would be completely stumped.

I hope what I've typed with limited knowledge makes sense and also congrats on becoming the main maintainer! :)

[Zedifuu]

For Example after looking at the TweetStream src, we call the .reconnect() in onConnectionError on line 322, which is great if its a short connection error and twitter hasn't terminated/invalidated the previous connection their end after 20 seconds:

node-twitter-api-v2/src/stream/TweetStream.ts

Lines 303 to 333 in 27fd130

	protected async onConnectionError(retryOccurence = 0) {
	this.unbindTimeouts();
	// Close the request if necessary
	this.closeWithoutEmit();
	// Terminate stream by events if necessary (no auto-reconnect or retries exceeded)
	if (!this.autoReconnect) {
	this.emit(ETwitterStreamEvent.ConnectionClosed);
	return;
	}
	if (retryOccurence >= this.autoReconnectRetries) {
	this.emit(ETwitterStreamEvent.ReconnectLimitExceeded);
	this.emit(ETwitterStreamEvent.ConnectionClosed);
	return;
	}
	// If all other conditions fails, do a reconnect attempt
	try {
	this.emit(ETwitterStreamEvent.ReconnectAttempt, retryOccurence);
	await this.reconnect();
	} catch (e) {
	this.emit(ETwitterStreamEvent.ReconnectError, retryOccurence);
	this.emit(ETwitterStreamEvent.Error, {
	type: ETwitterStreamEvent.ReconnectError,
	error: e,
	message: `Reconnect error - ${retryOccurence + 1} attempts made yet.`,
	});
	this.makeAutoReconnectRetry(retryOccurence);
	}
	}

Would it make more sense to .clone() and create a new connection? and destroy the old instance on our end? :

node-twitter-api-v2/src/stream/TweetStream.ts

Lines 208 to 228 in 27fd130

	/**
	* Make a new request that creates a new `TweetStream` instance with
	* the same parameters, and bind current listeners to new stream.
	*/
	async clone() {
	const newRequest = new RequestHandlerHelper<T>(this.requestData);
	const newStream = await newRequest.makeRequestAsStream();
	// Clone attached listeners
	const listenerNames = this.eventNames();
	for (const listener of listenerNames) {
	const callbacks = this.listeners(listener);
	for (const callback of callbacks) {
	newStream.on(listener, callback as any);
	}
	}
	return newStream;
	}

rather than try and .reconnect() to a possibly terminated connection, which may be making it sit there looping continuing to attempt to connect to something that's been terminated and no longer exist on twitters end?

node-twitter-api-v2/src/stream/TweetStream.ts

Lines 274 to 301 in 27fd130

	/** Make a new request to (re)connect to Twitter. */
	async reconnect() {
	if (this.connectionProcessRunning) {
	throw new Error('Connection process is already running.');
	}
	this.connectionProcessRunning = true;
	try {
	let initialConnection = true;
	if (this.req) {
	initialConnection = false;
	this.closeWithoutEmit();
	}
	const { req, res } = await new RequestHandlerHelper(this.requestData).makeRequestAndResolveWhenReady();
	this.req = req;
	this.res = res;
	this.emit(initialConnection ? ETwitterStreamEvent.Connected : ETwitterStreamEvent.Reconnected);
	this.parser.reset();
	this.initEventsFromRequest();
	} finally {
	this.connectionProcessRunning = false;
	}
	}

Again I'm really not sure how a lot of this works and could be getting it completely wrong but suggestions are suggestion there is the chance that I may be right 😅

[barca-reddit]

At this point I am almost certain this is a problem with the Twitter API itself, there are a few threads on twittercommunity.com such as this one or this one dating back from October last year. It doesn't seem like Twitter devs care enough to fix it. Some user is suggesting to "Simply wrap your requests calls in a “with” context block and it closes unused connections that come about when your script stops unexpectedly." but I honestly don't even know what that's supposed to mean.

Also I got another ECONNRESET today after 4-5 days of smooth sailing. The one where you have to wait around 15 minutes and the connection eventually recovers, but you get TooManyConnections thrown at you before it does. I have written a handy method to actually test all currently the established connections on my Heroku server. Not the most elegant one, but it did what it was supposed to do:

    async sockets() {
        try {
            const { stdout, stderr } = await exec('ss -t | grep 104.244');
            if (stdout) {
                return stdout.replace(/\s+/g, ' ');
            }
            else if (stderr) {
                return stderr;
            }
        } catch (error) {
            if (error?.code === 1) {
                return 'No open sockets at 104.244.x.x'
            }
            else {
                return error;
            }
        }
    }

What this showed me is that there were in fact zero established or ongoing connections while ETwitterStreamEvent.Error was emitting this error:

{
    "type": "reconnect error",
    "error": {
        "error": true,
        "type": "response",
        "code": 429,
        "headers": {
            "date": "Wed, 27 Oct 2021 18:40:00 UTC",
            "server": "tsa_b",
            "content-type": "application/json; charset=utf-8",
            "cache-control": "no-cache, no-store, max-age=0",
            "content-length": "213",
            "x-access-level": "read",
            "x-frame-options": "SAMEORIGIN",
            "x-xss-protection": "0",
            "x-rate-limit-limit": "50",
            "x-rate-limit-reset": "1635360749",
            "content-disposition": "attachment; filename=json.json",
            "x-content-type-options": "nosniff",
            "x-rate-limit-remaining": "44",
            "strict-transport-security": "max-age=631138519",
            "x-response-time": "88",
            "connection": "close"
        },
        "rateLimit": {
            "limit": 50,
            "remaining": 44,
            "reset": 1635360749
        },
        "data": {
            "title": "ConnectionException",
            "detail": "This stream is currently at the maximum allowed connection limit.",
            "connection_issue": "TooManyConnections",
            "type": "https://api.twitter.com/2/problems/streaming-connection"
        }
    },
    "message": "Reconnect error - 6 attempts made yet."
}

Obviously this can not be true, because there were no active/established/ongoing TCP connections to 104.244.x.x, I double-triple-checked the netstat/ss command, but for some reason the Twitter API and their servers probably think we're still connected for some reason.

Here is the interesting part. During that 15 minute window of error emitting, I was around and managed to do some more tests. I have written a command for my Discord bot that lets me manually call the stream.destroy() method, and I am also able to manually call the Twitter.startStream() class method, essentially restarting the stream using Discord commands. This only restarts the stream, but not the whole server/bot (I explain why this is important below).

After doing exactly that it did not work out and I was still getting TooManyConnections and I couldn't manage to (re)connect. So I shut down the Heroku server, and then I started my local machine dev server. The result was that I was able to connect successfully on the first attempt, no errors whatsoever. What I did next was quickly stop my dev server before the 15 min period ends, and then I started the Heroku server again, and it managed to connect there on the first attempt as well, before that 15 minute window has expired.

That leads me to believe it might be possible to find a workaround around this problem, instead of having to wait 15 minutes every time, although it might end up as a "hacky" solution that I am not even 100% sure should become part of this library's code, it's not up to me to decide anyway.

In any case, now I am just wondering what are the fundamental differences between using the stream.destroy(), autoReconnect and other methods this library provides, compared to manually restarting the whole server, because apparently that did the trick and we managed to circumvent the 15 min waiting time, which I am almost sure is an error/bug on Twitter's API.

EDIT: Ignore the last 4 paragraphs, the error happened again and neither restarting the Heroku server, nor launching the bot on my local dev server managed to successfully connect, same error TooManyConnections is being thrown. After about 15 minutes it managed to connect again. No idea why it worked few hours ago, probably something unrelated. I also opened an issue on twittercommunity.com.

[Zedifuu]

See I'm not entirely sure that it is an error with their Twitter API I think their connection limit and rate limits are functioning correctly as per their documentation, I think it may be a case of the reconnection strategy of the package may need revised after reviewing the API reference and documentation. It could be we are trying to reconnect too quickly which is causing problems.

I'm not sure though, I'll include the API Reference related to this issue:
Handling Disconnections | Rate Limits | A wee stack article

Note that every HTTP 429 received increases the time you must wait until rate limiting will no longer be in effect for your account.

[barca-reddit]

@Zedifuu you might be on to something, but I'm not sure either. The full quote here is:

Back off exponentially for HTTP 429 errors Rate limit exceeded. Start with a 1 minute wait and double each attempt. Note that every HTTP 429 received increases the time you must wait until rate limiting will no longer be in effect for your account.

The thing is though, at least in my case I am not getting Rate limit exceeded but rather TooManyConnections. So in other words while the error is indeed 429, it's of different type and the error data that Twitter sends back tells me I have remaining rate limit, for example:

"rateLimit": {
    "limit": 50,
    "remaining": 44,
    "reset": 1635360749
}

Are you using the custom .nextRetryTimeout method of TweetStream by the way?

https://github.com/PLhery/node-twitter-api-v2/blob/master/doc/streaming.md#methods--properties

It basically lets you define your own login on how long to wait between reconnect attempts. Mine is currently set to 30 seconds flat, but it might be worth experimenting with 1 minute or 2 minutes for example, just to see if it will make any different. It's going to be really annoying to reproduce though, you basically have to deploy to production like that and then wait couple of day until the connection drops...

[Zedifuu]

So there isn't really a response code for TooManyConnections if you require limiting the number of Connections that you have to server/service it's common practise just to use the same response code that you would for someone sending too many requests to a server/service. So in this case even though it's responding saying 429 TooManyConnections it's still triggering the same rate limit rules as what would be used in the event of too many requests.

Oooooooo I have not used .nextRetryTimeout yet, only recently bumped my version... Very useful!
I'll look at writing some logic and report back. We have however given alkihis some amount of reading material though, I look forward to their input :>

and then there is of course looking at other established packages as well and understanding how they document & tackle the same problem. Cause we're all using the same endpoints here, so it's the case that this its' already been probably discussed.

For example that rate limit segment you just posted there, have a look at the troubleshooting section of Twitter-lite:
Twitter-lite Troubleshooting rate limits

Did that rate limit by any chance expire 26/27 hrs ago? :>

[Zedifuu]

I have found when working in my dev env, if I ctrl+c my script and try to restart immediately I get 429: TooManyConnections
If I wait 20 seconds after I've killed the script and try again everything works fine,
twitter-lite confirms this 20/30secs theory:

Least were not getting told to Enhance Our Calm :P

So if you're receiving TooManyConnections we could just be going too fast and the old stream hasn't been destroyed by twitter yet. causing the 429.

From the looks of things .closeWithoutEmit is used in onConnectionError and .reconnect.
It essentially calls .req.destroy without firing an event emitter, the above functions have no delay after using and creating a new stream, so twitter could still be closing the stream from destroy, when we try and immediately connect again, causing TooManyConnections at times?

node-twitter-api-v2/src/stream/TweetStream.ts

Lines 306 to 322 in d73426e

	this.closeWithoutEmit();
	// Terminate stream by events if necessary (no auto-reconnect or retries exceeded)
	if (!this.autoReconnect) {
	this.emit(ETwitterStreamEvent.ConnectionClosed);
	return;
	}
	if (retryOccurence >= this.autoReconnectRetries) {
	this.emit(ETwitterStreamEvent.ReconnectLimitExceeded);
	this.emit(ETwitterStreamEvent.ConnectionClosed);
	return;
	}
	// If all other conditions fails, do a reconnect attempt
	try {
	this.emit(ETwitterStreamEvent.ReconnectAttempt, retryOccurence);
	await this.reconnect();

node-twitter-api-v2/src/stream/TweetStream.ts

Lines 287 to 290 in d73426e

	this.closeWithoutEmit();
	}
	const { req, res } = await new RequestHandlerHelper(this.requestData).makeRequestAndResolveWhenReady();

Maybe add 20/30 sec, delay before starting a new connection in the above functions?

[barca-reddit]

@Zedifuu a lot of info to process here, but assuming that you are correct about everything, what I was thinking modifying the nextRetryTimeout() function that you pass as a parameter to the connect() method in the following way (just an example):

await this.stream.connect({
    autoReconnect: true,
    autoReconnectRetries: Infinity,
    nextRetryTimeout: (tryOccurence, error) => {
        if (error?.code === 429) {
            /*
                So the idea here is the following:

                1. Pass the error object down the nextRetryTimeout so we have access to it.
                2. Check if the error code is 429.
            */
            if (tryOccurence === 0) {
                /*
                    If this is the first time you're getting 429-ed,
                    don't even attempt to reconnect immediately,
                    and instead wait 1 minute before making another attempt.
                */

                return 60000; // 60000 milliseconds = 1 minute
            }

            else {
                /*
                    If the first reconnect attempt after 60 seconds wasn't succesful,
                    futher increase the wait time between upcoming attempts, up to 5 minutes.
                */

                return tryOccurence * 60000 < 300000 ? tryOccurence * 60000 : 300000; // or some other logic here

            }
        }
        else {
            /*
                If this isn't a 429 error, then handle things the usual way.
            */

            return 30000;
        }
    },
});

In order for this to work, the nextRetryTimeout function needs access to the error that Twitter throws, which it currently does not. I am thinking about possibly modifying the code of the library on my end to see if this would work but what do you think about this?

From the looks of things .closeWithoutEmit is used in onConnectionError and .reconnect.
It essentially calls .req.destroy without firing an event emitter, the above functions have no delay after using and creating a new stream, so twitter could still be closing the stream from destroy, when we try and immediately connect again, causing TooManyConnections at times?

Well, if this is indeed true, I am not sure if the initial check of if (tryOccurence === 0) { would be a delayed one or an immediate one. I suppose I need to test this somehow.

[Zedifuu]

Ooooo nice, only thing though, .connect calls .reconnect and .reconnect regardless of the strategy still destroys then immediately tries to create a new connection a fraction of a second later.... I'm not sure but I feel this could cause unexpected behaviour. And may be triggering the failure point in the first place that is causing us to have to rethink the reconnection strategy?

node-twitter-api-v2/src/stream/TweetStream.ts

Line 253 in d73426e

await this.reconnect();

I'm looking at as we're getting the 429 in the first place, a sufficient reconnect strategy we should not be getting a 429, so if the methods were using to space out the reconnect attempts is sound, are the functions we're using to reconnect at fault?

[barca-reddit]

Ooooo nice, only thing though, .connect calls .reconnect and .reconnect regardless of the strategy still destroys then immediately tries to create a new connection a fraction of a second later.... I'm not sure but I feel this could cause unexpected behaviour. And may be triggering the failure point in the first place that is causing us to have to rethink the reconnection strategy?

node-twitter-api-v2/src/stream/TweetStream.ts

Line 253 in d73426e

await this.reconnect();

I'm looking at as we're getting the 429 in the first place, a sufficient reconnect strategy we should not be getting a 429, so if the methods were using to space out the reconnect attempts is sound, are the functions we're using to reconnect at fault?

Very good point, and very good question, somehow I did not think of that. I suppose we can wait for the developer's (@alkihis) input on this, and perhaps in the meantime until they reply, try something on our own.

I am thinking, maybe just to prove the point, add an artificial timeout of 1 or 2 minutes, right before line 253, in order to delay the initial reconnect() and see if that helps with the issue. The big problem remains though, if you are unlucky, you would have to wait at least a couple of days until you get an ECONNRESET in order to reproduce the problem.

[Zedifuu]

I suppose we can wait for the developer's (@alkihis)

My thoughts exactly :) We certainly have given a lot of ideas and reading material anyway 😄

Thats an idea! how about on line 289? This would however delay all connections including initial which isn't great

node-twitter-api-v2/src/stream/TweetStream.ts

Lines 285 to 290 in d73426e

	if (this.req) {
	initialConnection = false;
	this.closeWithoutEmit();
	}
	const { req, res } = await new RequestHandlerHelper(this.requestData).makeRequestAndResolveWhenReady();

in the reconnect function ? Since everything ( onConnectionError, .connect) seems to be calling reconnect?

[barca-reddit]

Gonna keep it ultra short this time. Another ECONNRESET today on my production server. I made 3 reconnect attempts in 90 seconds, then I called stream.destroy() and waited 5 minutes without doing anything. Tried to connect after that, but got the same error as before. After two additional reconnect attempts I shut down the whole server for another 3-4 minutes, but again, TooManyConnections after reboot.

15 minutes since the initial ECONNRESET it managed to reconnect. It's worth pointing out that I did not do any modifications to delay the first reconnect attempt, so just running the old code as before. But it seems to me that once you get stuck in this hole, there is no coming out unless you wait 15 minutes.

[alkihis]

Okay, a lot of discussion happend here in a week.
Here's my detailed response for all the debated things in that issue during the last couple of days.

• About occurencies of stream errors

  I've set, last sunday (October 24), on my personal VPS a running Twitter API v2 stream using the last version of the package (1.6.4).
  All connect/reconnect and error events has been logged since that day.

  During the last 8 days:

  • 27 disconnections, 24 initiated by Twitter (operational disconnect), 3 manually initiated (with tcpkill)
  • For every disconnection initiated by Twitter, the first reconnect attempt (made immediately after connection loss) is successful, every time
  • For each disconnection initiated by me, the first reconnect attempt always fails (Twitter seems to have a latency to detect abruptly closed streams). Stream has been recovered successfully after the second attempt (5 seconds after first attempt)

  So, on my side, no abnormal behavior has been detected.

• About .connect/.reconnect/.clone methods

  Methods .connect/.reconnect behaves like they were meant to work.
  No, it's not strange that .connect call .reconnect, because this is basically the same thing:

  1. If res/req are stored in instance, destroy them, clean the possibly started timeouts
  2. Make a new request to Twitter, obtain IncomingResponse (res) and ClientRequest (req) objects, and store them in instance
  3. If the request FAILS, initiate reconnect timeout: call nextRetryTimeout to get the number of ms to await before retrying, then go back to step 1
  4. If the request SUCCEEDS, bind all the listeners to those two objects in order to catch data payloads and errors

  Method .clone make the two first steps, but instead of storing req and res to current instance, it creates a fresh one and bind all the current listeners of this instance to the new one.
  As Twitter do not support two concurrent steams for now, this method is completely useless.

• Why reconnection always fails

  In Twitter documentation, they recommand to increment the waiting time between two requests with a exponential-like function, up to 30 seconds in order to minimize time where stream will be left without connect.

  Note that 15-minute window usually don't apply for streams, rate limiting is made quite differently and documentation found for REST-like endpoints is not really valid for stream-like endpoints.
  I don't find any documentation, in v1 and v2 API, directly related to streaming endpoints, that mentionned a 15-minute rate limit window.

  @barca-reddit mentionned that, even if they're any connection to Twitter after the disconnection (checked with a list of open sockets), Twitter sometimes replies that rate limit is hit.

  Even if I don't encounter this problem, I suppose the problem is more on Twitter side than on client side. Even more if a server reboot doesn't fix the problem sometimes!

  If this issue resolves by itself by waiting 15 minutes (and not 30 seconds), I suggest that we can use a nextRetryTimeout that return a ms number up to 900000 (15 minutes) instead of the current 2 minutes cap.
  It can be interesting to know if this issue also affects streams for v1 API.

• Why .reconnect is fired immediately after connection error

  As I said before, we want to minimize time where stream is "offline". Reconnection process can be trigged after a 2 minutes timeout (.ConnectionLost event), as it is fired when a ECONNRESET happens (propagated through error event of ClientRequest).

  If we come from a timeout, we don't want to wait any longer, we want a reconnection now. And, even if a reconnection is started "too soon", next attempts will follow later (at least we tried to recover as fast as possible!).

  Adding an artificial delay of 1-2 minutes after the initial disconnection (in a 429 error case) will not resolve the issue at all, because the problem seems to be a rate-limit cleanup issue on Twitter side.

  Also, reminder that before a reconnection attempt, previous request is always destroyed.

  One last thing, .nextRetryTimeout is called AFTER a reconnect attempt, so it's NOT called before the first one: parameter tryOccurence is never 0.

The changes discussed before (up to 15 minutes for .nextRetryTimeout default function, and error parameter in .nextRetryTimeout for custom implementations) will be pushed soon on npm.

Hope it will help you to resolve your issues.

[barca-reddit]

Thank you for dedicating time to test this @alkihis, and sad to see that unfortunately you were unable to reproduce the exact same error/behavior. I suppose it's kinda random, but which exact clients or environments are affected here is hard to tell.

As I said before, we want to minimize time where stream is "offline". Reconnection process can be trigged after a 2 minutes timeout (.ConnectionLost event), as it is fired when a ECONNRESET happens (propagated through error event of ClientRequest).

Sorry, I am not sure I completely understand this part, are you saying that the ECONNRESETs are happening because of a 2 minute connection timeout, or are you simply saying that some errors might occur because of a timeout?

If it's the former and you are certain about this, then I guess there is not much that can be done here. @Zedifuu speculated that an immediate reconnect attempt is what makes matters worse, but if you're right about this then that theory would be invalid.

As for the upcoming changes with a .nextRetryTimeout of up to 15 minutes, I am not really sure that solves anything here, because the whole idea here was to somehow avoid having to wait 15 minutes before reconnects in first place (if that is even possible), but I guess there is nothing else you can do.

Having the error parameter in there would come in handy though. For example if you see the TooManyConnections in there you just know you have to bite the bullet, so just return 900000, no point making further reconnect attempts.

The backfill_minutes parameter of GET /2/tweets/search/stream would have been really useful here, but sadly it only allows for up to five minutes worth of data, and it's only available to the academic research product track.

[barca-reddit]

Hey @alkihis, I just wanted to let you know that I haven't had any more 15 minute reconnect loops since the 15th of December last year, so that's around 20 days ago. I really haven't done or changed anything since my last comment in November, because I ran out of time and patience to deal with this issue and just decided to live with it.

But in any case, this just confirms what I suspected, which is that it was a problem with Twitter (or less likely with Heroku) that they have eventually fixed, and not a problem with this library.

So that's all, and finally I also wanted to say thank you again for providing help and even going lengths trying to reproduce the problem on your end, it is appreciated.

I'm closing this issue now because it's resolved (and hopefully it stays that way). Cheers!

[Zedifuu]

Hey guys,

I forgot to also check back in, I have also stopped having reconnect rate limits also.

I have a funny feeling this was fixed quietly by twitter when they released API v2 from Early Access

I feel I was at most part trying to diagnose an issue that wasn't caused by us 😅
Sorry for fuelling that rabbit hole!

Wish you guys all the best with what you create And again @alkihis excellent work again, Thank you both for the chat!

[barjabeen12]

Hey is there anyone who is still getting this issue, i am facing this issue with stream. Now stream just stop listening to events and doesn't give any error. i have to manually restart the stream after 2 to 3 days.

[barca-reddit]

Hey is there anyone who is still getting this issue, i am facing this issue with stream. Now stream just stop listening to events and doesn't give any error. i have to manually restart the stream after 2 to 3 days.

This is not the same issue as the one that we were discussing here for the most part. At least in my case I was unable to recover the connection after getting an ECONNRESET error and it stays down for like 15 minutes until it manages to recover. But I don't think I've ever had an instance where the connection dropped without any errors.

What I could quickly suggest is to bind a listener for the following events: ConnectionError, ConnectionClosed, ConnectionLost, ConnectError, ReconnectError, ReconnectLimitExceeded, DataError, TweetParseError and Error which would hopefully give you more details on why the disconnect is occurring.

https://github.com/PLhery/node-twitter-api-v2/blob/master/doc/streaming.md#events

I would also suggest creating a separate issue, the maintainers and devs of this library are quite helpful so they would probably help if you provide them with additional details.

---

By the way, despite this comment shortly after I started getting ECONNRESETs again, but I just don't have time, desire or energy to go down this rabbit hole again, so whatever. A few disconnects per month (5-10) isn't that big of a deal. It's most likely an issue with Twitter or less likely with Heroku.