Request Throttling

[FdezRomero]

Hi all!

This is a question rather than an issue: What is in your opinion the best way to throttle requests to the Amazon Product API? The initial limit is 1 req/s, increasing by 1 req/s for every $4,700 in goods sold by your awsTag.

I'm currently using Bottleneck with a limit of 1 request per 2000ms and I'm still getting many RequestThrottled errors. So it doesn't seem to be playing well with this module or the API.

Are any of you throttling your requests? How? It would be great if this feature was included, as similar projects in other languages have a MaxQPS option.

Thanks!

[t3chnoboy]

Hey @FdezRomero!
It think it is a good idea to add an option to limit requests.
https://github.com/jhurliman/node-rate-limiter comes to mind. Any advantages of using bottleneck over it?
Can you show an example of that option in other languages?

[FdezRomero]

Hi @t3chnoboy,

I chose Bottleneck over node-rate-limiter because it allows to create a different keys/queues for each API endpoint and seemed very lightweight and performant (the creators use it for production DNS). However, I'm not sure if some endpoints are counted together for the rate limiting or not, I found contradictory information about this. Anyway, I have no preference in using these libraries or a custom implementation as long as it supports concurrency.

You can find an example of the MaxQPS I mentioned before in bottlenose, a client written in Python (I haven't used it myself).

Thanks!

[kaijuliu]

Hi @FdezRomero. I recently ran into the same throttling issue and found that global throttling, i.e. not per-endpoint throttling, was the solution. Putting https://github.com/jhurliman/node-rate-limiter, as @t3chnoboy suggested, in front of all requests worked well.

[FdezRomero]

Thanks @kaijuliu, knowing that the limit is actually global and not per endpoint really helps 👍
I read some time ago a comment saying that the European stores acted like a cluster and the rest were independent, but I never found official information from Amazon...

[masterT]

@t3chnoboy can we close this issue?