[Pixiv] Retrying after a Rate Limit error

[MysteryDash]

Hello,

When pulling a large amount of pictures from Pixiv, we'll usually stumble upon a Rate Limit error. However, it doesn't seem like the script waits or retries after a failure. Any way to fix that?

[mikf]

Do you mean "429 Too Many Requests" errors when you say "Rate Limit error"? Because 429 HTTP status codes are retried, same as 5xx statuses, but only up to 4 times by default with a maximum wait time of 8 seconds in between.

Try increasing the maximum retry count (-R, --retries), and setting a short sleep interval between downloads (--sleep) as well as reducing your download speed (-r, --limit-rate) to prevent getting these "errors" in the first place.

[MysteryDash]

I'm talking about this message:

[pixiv][error] API request failed: {"error":{"user_message":"","message":"Rate Limit","reason":"","user_message_details":{}}}

To be more specific, it occurs when I run the command on a search that has already been previously pulled (so it skips every picture that it has already downloaded before).
Changing the maximum retry count doesn't seem to work, setting a short sleep interval doesn't seem to be doing anything (which seems to be expected since no picture is being downloaded), and the limit-rate argument seems to only affect the downloads, not the calls to the API.

[mikf]

Ah, I see. I've never gotten this message myself, so I wasn't exactly sure what you meant.

The only thing you could do in this situation is stopping the search when skipping over every picture with --abort, and I'll look into implementing some form of wait time between API calls as well as internally retrying those "Rate Limit" errors.

[KonoVitoDa]

I just subscribed for the Pixiv Premium 1 month trial, hoping it would increase my rate limit, but I'm still getting limited:

[pixiv][info] Waiting until 14:59:18 (rate limit)

Is there a way to increase this limit with a Premium account?