Add some throttling to the new /upload/ api

[AlexandraMoga]

Follow up for #8511

Describe the problem and steps to reproduce it:

1. Send a few consecutive requests to the new api/v5/addons/upload/ endpoint
2. Check the API response

What happened?

Each request is successfully processed

What did you expect to happen?

The API should be throttled after several consecutive requests

[eviljeff]

@diox @wagnerand opinions on this?

To quote myself from #8511: I deliberately didn't add any [throttles], because a user could end up having to submit multiple uploads if they didn't test with the linter before submission, or for non-linter validation errors, so the existing limits would be too low. But we potentially should have some throttling

So, we can't use the same shared throttles as devhub, the old signing api, or the new addon submission api, because they're a calibrated for creating new add-ons. (And even if in the best scenario of a single upload plus a single addon submission, we'd be halving the threshold if we share the throttle). If want some throttling we need to settle on the limits.

Starting point would be the existing submission throttle limits, but not shared. I'd lean toward higher limits though, because of the chances of linter or non-linter errors thrown at that first stage.

[wagnerand]

Can the limit apply to submissions that passed validation?

[eviljeff]

Can the limit apply to submissions that passed validation?

I don't believe so - the validation is running in a celery task so we don't know if a request turned out to be for a successful upload until long after the request has finished.

[diox]

Agreed it should be a separate throttle, more generous than the ones used for new add-on/version.

Ultimately there is not much to gain for an attacker (besides costing us CPU/storage) by submitting a lot of uploads - there is not much information to gain as scanners actions will be applied in the following step, when the version is created, which already has rate-limiting.

[eviljeff]

Reference the existing throttles:
https://github.com/mozilla/addons-server/blob/master/src/olympia/api/throttling.py#L99:L131

Duplicate the same 5 throttles but double the values?

[diox]

I think you could re-use the IP-based ones as-is and duplicate the user ones doubling the values, to start.

Edit: actually you'll need to duplicate them all, even those we don't need to change the values of, because you want a separate scope.

[AlexandraMoga]

I've verified the rate limiting on stage and these are some of my results:

• the new /addons/upload/ api:
  • request limit per user/minute is 6
  • request limit per user/hour is 20
  • request limit per IP/minute is 6 - @eviljeff so I can't see the IP rate limit being doubled as in the case of user limits. Shouldn't it be 12, or am I misreading the Edit part in Add some throttling to the new /upload/ api #8537?
  • currently, there is no way to bypass these limits, since the exceptions we have in place don't seem to apply to this api
• the new addon submission api, the old signing api and devhub:
  • request limit per user/minute is 3
  • request limit per user/hour is 10
  • request limit per user/day is 24
  • request limit per IP/minute is 6
  • these rate limits can be bypassed with the old signing api and in devhub, but not with the new submission api

[eviljeff]

  * request limit per IP/minute is 6 - @eviljeff so I can't see the IP rate limit being doubled as in the case of user limits. Shouldn't it be 12, or am I misreading the Edit part in [Add some throttling to the new /upload/ api mozilla/addons#8537 (comment)](https://github.com/mozilla/addons/issues/8537)?

I read @diox 's comment as all the throttle classes needed to be duplicated (different scopes) with the user throttle limits doubled and the ip throttle limits kept the same.

[diox]

This. I don't think we need to be more generous with the IP based ones for now. We'll revisit if needed when the API starts getting more use.