Multiple zones definition and processing order

[sriccio]

Hello,

First, I would like to thank you for this amazing module !

I recently did some tests with it and I have some questions about multi zone handling. I've this configuration:

        # Rate limit
        rate_limit {
                zone target_per_ip {
                        key target_per_ip-{client_ip}-{http.request.host}
                        events 1200
                        window 1m
                }
                zone per_ip {
                        key per_ip-{client_ip}
                        events 3000
                        window 1m
                }
                zone per_target {
                       key per_target-{http.request.host}
                       events 6000
                       window 1m
                }
                log_key
        }
        reverse_proxy http://127.0.0.80:80

What I'm trying to achieve is:

• Rate limit one specific IP that is constantly hammering one specific target domain (target_per_ip).
• Rate limit one specific IP that is constantly hammering any target domain (per_ip)
• Rate limit one specific target domain whois is constantly hammered by any source IP (per_target)

When doing some tests for exemple with ab from a remote host and some agressive parameters, the first rate limit to trigger is target_per_ip which is correct.

{"level":"info","ts":1733027294.5232427,"logger":"http.handlers.rate_limit","msg":"rate limit exceeded","zone":"target_per_ip","wait":57.565922146,"remote_ip":"94.x.97.100","key":"target_per_ip-94.x.97.100-www.target.ch"}
{"level":"info","ts":1733027294.52412,"logger":"http.handlers.rate_limit","msg":"rate limit exceeded","zone":"target_per_ip","wait":57.565031375,"remote_ip":"94.x.97.100","key":"target_per_ip-94.x.97.100-www.target.ch"}
{"level":"info","ts":1733027294.525319,"logger":"http.handlers.rate_limit","msg":"rate limit exceeded","zone":"target_per_ip","wait":57.563836569,"remote_ip":"94.x.97.100","key":"target_per_ip-94.x.97.100-www.target.ch"}

Then after a few seconds it triggers the second per_ip limit which is a bit more tolerant

{"level":"info","ts":1733027182.089967,"logger":"http.handlers.rate_limit","msg":"rate limit exceeded","zone":"per_ip","wait":53.771214735,"remote_ip":"94.x.97.100","key":"per_ip-94.x.97.100"}
{"level":"info","ts":1733027182.090872,"logger":"http.handlers.rate_limit","msg":"rate limit exceeded","zone":"per_ip","wait":53.770307777,"remote_ip":"94.x.97.100","key":"per_ip-94.x.97.100"}
{"level":"info","ts":1733027182.0909524,"logger":"http.handlers.rate_limit","msg":"rate limit exceeded","zone":"per_ip","wait":53.770228556,"remote_ip":"94.x.97.100","key":"per_ip-94.x.97.100"}

And finally it triggers the third limit per_target which is even more tolerant

{"level":"info","ts":1733027183.5868707,"logger":"http.handlers.rate_limit","msg":"rate limit exceeded","zone":"per_target","wait":52.27425604,"remote_ip":"94.x.97.100","key":"per_t
arget-www.target.ch"}
{"level":"info","ts":1733027183.587857,"logger":"http.handlers.rate_limit","msg":"rate limit exceeded","zone":"per_target","wait":52.273271302,"remote_ip":"94.x.97.100","key":"per_t
arget-www.target.ch"}
{"level":"info","ts":1733027183.5879197,"logger":"http.handlers.rate_limit","msg":"rate limit exceeded","zone":"per_target","wait":52.273207694,"remote_ip":"94.x97.100","key":"per_
target-www.target.ch"}

My question is; as soon as the request hits one of the limit, in this example per_ip and is returned with a 429, the other zones counters seems to increase even if the request is denied with a 429.

In the end the "attacker" blocked with 429 in previous zones is able to trigger a 429 returnet to anyone else trying to visit the target domain even it's own request are 429'ed.

Is there some kind of zone ordering, or some way to define that a request already returned with a 429 because it exceed a zone doesn't increase the other zones counters ?
I'm not sure if my question is correctly asked :)

Side question:
I'm using {client_ip} in the placeholder because it should be populated by real client IP from trusted proxies (for example Cloudflare IP ranges). Is this a right way to do it ? Will the trusted proxies replace client_ip with the real IP before rate limit is checked ?

We define them this way (with modules to automatically get cloudflare IP ranges)

        servers {
                protocols h1 h2
                trusted_proxies cloudflare {
                        interval 12h
                        timeout 15s
                }
        }

Thanks a lot for your time reading and the feedback !

Kind regards

[mholt]

So to clarify, do you mean that when a request exceeds the per_ip rate limit, it also counts against the more permissive per_target rate limit?

[sriccio]

Hello @mholt, thanks for the reply.

Yes, it's what I am experiencing. I was not expecting a request being blocked by one of the zone still being counted as a hit in the other zones.

It's probably by design and I am doing it wrong to achieve what I try to do.

It seems to me that an attacker being rate limited by the per_ip zone and getting returned with a 429 http code still increases the other per_target zone even if the request is not served.

Kind regards

[mholt]

Hmm, that is odd if that's really what's happening. While iterating the zones, we return immediately at the first one that is exceeded. They are sorted by order of how restrictive they are (in terms of events per window).