I have both Apache and Modsecurity working together. I’m trying to limit hit rate by request’s header (like “facebookexternalhit”). And then return a friendly “429 Too Many Requests” and “Retry-After: 3”.
I know I can read a file of headers like:
SecRule REQUEST_HEADERS:User-Agent "@pmFromFile ratelimit-bots.txt"
But I’m getting trouble building the rule.
Any help would be really appreciated. Thank you.
2
Answers
After 2 days of researching and understanding how Modsecurity works, I finally did it. FYI I'm using Apache 2.4.37 and Modsecurity 2.9.2 This is what I did:
In my custom file rules:
/etc/modsecurity/modsecurity_custom.conf
I've added the following rule:Explanation:
Note: I want to limit to 1 request every 3 seconds.
You could improve this rule by adding @pmf and a .data file, then initializing global collection like
initcol:global=%{MATCHED_VAR}
, so you are not limited just to a single match by rule. I didn't test this last step (this is what I needed right now). I'll update my answer in case I do.UPDATE:
I've adapted the rule to be able to have a file with all user agents I want to rate limit, so a single rule can be used across multiple bots/crawlers:
So, the file with user agents (one per line) is located inside a subdirectory under the same directory of this rule:
/etc/modsecurity/data/ratelimit-clients.data
. Then we use @pmf to read and parse the file (https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#pmfromfile). We initialize the USER collection with the user agent:setuid:%{tx.ua_hash}
(tx.ua_hash is in the global scope in/usr/share/modsecurity-crs/modsecurity_crs_10_setup.conf
). And we simply use user as collection instead of global. That's all!Might be better to use "deprecatevar",
And you can allow a bit bigger burst leneanancy