Merge pull request #22 from simplificator/robin-francois/master

Robin francois/master: Adding configuration elements to block based on user-agents

README.md

@@ -13,7 +13,8 @@ N/A
 If you only want to install Caddy, you don't need to set any variables. If you want to configure Caddy as a reverse proxy as well, you can provide an array of objects named `caddy_sites` with the following values:
 
 * `additional_forwarding_ports`: Allows to define a list with additional ports where Caddy should listen for this domain and forward to HTTPS.
-* `allowlist`: An array if IP addresses in CIDR-notation which are allowed to access this site (Optional). All other visitors receive a 404 error.
+* `allowlist`: An array of IP addresses in CIDR-notation which are allowed to access this site (Optional). All other visitors receive a 404 error.
+* `useragent_blocklist`: An array of User-Agents which are blocked to access this site (Optional), wildcard characters (*) need to be used for broader matching.
 * `certificate_file`: You can set this variable if you want to provide the certificate by yourself (Optional). The certificate needs permissions `0640`, with root as Owner and Caddy as Group.
 * `certificate_key`: You can set this variable if you want to provide the certificate by yourself (Optional).
 * `domain`: The domain caddy should listen to.

molecule/reverse-proxy/converge.yml

@@ -20,6 +20,8 @@
             code: 401
         allowlist:
           - 8.8.8.8/32
+        useragent_blocklist:
+          - "*amazonbot*"
         additional_forwarding_ports:
           - '8080'
           - '1337'

molecule/reverse-proxy/files/Caddyfile.expected

@@ -4,6 +4,16 @@
 
 
 example.com {
+  @badbots {
+    header User-Agent *amazonbot*
+  }
+
+  handle @badbots {
+      respond 403 {
+          body "Access forbidden."
+      }
+  }
+
   @allowlist {
     remote_ip  8.8.8.8/32
   }

templates/Caddyfile.j2

@@ -5,6 +5,21 @@
 
 {% for site in caddy_sites %}
 {{ site.domain }} {
+  {%- if site.useragent_blocklist is defined %}
+  @badbots {
+    {%- for ua in site.useragent_blocklist %}
+    header User-Agent {{ ua }}
+    {%- endfor %}
+  }
+
+  handle @badbots {
+      respond 403 {
+          body "Access forbidden."
+      }
+  }
+
+  {%- endif %}
+
   {%- if site.allowlist is defined %}
   @allowlist {
     remote_ip {% for ip in site.allowlist %} {{ ip }}{% endfor %}