Hi,

the web filter log is for web filtering.
Here, we're talking about the Webserver Protection feature and the log is called reverseproxy.log.

If you change the site path route '/' to anything but not '/', you cannot access '/'.
I think you're trying to rewrite the path from '/' to '/yourpath' but that does not work.

Sabine

i left it as is and nothing change still i'm getting the same error

Forbiden

i left it as is and nothing change still i'm getting the same error

Forbiden

I edited your original post from "web protection" to "web server protection."

In WebAdmin, check the Web Application Firewall log and show us a representative line from there when you get the Forbidden screen in the client.

Cheers - Bob

2016:02:05-10:04:16 lebutm-1 reverseproxy: [Fri Feb 05 10:04:16.629081 2016] [core:warn] [pid 6123:tid 4148180672] AH00111: Config variable ${URLHardening_HTTP_Hostname} is not defined
2016:02:05-10:04:17 lebutm-1 reverseproxy: [Fri Feb 05 10:04:17.000474 2016] [mpm_worker:notice] [pid 6123:tid 4148180672] AH00292: Apache/2.4.10 (Unix) OpenSSL/1.0.1k configured -- resuming normal operations
2016:02:05-10:04:17 lebutm-1 reverseproxy: [Fri Feb 05 10:04:17.000509 2016] [core:notice] [pid 6123:tid 4148180672] AH00094: Command line: '/usr/apache/bin/httpd'
2016:02:05-10:04:17 lebutm-1 reverseproxy: [Fri Feb 05 10:04:17.000554 2016] [mpm_worker:warn] [pid 6123:tid 4148180672] AH00291: long lost child came home! (pid 17320)
2016:02:05-10:04:17 lebutm-1 reverseproxy: [Fri Feb 05 10:04:17.000592 2016] [mpm_worker:warn] [pid 6123:tid 4148180672] AH00291: long lost child came home! (pid 17325)
2016:02:05-10:04:17 lebutm-1 reverseproxy: id="0299" srcip="127.0.0.1" localip="127.0.0.1" size="56" user="-" host="127.0.0.1" method="GET" statuscode="200" reason="-" extra="-" exceptions="-" time="1301" url="/lb-status" server="localhost" referer="-" cookie="-" set-cookie="-"
2016:02:05-10:04:35 lebutm-1 reverseproxy: [Fri Feb 05 10:04:35.751537 2016] [url_hardening:error] [pid 22336:tid 4047461232] [client 10.160.2.101:62045] Hostname in HTTP request (194.126.141.186) does not match the server name (utm)
2016:02:05-10:04:35 lebutm-1 reverseproxy: id="0299" srcip="10.160.2.101" localip="194.126.141.186" size="209" user="-" host="10.160.2.101" method="GET" statuscode="403" reason="-" extra="-" exceptions="SkipURLHardening, SkipFormHardening, SkipFormHardeningMissingToken" time="11101" url="/" server="utm" referer="-" cookie="-" set-cookie="lbsfbnrudskric_cookie=;Max-Age=0;path=/;httponly;secure"
2016:02:05-10:04:35 lebutm-1 reverseproxy: [Fri Feb 05 10:04:35.806090 2016] [url_hardening:error] [pid 22336:tid 4047461232] [client 10.160.2.101:62045] Hostname in HTTP request (194.126.141.186) does not match the server name (utm), referer: https://194.126.141.186/
2016:02:05-10:04:35 lebutm-1 reverseproxy: id="0299" srcip="10.160.2.101" localip="194.126.141.186" size="220" user="-" host="10.160.2.101" method="GET" statuscode="403" reason="-" extra="-" exceptions="-" time="3725" url="/favicon.ico" server="utm" referer="https://194.126.141.186/" cookie="-" set-cookie="lbsfbnrudskric_cookie=;Max-Age=0;path=/;httponly;secure"
2016:02:05-10:05:04 lebutm-1 reverseproxy: [Fri Feb 05 10:05:04.947398 2016] [url_hardening:error] [pid 22336:tid 4072639344] [client 10.160.2.101:62062] Hostname in HTTP request (194.126.141.186) does not match the server name (utm)
2016:02:05-10:05:04 lebutm-1 reverseproxy: id="0299" srcip="10.160.2.101" localip="194.126.141.186" size="209" user="-" host="10.160.2.101" method="GET" statuscode="403" reason="-" extra="-" exceptions="SkipURLHardening, SkipFormHardening, SkipFormHardeningMissingToken" time="1862" url="/" server="utm" referer="-" cookie="-" set-cookie="lbsfbnrudskric_cookie=;Max-Age=0;path=/;httponly;secure"
2016:02:05-10:05:04 lebutm-1 reverseproxy: [Fri Feb 05 10:05:04.969642 2016] [url_hardening:error] [pid 22336:tid 4072639344] [client 10.160.2.101:62062] Hostname in HTTP request (194.126.141.186) does not match the server name (utm), referer: https://194.126.141.186/
2016:02:05-10:05:04 lebutm-1 reverseproxy: id="0299" srcip="10.160.2.101" localip="194.126.141.186" size="220" user="-" host="10.160.2.101" method="GET" statuscode="403" reason="-" extra="-" exceptions="-" time="905" url="/favicon.ico" server="utm" referer="https://194.126.141.186/" cookie="-" set-cookie="lbsfbnrudskric_cookie=;Max-Age=0;path=/;httponly;secure"

Sorry for sort of hijacking this thread.

What exactly is the IP listed between the brackets? I'm having HTTP DDoS attacks that fill the reverseproxy log at a very fast rate, but the IP listed not in our public range?

Harro, (194.126.141.186) above would appear to be the public IP that his Virtual Server uses.  I wanted to see if he had either 'Rewrite HTML' or 'Pass host header' selected.

Cheers - Bob

Bob,

Thanks, I thought so.

My issue is that in the reverseproxy.log lines I have an unknown IP. Not only not a virtual server, not even an IP in one of our ranges. So this must be a fake/constructed HTTP header, since the "Host" header contains a different IP from the virtual server that is being connected to.

Is there a way to have the firewall or the WAF drop all these url_hardening errors? Now I have to add host entries for each of the abusers source IP's so I can block them in my blackhole route. And since they are a different set every day (we get attached on a daily basis), thats a lot of manual work...

Can you show us an example from the WAF log?

Cheers - Bob

Sure, not problem.

2018:04:04-05:09:40 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:40 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:40 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:40 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:40 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:40 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:40 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:40 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name) 2018:04:04-05:09:41 firewall-1 httpd[62734]: [url_hardening:error] [pid 62734:tid 4019133296] [client 79.119.173.199:46660] Hostname in HTTP request (5.135.179.221) does not match the server name (website.name)

Typically every attack cycle comes from 5-6 IP addresses, at about 10.000 req/s.

What I think is that these are requests for one of our valid public IP's on the external interface (and not on hostname), with in the http header "host: 5.135.179.221" (which is not ours). And "webserver.name" is simply the first defined in the WAF on that IP, so the logentry is slighly misleading.

We have an Arbor Networks DDoS system in front of the UTM, and that does kick in initially, but then the number of concurrent requests seem to go down below the threshold for that system, and still hits the UTM. Since we also host high volume sites, we can't lower that threshold, so I'm looking for a solution that doesn't block legitimate traffic.

Now, every time our monitoring system sends a high CPU alert out, I have to manually check the reverseproxy.log for these IP's, and add them to as individual hosts to the network group that I have defined for the blackhole route. Once done, order is restored and CPU drops from 90-95% to 15-20%. Until the next attack cycle. This happens about 2-4 time every day now, so I'm looking for a less manual solution. ;-)

I think you're stuck with the manual approach until you decide to hire someone with strong RESTful API skills.  Maybe Sophos has a consultant that could do this.  Don't share the solution with us, but please let us know if you've found someone that can do this.

Cheers - Bob

Was afraid of that.

API skills isn't a problem, we've got a full house of those. Challenge will be to find a way to get a trigger from the RP.

What I have a bit of an issue with is that obviously there is something that checks and validates the HTTP header in a URL hardening process, but I can't find what it is exactly (it's not httpd, it might be mod_security?), and there also doesn't seeem to be a way to change the detection into a rejection...