High CPU usage (HTTPD) disrupts work

For the time being I've disabled the UTMs User Portal and things seem to be better. I'm not yet sure if this was the cause, however, since the connection issues were always intermittent... However I'm a bit worried that the portal, if enabled, would cause so much trouble...

Cześć Mateusz,

That's indeed strange.  I agree with your analysis.  Please post the solution here when Sophos Support fixes it.  I hope the fix isn't a re-imaging.

Cheers - Bob

I think I shot myself in the foot. Thinking I've managed to get a handle on the issue I've opened a low-priority support ticket. Unfortunately, the issues are still there and are interfering with regular work... And the low-priority ticket seems to be getting very little traction.

So I've tried calling the support number, and... I'm on hold for 20m listening to the awful jingle with no one answering. This does not fill me with confidence I'll get this resolved any time soon.

How large are the httpd logs?  I had an issue previously where a wifi user wasn't authenticated correctly and the log file was 20+ MB and spiked the cpu.  Can't remember if it spawned multiple processes or not.

Hmm, WebFiltering is 70MB... and Web Application Firewall is almost 450MB... O_o

Ok, the large log files aren't the problem (I've removed the large files and the problems persist). I've managed to SSH to the server and stop the HTTPPROXY service (using the script in /var/mdw/scripts) and the whole thing became responsive as soon as the operation was complete.

However, that doesn't really resolve the issue - now we don't have the proxy (duh).

However, I've went into the Web Application Firewall section and turned everything off... I'd expect the system to "chill out" with every "virtual webserver" disabled, right? However as soon as I turned the httpproxy service back on everything was dying once more. OK, so... what else is this service responsible for? 

EDIT: Apparently I'm a bit of an idiot - I've missed the 10-item limit on the Web application firewall page, and I did NOT, in fact, disable all the sites that we had.

Once I've disabled every item on the list and started the service - all was quiet. OK, so Proxy is at fault, but... why? I don't think we have that heavy traffic. We don't even have that big of an internet connection to begin with - we can't host huge quantities of data to customers all around the world - we have but a few connections AFAIK. We aren't even using the proxy in any really "heavy" manner either - it's all just redirection onto our own servers, with no filtering nor profiles applied... But the moment I enable anything that's supposed to handle external connections HTTPD processes get spawned in large quantities and start eating up CPU like crazy.

Hmm, WebFiltering is 70MB... and Web Application Firewall is almost 450MB... O_o

Ok, the large log files aren't the problem (I've removed the large files and the problems persist). I've managed to SSH to the server and stop the HTTPPROXY service (using the script in /var/mdw/scripts) and the whole thing became responsive as soon as the operation was complete.

However, that doesn't really resolve the issue - now we don't have the proxy (duh).

However, I've went into the Web Application Firewall section and turned everything off... I'd expect the system to "chill out" with every "virtual webserver" disabled, right? However as soon as I turned the httpproxy service back on everything was dying once more. OK, so... what else is this service responsible for? 

EDIT: Apparently I'm a bit of an idiot - I've missed the 10-item limit on the Web application firewall page, and I did NOT, in fact, disable all the sites that we had.

Once I've disabled every item on the list and started the service - all was quiet. OK, so Proxy is at fault, but... why? I don't think we have that heavy traffic. We don't even have that big of an internet connection to begin with - we can't host huge quantities of data to customers all around the world - we have but a few connections AFAIK. We aren't even using the proxy in any really "heavy" manner either - it's all just redirection onto our own servers, with no filtering nor profiles applied... But the moment I enable anything that's supposed to handle external connections HTTPD processes get spawned in large quantities and start eating up CPU like crazy.

Well it's a totally different issue than mine but that WAF log seems huge. I'm guessing you will need to go through the log file to see what's generating all the logging if that's the issue.