Upgrade from 9.3x to 9.401-11: 1) Application Control does not work anymore. 2) Web Protection breaks DNS lookups

Check your IPS logs - I began experiencing the same DNS issue after the upgrade to 9.354 and it persisted through 9.4.  I noticed an incredible amount of UDP flood entries in the IPS logs which I suspect are being caused by Google's experimental QUIC protocol.  I attempted to block QUIC through app control but had limited success.  Eventually, I had to whitelist some of the Google IP addresses that were being logged.  DNS issue cleared up immediately.  Please let me know what you find - I have a support case open on this issue.  Thank you.

I don't have anything in the IPS logs as you've suggested.

Apple have just released a later version of 10.11.5, which at the moment appears to fix a couple of the wifi issues, and this morning it's been a little more stable, although I'm seeing this in the Web Filter logs and it has frozen a couple of times, I'm not getting the WiFi disconnects, so think this is a UTM issue.

2016:04:21-12:03:37 phobos httpproxy[16820]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="dns_expire" file="dns.c" line="195" message="dns query cdad (safebrowsing-cache.google.com) timed out, retransmitting (retry 1)"

2016:04:21-12:03:47 phobos httpproxy[16820]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="dns_expire" file="dns.c" line="195" message="dns query 2383 (safebrowsing-cache.google.com) timed out, retransmitting (retry 1)"

2016:04:21-13:25:27 phobos httpproxy[16820]: id="0003" severity="info" sys="SecureWeb" sub="http" request="(nil)" function="dns_expire" file="dns.c" line="195" message="dns query 92ed (www.ebay.co.uk) timed out, retransmitting (retry 1)"

I'm going to try again with the pharming protection disabled and see what happens.

Been testing this weekend, not sure if this helps but it does appear that there is a bug with DNS resolution and the UTM 9.4 release.

Firstly, I use the UTM as a DNS server as well as a perimeter firewall, and configured to use the ISP forwarders.

With the DHCP configuration set as using the UTM as the DNS server, quite often I will get the errors indicated as above, and also running NameBench resulted in quite a few DNS lookup failures.

I then built a DNS server on a Windows 2012 R2 demo license, this was allowed through the firewall and configured clients to use that for DNS resolution.

Then I ran NameBench and this didn't report any DNS lookup failures, and the Macs on the network now always seem to resolve and display the pages correctly.......although.....

Still getting the http.log file still has the retransmitting errors in there with pharming  protection enabled, this hasn't been so much of a problem just means occasionally web pages take a little longer to load, but at least I'm not getting the dreaded page cannot be displayed any more.

Going by my tests, it would appear that there is an issue with DNS name resolution in 9.4 that needs fixing.

When you configured your clients to use Windows 2012's DNS, did you change the DNS forwarder in the UTM to use Window's DNS as well? 

When you configured your clients to use Windows 2012's DNS, did you change the DNS forwarder in the UTM to use Window's DNS as well? 

I had the UTM pointing to the Windows 2012 R2 server.

To ensure it wasn't my network etc, I used a 4G dongle connected to the server to route out any traffic going to google DNS through that path.

Taking the VDSL line totally out the equation for any DNS lookups.

Applied the update 9.402-7 this morning, and must say that I've not had any DNS issues since.

Hopefully this means that this issue is now resolved.

Keep us updated on any issues you find in 9.402-7 going forward. Thanks!

For what it's worth, I'm pretty sure the DNS issues aren't resolved yet.  I'm using the SG115w... is it possible that the 'w' version hasn't been fixed yet?  I actually experienced a DNS storm about 4 months ago that would've bricked the UTM if I hadn't noticed that the firewall log was growing at a rate of ~120MB every 10 minutes.  I blamed our VoIP provider and found a new one.  But I still had to stop using the UTM as a DNS proxy, and Namebench still timed out when testing the UTM.  I'm on 9409-9.

Brian, do you still have a sample line from the Firewall log file?

Cheers - Bob

DNS drops were also reported here

https://community.sophos.com/products/unified-threat-management/f/network-protection-firewall-nat-qos-ips/84469/dns-packets-dropped

I commented I was having similar issues. What has fixed it for me was re-doing the settings. I've removed the internal DNS servers from the "Allowed Networks" DNS and applied the setting. I then re-added them again, and applied. I also did the same for the firewall rule I had created. It hasn't failed in weeks  

Bob,

Unfortunately not, they were too large so I deleted them.

Looking at the post under mine and the thread it links to, I don't know that I ever "cleanly" recreated the forwarders and allowed networks, although I definitely deleted and re-entered the config at some point.  And even though I entered the networks in the "Allowed" window, I also explicitly permitted access to the DNS servers via a firewall rule.  At the time my primary DNS was from my ISP with a Level 3 server as backup.

Thanks,

Brian