Uncategorized sites and CDN's

Hi Tom,

Many people block uncat, normally this is not a problem however sometimes content will fail to fill in sites.  For example cnn only hosts their basic page/css on their domain.. Nearly everything else is hosted by regional cdn providers.  In this case most of all of those cdns are identified, but there may be cases where smaller sites pull content from an unidentified site.

99% of the time there are no issues with it and the only real visible effect is the page may have missing content.  In terms of .tld sites been blocked, its a good bet the vast majority of them are categorised. 

In that case when facebook.com shows as uncategorised there are a few different things to check.

#1 - as per the below notice for opendns users, this scenario can happen (granted not often) where an ISP or DNS provider will see all these DNS requests and assume its malicious.  Their answer to stopping them is to resolve it to 127.0.0.1.  effectively causing lookups to fail resulting in a blocked web page.  If this happens the only solution is allowing uncat or changing DNS until the issue is resolved.

https://community.sophos.com/products/web-appliance/f/web-appliance-hardware-or-virtual/90720/issue-sophos-web-appliance-categorization-failing

#2 - this could be caused by stale policy on the full web control machine.  I would have a quick run through my KB on web control policy, ensure the policy test is correct and do the troubleshooting with the workstation to determine if the issue is with the web appliance or workstation.

https://community.sophos.com/kb/en-us/122384

#3 - the last test would be to try and resolve the servers directly, so nslookup sxl31.sophosxl.net or similar, it should resolve to an external ip address.   If those fail, you may have upstream IPS or similar that is casuing the look up to fail.  The Web appliance should be exempt from any upstream IPs or additional web proxies.

Hi Tom,

Many people block uncat, normally this is not a problem however sometimes content will fail to fill in sites.  For example cnn only hosts their basic page/css on their domain.. Nearly everything else is hosted by regional cdn providers.  In this case most of all of those cdns are identified, but there may be cases where smaller sites pull content from an unidentified site.

99% of the time there are no issues with it and the only real visible effect is the page may have missing content.  In terms of .tld sites been blocked, its a good bet the vast majority of them are categorised. 

In that case when facebook.com shows as uncategorised there are a few different things to check.

#1 - as per the below notice for opendns users, this scenario can happen (granted not often) where an ISP or DNS provider will see all these DNS requests and assume its malicious.  Their answer to stopping them is to resolve it to 127.0.0.1.  effectively causing lookups to fail resulting in a blocked web page.  If this happens the only solution is allowing uncat or changing DNS until the issue is resolved.

https://community.sophos.com/products/web-appliance/f/web-appliance-hardware-or-virtual/90720/issue-sophos-web-appliance-categorization-failing

#2 - this could be caused by stale policy on the full web control machine.  I would have a quick run through my KB on web control policy, ensure the policy test is correct and do the troubleshooting with the workstation to determine if the issue is with the web appliance or workstation.

https://community.sophos.com/kb/en-us/122384

#3 - the last test would be to try and resolve the servers directly, so nslookup sxl31.sophosxl.net or similar, it should resolve to an external ip address.   If those fail, you may have upstream IPS or similar that is casuing the look up to fail.  The Web appliance should be exempt from any upstream IPs or additional web proxies.

Thank you this its really helpful i'll take a look at your suggestions and post my results. 

Again thanks for the response. 

Tom