Sophos UTM offers mighty tools to block a huge number of advertisers, webtrackers, analyzers and other stuff running in the background you usually do not want to have for traffic, performance, privacy or annoyance reasons.
However - a warning beforehand. Blocking those advertisers, trackers, adblock detectors etc. may brick specific sites, or limit functionality in unexpected ways. But for me it works quite well, maybe you will need some finetuning by allowing some services for some reason (or add more to block more). So don´t blame me, if those steps breaks your favourite sites [:D]
Technically you have lot of features to use. Mainly those are:
- URL Filter / Categorization
- Application Control
- Manual recategorization of some annoying "Internet Services"
- Manual Blacklists (where you shouldn't have to add a lot anymore ;o)
The theory behind speedup of webbrowsing due blocking annoyances is simple. Marketing people of every companiy loves to track and analyze the usage of their website using webstats, trackers etc. Free Websites love to add ads to their sites to generate revenue.
Basically I'm not against decent use of ads, as it helps to finance free content in the web (or free apps on your phones, tablets etc.). Sadly there is no more strict border between "acceptable ads" and annoying the user, because a website becomes overloaded by ads. I don't even start to talk about those free apps/games used by my children, where always and everywhere pops up those in-app ads, and I have to take care, that the don't open them or buy by accident something. In the past many years ago you had a simple ad banner in the top, which was acceptable for me. Today that crap is embedded everywhere within websites, which annoyes me - Facebook is a nice "bad example". I hate facebook in the meanwhile (besides of all other privacy issues with it's regular privacy changes).
However: Every website opens in the background connections to all those trackers, analyzers, advertiser content servers, which slows down websurfing, as:
a) A webbrowser uses a limited number of concurrent connections to a webserver. Each connection which downloads "unnecessary" content will delay the download of the wanted, useful content
b) Unwanted stuff generates additional traffic and load on the UTM
c) It those trackers, analyzers or advertisers for some reason are slow, they will put additional delay to your surfing experience, how fast a website is loaded and displayed in the web browser
Here's a small guide to get rid of a lot of those trackers, analyzers and advertisers. I still use in my webbrowsers additional adblocker apps, which will play as "afterburner" and rip the remaining annoyances out of the websites, but with mentioned methods it becomes already quite performant and ad free also without such "afterburners" ;o)
Requirements:
- The UTM's Web Proxy has to be used
- Application Control has to be used
- As some ads and trackers work via HTTPS, HTTPS scanning will enhance filtering - especially within HTTPS websites, but the newly introduced "URL filtering only" https scanner in UTM9.2 also already should do lot good for you too ;o)
First - Let's start with the easy part:
- Block with the URL filter in the webproxy the "web ads" category. Already does lot of filtering for you...
Second - little more time required:
Build a new block rule in the application control. All the applications you want to block are found in the category "web services". Sadly there is no separate category for advertisers, trackers and analyzers - I openened a feature request for that, feel free to add your votes here ==>
Enhance Application Control App Categorization
However, there are also other services in that category as CNET or Mozilla download servers, CDN's as Akamai and other stuff you may not want to block, so you have to crawl through that "web services" list by yourself and check in the info/description of the application, if it's really a tracker, analyzer or advertiser. You finally may find >150 such applications which fit into those categories (didn't count them exactly). So create a block rule for applications (not groups), filter for "web services" and search for following terms, which should bring up most of those entries fast:
"ads"
"track"
"analy"
But always check the application description, as not all advertisers have a speaking "ads" in the name as example, and otherwise there are also allowed sites as "CNET" or "MOZILLA", which you may not want to block ;o))
Advertiser descriptions usually describe something like that here: "Visiting websites that use spoke (formerly Telecom Express) to generate ads." or "Visiting websites that use Webtrends to generate ads and collect user analytics."
Third - finetuning by recategorizing unwanted domains using the UTM9.2 "Websites" option to regategorize such site locally from category xyz to "web ads"
This can be done by yourself by monitoring ad traffic - in my case for example especially from my mobiles apps in-app ads. This list below may not fit for everyone, but it's a good start. my recategorized list of domains ("incl. subdomains" checked too) is:
EDIT 07-10-2014 - Added more entries to list:
4seeresults.com
ad.leadboltads.net
adadvisor.net
addthis.com
ads.mopub.com
adserver.idg.de
amazon-adsystem.com
analytics.twitter.com
appads.com
applifier.com
applovin.com
appsdt.com
appsher.com
appspot.com
bestofmedia.com
brightcove.com
bullbitz.com
crittercism.com
crowdscience.com
data.flurry.com
deployads.com
disqus.com
dsply.com
eloqua.com
etracker.com
etracker.de
flurry.com
formalyzer.com
gigya.com
glpals.com
google-analytics.com
googletagservices.com
graph.facebook.com
guruads.de
iadsdk.apple.com
inmobi.com
insightera.com
kissmetrics.com
kochava.com
kraken-measurements.s3.amazonaws.com
localytics.com
maxymiser.net
meetrics.net
msads.net
nmetrics.samsung.com
And those here below should be imported as URLs without "including subdomains"
m.youtube.com/_get_ads
csi.gstatic.com/csi
apps.skype.com/adcontrol/
api.skype.com/configuration/sections/ads-in-client
Fourth - Manual Blacklists
If ads or other annoyances are not domain based delivered, but by a path within a generally allowed URL, you may create manual blacklist entries for those specific paths in the web proxies filteractions. I didn't use it up to now - but keep it in mind if you have to...
Hope this helps one or the other UTM admin to speed up things even more and get rid of some annoyances in webpages. I'm happy about every feedback here in that thread, or about every vote in the feature portal for those "new" application control categories "web advertisers", "web trackers" and "web analyzers", which would ease things to block such applications quickly.
However - my experience by blocking those annoyances is quite positive up to now, and I hope lot of other people also can benefit by this mighty featureset to optimize their surfing experience.
/Sascha [H]
BTW:
Troubeshooting also shouldn't be too hard, as every block will be logged in the http.log, and the method used to block is also mentioned in the log as:
"web request blocked, forbidden category detected" ==> URL Filter Web Proxy
"web request blocked, forbidden application detected" ==> Application Control
2014:02:22-14:45:33 asg01 httpproxy[18869]: id="0066" severity="info" sys="SecureWeb" sub="http" name="web request blocked, forbidden application detected" action="block" method="GET" srcip="192.168.10.208" dstip="" user="" statuscode="403" cached="0" profile="REF_HttProContaLanclNetwo4 (LAN_CLI)" filteraction="REF_LmvZxpuYeo (LAN_STD)" size="3205" request="0x14e8aee0" url="www.google-analytics.com/ga.js" exceptions="" error="" authtime="0" dnstime="0" cattime="0" avscantime="0" fullreqtime="1224" device="0" auth="0" country="United States" application="GOOGANAL"
2014:02:22-14:45:33 asg01 httpproxy[18869]: id="0060" severity="info" sys="SecureWeb" sub="http" name="web request blocked, forbidden category detected" action="block" method="GET" srcip="192.168.10.208" dstip="" user="" statuscode="403" cached="0" profile="REF_HttProContaLanclNetwo4 (LAN_CLI)" filteraction="REF_LmvZxpuYeo (LAN_STD)" size="3278" request="0x1d3bc880" url="20minde.wemfbox.ch/.../home
This thread was automatically locked due to age.
