Sophos proxy internet access / firewall rules

Plenty of people want to do this.   If you poke around in the forum, you should be able to find your answer.

But can you answer my question, posted almost a year ago, about why this is a useful thing to do?

community.sophos.com/.../376015

I'm just trying to be a bit more anonymous on the web, leaving a small footprint as possible to make it harder for search engine operators / big data collectors to collect data about me. I believe this is a personal right, which is now very much supported here in Europe by the GDPR Directive.

Of course, you do not know if the data will be shared by the VPN provider, but it's the same when you go out of your house, you lock your front door, trying to protect yourself, and not leaving the door easy for everyone to be open.

I think the big data people are not very dependent on your IP address.   Every time you choose a store to place an order you give them implied information about your location as well as your email address.   I assume that they do not provide your email address to their tracking partners -- not because they care about your privacy, but simply because they consider their customer list to be proprietary information.   So instead they create (or reuse) a token that represents you, and achieve the same result.

Here is my understanding of why this works:

• The first site passes something that represent you to the tracking site as it invokes the tracking site as an embedded page element.
• The tracking site drops a cookie that represents you, which it is allowed to do because it has been referenced in the master web page.
• The next site you visit uses the same process.   
• The tracking site has access to all the cookies it created from all participating websites, so it can match the "you" on site 1 to the "you" on sites 2 through N.

Your IP address is part of this data collection, but they are interested in tracking you at home, at work, and on your cell phone.  So I don't thin the IP address matters much, and I don't trust the anonymizers.   Servers cost money, so they are either using your data or they are plotting something against you (or your country).   I just don't believe there are any charities out there.

On the other hand, UTM is actually a pretty powerful tool for fighting the tracking people, but it requires some work.    The general process is:

• Implement https inspection so you know everything possible about the invisible links that are used in the websites you access.
• Analyze the logs to separate the sites you recognize from the sites that are not recognizable.
• Look up the unrecognized FQDNs to identify which ones are tracking sites.
• Create blocks for the tracking site domain.

The biggest problem with this is that UTM does not provide a good enough tools for parsing the web logs to this purpose.   I parse my logs into a SQL database.   An introduction to the process is pinned to the top of the Reporting sub-forum.    My web log parsing is more complex than the examples in that post, so it is not posted, but I will provide it on request for those who get the basic stuff working.

All of this may be too intimiding for a home user, but it can work. 

SQL Express is free, but you will need to be able to write your own SQL queries for analyzing the information once it is in database format. 

Follow-up:

I started using the process I described above, and as a result I block these four domains:

• adnxs.com
• demdex.net
• dotomi.com
• hotjar.com

It is easy enough to confirm that a site is a tracker once you have the domain name.   Just google the name, and the browser will give you a summary of the vendor's webpage without the risk of navigating to the page itself.   The real tracking sites are big business, so there websites are safe.

Eventually I gave up on the process.  All along, my fear was that the blocked sites would have a visible impact on the user experience.    This was aggravated when I learned of one tracking site that is implemented as a content delivery network -- you get their images on your favorite website, they get you in trade.   Don't remember the name, but it "took the wind out of my sails".

As explained in my Web Filtering Lessons Learned post, the recommended way to block these companies is to create a website exception for the domain (with the checkbox enabled for including subnets).   Apply a tag to the exception and name it "Tracking Sites Blocked".   Then in your Filter Actions, specify that sites tagged with "Tracking Sites Blocked" are always blocked.

I think in my environment it is important to have the UTM configured correctly on a technical level. A good tip from you was to implement https inception, but I have not really come up with a good description of how to set this up correctly, that not so much cert errors popup.

Enabled Decrypt and Scan, re-generated the Signing CA in the Filtering Options under HTTPS CAs and exported it as a PKCS # 12 certificate with password. Then imported under Windows and MAC. I get a lot of SSL errors, some websites work, but many do not. Also it makes a difference if I use Safari, Edge, or Mozilla Firefox.

Is there a best practice guide to getting the decrypt and scan option up and running, also for a Home User?

Thank you
Sally

For Apple, there is an extra step for activating a root certificate.   See this link.

https://community.sophos.com/kb/en-us/126895

Firefox does not use the Windows certificate system.   Instead it is part of each user's personal profile, which will float between PCs if you use the option to log in.   This may be cute for some purposes, but it is one of several things that makes Mozilla products unsuitable for use in a business.   On my version, the click sequence seems to be:

• Gear icon (or about:preferences)
• Security section... Certificates sub-section... View Certificates button
• Authorities tab
• Import button

Each logon account needs to repeat this process.

For Chrome and IE, my guess is that you loaded the wrong certificate or loaded it into the wrong certificate store.

Wrong certificate?

• UTM has multiple root certificates, one for Proxy, one for VPN, one for something else.   Make sure you exported the Proxy CA.

Wrong location?

• There are certificate libraries available for each users, each services, and the whole computer.   You probably want it in the computer context so that it applies to every login account.     
• Start... Run... MMC.exe
• Add/Remove Snap-In... Certificates... Computer Account...  Local Computer
• Look for the Proxy CA certificate in the Personal... Certificates and Intermediate Certificates... Certificates path.  If found, delete it.
• Import the certificate again, and choose the option to let Windows choose the certificate store (recommended), or manually choose the Trusted Certificate Authorities folder.

To make things worse...

UTM exports the root certificate with the private key included, which is why it insists on a password.   This is bad, because it makes every device where it is installed into a potential certificate issuer.   Exploiting the problem would be tricky, but it is a risk that should be mitigated.   The root certificate does not need the private key to do what it needs to do on the client devices.   Sophos should provide a button to export without the key; sometimes they do not seem to appreciate the need to configure things to be "secure by default".

To remedy the problem:

• Once you have it loaded successfully on a Windows PC, right-click on the certificate and choose Export.
• Follow the steps in the wizard, but uncheck the option for exporting the private key.   Also uncheck the option about deleting the certificate when export is complete.
• Export the certificate to a new filename.  You will not need a password.
• Remove the signed certificate from your desktop devices, then Import the no-password version of the certificate back into them.

Hi Sally,

I edited your post and removed all of the extra blank lines so that I could read what you posted.  Just because you know what you said doesn't mean it's immediately clear to the rest of us.

Doug's Wiki post, Securing and Configuring Web Filtering, might help you understand the difference between the use of the "Web Surfing" group and 'Allowed Target Services'.

A Generic Proxy is not what you want.  From the context-sensitive on-line Help:

A generic proxy, also known as a port forwarder, combines both features of DNAT and masquerading, forwarding all incoming traffic for a specific service to an arbitrary server. The difference to standard DNAT, however, is that a generic proxy also replaces the source IP address of a request with the IP address of the interface for outgoing connections. In addition, the destination (target) port number can be changed as well.

Cheers - Bob
PS After all, this is a question about Web Protection, so I'll move it to that forum.  Let me know if you want me to move it back here.

Hi BAlfson,

thank you for sending me the article, and the information regarding the generic proxy. I have yet https decrypt / scan working with transparent proxy. When downloading the Proxy CA with PKCS#12 / Password and import it to Iphone, i was still getting ssl errors on the iphone. Then i downloaded the proxy ca cert as pem and imported it to iphone now its working. For Firefox i had to import the cert manually till the ssl errors where gone. Now browsing with Edge, Firefox, Chrome on MAC / Iphone / Windows is working...:)

Now i read that the most secure option would be, use standard Proxy and as Fallback Transparent Proxy. I went to Web Filter Profiles - created new Profile with Standard Mode - Authentication None https decrypt and scan enabled. 

Added then the UTM IP / 8080 to Firefox Network Proxy Manual, and can access Internet, works.

How do I have to configure the Fallback Option, is Fallback like if I don’t enter the Proxy Config to the Browser, that the devices then anyhow access Internet via Transparent Profile, or where can I set this?

Thx

Sally

I usually set the Default Profile in Transparent mode and make it more-restrictive, but that up to you.  Yes, if the browser isn't configured to use the Proxy, it's requests will be handled by the Profile in Transparent mode.

Cheers - Bob

If in this configuration with Standard Proxy and Transparent Proxy some application not work, like Update or cannot reach direct a server outside, will be the first step to check on the Transparent Proxy and define an exception there?

For example Youtube Videos are not working on iPhone 6. Checked with Youtube App and Website. When starting the Videos I get an error message, saying I should try again later. Youtube on the PC is working. Checked here in the Forum and added this Exception to the Transparent Proxy:

Skiped all Checks: Authentication / Caching / Block by download size / Antivirus / Extension blocking / MIME type blocking / URL Filter / Content Removal / SSL scanning / Certificate trust check / Certificate date check / Accessed pages / Blocked pages / Do not display Download/Scan progress page

Exceptions:

^http://[A-Za-z0-9.-]+\.googlevideo\.com/videoplayback
^http://[A-Za-z0-9.-]+\.youtube\.com/videoplayback
^http://[A-Za-z0-9.]+\.googlevideo\.com/videoplayback
^http://[A-Za-z0-9.]+\.youtube\.com/videoplayback
^http://[a-za-z0-9.-]+\.youtube\.com/videoplayback
^http://[A-Za-z0-9.]+\.gdata\.youtube\.com/
^http://[A-Za-z0-9.]+\.googlevideo\.com/
^https?//[A-Za-z0-9.]*youtube\.com
^https?://[A-Za-z0-9.]+\.googlevideo\.com
^https?://[A-Za-z0-9.]+\.googlevideo\.com/videoplayback
^https?://[A-Za-z0-9.]+\.youtube\.com/videoplayback
^https?//[A-Za-z0-9.]+\.youtube\.com
youtube.com
https?//youtube.com

But still, the youtube videos on iPhone are not playing. Then I deactivated the Standard Proxy, still same result. But when switching back from decrypt / scan to only url filtering the videos work..

Also what for me is not yet clear is, how can I force the Standard Proxy to all devices? When I see under Filtering Options - Misc - Auto Proxy Config , do I just have to mark Enable Auto Proxy Config, to push this to all the Clients? 

Thx

Sally