User reports don't show hardly any of the traffic users are generating

Tim, it's hard to work without examples of what you're seeing...

In 'Reporting Settings', you can select the detail retained in Reporting.

If you have a firewall rule like 'Internal (Network) -> Web Surfing -> Internet : Allow', accesses can occur that don't transit the Proxy.  This can happen if Web Filtering is in Standard mode, but the individual client isn't configured correctly.  It can happen with HTTPS if the Proxy is in Transparent but is not inspecting HTTPS traffic.  It also can happen in Transparent if someone uses anything other than port 80/443 and there is a firewall rule allowing the traffic.

Cheers - Bob

There was a firewall rule like 'Internal (Network) -> Web Surfing -> Internet : Allow', so I disabled it. So I guess this indicates that the proxy component does not rely on the firewall for access.

Web Filtering was in Standard mode with 'Block access on authentication failure' disabled.

Created a new web filter profile scoped to my computer to test. No change when blocking enabled. Checked policies and found that a custom Filter Action > Additional Options > Activity Logging has "Log accessed pages" disabled. After I enable that more traffic started showing up in the logs.

Basically this was dropping all the authenticated traffic from the Web Filter log, which is apparently used for reporting. Going to let it run tomorrow with all users and see how it reports.

There was a firewall rule like 'Internal (Network) -> Web Surfing -> Internet : Allow', so I disabled it. So I guess this indicates that the proxy component does not rely on the firewall for access.

Web Filtering was in Standard mode with 'Block access on authentication failure' disabled.

Created a new web filter profile scoped to my computer to test. No change when blocking enabled. Checked policies and found that a custom Filter Action > Additional Options > Activity Logging has "Log accessed pages" disabled. After I enable that more traffic started showing up in the logs.

Basically this was dropping all the authenticated traffic from the Web Filter log, which is apparently used for reporting. Going to let it run tomorrow with all users and see how it reports.

Everything looked good today.

We are also evaluating Fastvue Sophos Reporter and noticed an omission. We can't get details on outbound file transfers. Fastvue support informed me that the Web Filter on Sophos does not report on uploaded data. The could be useful for tracking DLP and malware on the network. Anyone know if this is true?

DLP is done for Email, but not Web Filtering.  Web Filtering relies on Categorization and Reputation to block access to sites that accept stolen information.  Anti-Virus and Advanced Threat Protection should address questions of malware.

Cheers - Bob

Wow......

I'm so glad I come across this post. This is a very easy mistake to make when setting up the UTM.

Most users (including myself) will basically bring up the UTM with a lan and then a wan. To get access to the internet, you would then put rules eg dns & web browsing in the firewall rules.

Web filtering comes later once you know you are on the internet. And because you see things going through the logs etc, it is easy to assume that it's working and the initial dns/web browsing rules in the firewall would be easily forgotten about.

So ultimately, if you use the web proxy, dns proxy & smtp proxy, you don't need any of those rules in place on the firewall......

Maybe they should add some sort of warning or popup to say that you have both enabled and should disable one or the other eg a conflicts page on the UTM which lists questionable setups eg you have port 25 open on the firewall but the smtp proxy is enabled. You might want to review this?

It never once dawned on me that you get the lan going, the wan going and simply turn on web filtering and put your lan in the allowed networks to get to the internet. And I bet lots of others have done the same......

Careful with deleting the rules if you have hosts that are skipping the proxy, especially if running in transparent mode.  If they skip the proxy, and there are no rules allowing them to the internet, they will not have internet access.  I would definitely use a host group for each (the skip list and the firewall rule - haven't tried so assuming you can do this) to make sure you only have to make changes in one place.

I disabled the web browsing rule (not deleted) for all of our subnets and they are all still going through. Not had any complaints yet.

I've also enabled caching which does seem to have improved browsing performance with no complaints either.

Early days, but will keep you posted.

Louis, you might be interested in DNS Best Practice.

Cheers - Bob

Hi Bob,

yes we do that. I'm setting another UTM SG330 today so will try without adding any FW rules once we're connected and see how we get on.

It's just a break from the norm because most will be used to setting up DNS/Web browsing rules at the beginning.....

What a great idea!

I came across this when we were adding another UTM into the network. All web traffic was proxied and certain download sizes were in force.

The new UTM would not update as it was going through the proxy which was denying the download size.

Now the answer would be to put the new UTM in the skip host section on the proxy but you would also have to put a FW rule in to allow the new UTM to http.

So the great idea from darrellr is rather than adding hosts (the UTM in this case) to both the skip list and fw rules, simply creat a "Proxy skip Host Group" and add this to the skip hosts and fw rule. That way in the future, when you need a host to skip the proxy, you simply add it into the "skip proxy group" and voila......