Web Filtering Transparent Mode - Best Practice

Question

We have an SG310 and we have implemented Transparent Mode web filtering. When everything works well, users on the network can browse to safe websites. But occasionally the Sophos will block them from visiting a site that they have used before (and that we have deemed safe). As a workaround, we tell the user to browse to any non-https site, and then try again and this usually fixes the problem. 
 How/Why does this work as a quick fix? What's happening when the user goes to an http site (vs https)? 
 Is there a more elegant way to resolve these occasional authentication issues? And while we are on this topic, it looks like this just does not work for our Mac users. The Mac users end up enabling Wi-Fi on their Macs so that they can browse the Internet. Having that turned on in addition to the Ethernet on the corporate LAN causes weirdness. 
 Just wondering what's best practice for implementing Transparent Mode. 
 We tried using the standard proxy mode, but this quickly became a headache because every software app on the network that needed internet access had to be configured to point to the proxy server. (For example, UPS WorldShip or FedEx Ship Manager). This, plus our corporate policy that forces users to change their password every 90 days caused a lot of IT headaches. So I guess I am looking for the best of both worlds - a proxy but none of the manual work of using a proxy, and none of the glitches in transparent proxy.

DouglasFoster · Answer

Start with my post, which is pinned to the top of the web filtering sub-forum: Optimizing web proxy &ndash; Lessons Learned 
 Short version: 
 Do not consider Standard and Transparent as mutually exclusive. Instead use both. 
 Transparent mode has fundamental limitations, which are documented, and which you have discovered. It cannot perform transparent user authentication for https traffic, because the user information is in the encrypted part of the packet. As a workaround, it assumes the username associated with the last known http request from the same source IP is the username for the https request. If there is no previous http request, the request is handled as unauthenticated, and apparently you have configured a rule to block unauthenticated traffic. 
 I suggest reactivating Standard Mode with AD SSO authentication, plus Transparent Mode with None authentication, for the same source addresses. Your browsers will use Standard Mode with AD SSO. Most fat-client applications will not need any special configuration, because they will use transparent mode by default and will not be asked for authentication. There are a bunch of hidden operations like automatic updates for Java or Adobe, as well as operating system functions like error reporting, that use web protocols but may not work if authentication is required. 
 For best protection, fat client FTP applications should be manually configured to use Standard Web Proxy. I have another document in the Wiki section, written earlier, which discusses web proxy considerations including the three options for ftp proxy. 
 You will still have issues with a few sites and a few fat-client applications, but this should be manageable because you will have eliminated the most common sources of problems. 
 Side note: If an https site is blocked or warned, the CA root certificate needs to be deployed to the client device for the block/warn page to display without certificate errors. Many users assume that if they are not doing https inspection, they can skip this step.