Web filtering: differences between Web policy of None and Web policy of Allow All

Hello David Hay ,

Thank you for reaching out to the community, with the "Allow All" option you'll be able to see logging on the under the user access logs if you use "none" instead Firewall logging will not happen. 

Hi Vivek

Thank you for responding, but I have to disagree with you. about one thing. With Web policy set to None, as long as the rule is configured to Log firewall traffic, then the traffic through the rule will be logged and visible in the Firewall logs. And my question is not related to whether logs are written depending on whether the Web policy set to None or Allow All. It is differences in what would be causing a request that is successful when the policy is None, but fails when it is set to Allow All, and the failure is not logged in the Firewall or Web Filter logs.

Regards

David

Hey David Hay ,

By logs, I meant the logs under the /log/awarrenhttp_access.log 
For instance if you set it to none, you'll not see website browsing logs under the webfilter as well as /log/awarrenhttp for the site you browsed, but if I keep allow all and browse to facebook.com I'll be able to capture the logs [as it goes through the proxy] as seen under the /log/awarrenhttp.log
for e.g - 
1668686403.140310167 [ 7515/0x7f323aae7400] fwid=5 fwflag="VN" iap=1 aap=0 conn_id=1797583488 id="0001" name="http access" action="pass" method="CONNECT" srcip="192.168.97.104" dstip="31.13.79.35" user="administrator@sophos.creed" statuscode=200 cached=0 trxlen=581 rxlen=2411 url="">https://www.facebook.com/" referer="" type="" upload_file_name="" upload_file_type="" download_file_name="" download_file_type="" authtime=0 dnstime=2 cattime=141509 avscantime=0 fullreqtime=6216273 ua="" activity="" av_transaction_id="" categoryname="Social Networking" category="67" app_id=0 app_name="None" app_cat="None" exceptions=""

If there is a site works fine, with the option none, meaning there is no restrictions what-so-ever, working as a normal direct ISP connection to a laptop. With the allow all, a web proxy/DPI comes into the picture depending upon the option you enable it or not "Use web proxy instead of DPI engine" and with that "allow all" option if a site is not working meaning FW proxy/DPI is intervening the traffic and needs to be diagnosed further to understand the root cause !! 

Long Story Short - None means it will not be processed by proxy/DPI whereas Allow All means it will be go through proxy/DPI.

If web policy is None and Malware scanning is unselected, you cannot turn on proxy mode (WebAdmin will not allow you).  Basically, if you don't want the XG to do anything then it forces it into DPI mode.  If for some reason you want it to go through proxy mode then you'll need to set Allow All.

In DPI mode, there are several reasons that the XG may want to interpret the HTTP.
Any web policy except None will cause DPI to look at HTTP.
Malware scanning will cause DPI to look at HTTP.
ATP will cause DPI to look at HTTP.

When DPI looks at HTTP it will enforce the HTTP specification (traffic must conform to what it cab process) and it will log (in Web Filter logs, which will also power reports).  But if there is no reason to look at the HTTP at all then it won't.  So if web policy is None (and no malware or ATP) then web-in-snort DPI will not try to interpret the HTTP at all, will not enforce HTTP spec, and will not log web traffic.

I am not positive, but that might then enable FastPath for that traffic.