Question for excepting certain destination on the web filtering

Hi,

do you have a firewall rule that will allow this site to be accessed?

I suspect you need to add something like this *.google.com.*

Hi,

Can you please show us a picture of the configuration and the definitions added to the skip list. Alongside, show us few log lines from http.log that reflects the drop or block.

Refer, Sophos UTM Logfile information.

Thanks

Hi, 

Below is a configuration for "Skip Transparent Mode Destination Hosts/Nets". 

i tried to access www.npr.org, when i tested a exception.

I expected that any logs were not generated because www.npr.org set as an exception. 

But,  when i accessed the www.npr.org, below logs were generated.

=== http.log ==

2017:07:11-10:54:44 sg httpproxy[18755]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="xxx.xxx.xxx.xxx" dstip
="52.73.85.83" user="" group="" ad_domain="" statuscode="302" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFA
ction (Default content filter action)" size="0" request="0xddc06a00" url="soma.smaato.net/.../idsync
erId%3DSomaCookieUserId" referer="20501671p.rfihub.com/ca.html
tp%3A%2F%2Fwww.npr.org%2F2017%2F07%2F10%2F536533586%2Fchristie-blasts-n-j-caller-i-love-getting-calls-from-communists-in-montclair&pf=&ra=7047619021825811" error="" aut
htime="0" dnstime="112725" cattime="230298" avscantime="0" fullreqtime="777155" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHT
ML, like Gecko) Chrome/59.0.3071.115 Safari/537.36" exceptions="" category="178" reputation="neutral" categoryname="Internet Services" country="United States"

2017:07:11-10:54:44 sg httpproxy[18755]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="xxx.xxx.xxx.xxx" dstip
="103.71.26.126" user="" group="" ad_domain="" statuscode="302" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCF
FAction (Default content filter action)" size="0" request="0xddcd7800" url="sync.search.spotxchange.com/partner refere
r="20501671p.rfihub.com/ca.html
F07%2F10%2F536533586%2Fchristie-blasts-n-j-caller-i-love-getting-calls-from-communists-in-montclair&pf=&ra=7047619021825811" error="" authtime="0" dnstime="78491" catti
me="229016" avscantime="0" fullreqtime="501606" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071
.115 Safari/537.36" exceptions="" category="154" reputation="trusted" categoryname="Web Ads" country="Singapore" application="roketful" app-id="1003"

2017:07:11-10:54:44 sg httpproxy[18755]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="xxx.xxx.xxx.xxx" dstip
="54.230.255.150" user="" group="" ad_domain="" statuscode="302" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPC
FFAction (Default content filter action)" size="160" request="0x900f200" url="pdw-usr.userreport.com/.../rfl referer="http://20501671p.rf
ihub.com/ca.html?rfiidc=1038150097342898428&rfiaid=a862cfd1dad943b1ace9b9b56b2537ce&ver=9&rb=3035&ca=20501671&pe=http%3A%2F%2Fwww.npr.org%2F2017%2F07%2F10%2F536533586%2
Fchristie-blasts-n-j-caller-i-love-getting-calls-from-communists-in-montclair&pf=&ra=7047619021825811" error="" authtime="0" dnstime="244979" cattime="205326" avscantim
e="2065" fullreqtime="467326" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36
" exceptions="" category="178" reputation="neutral" categoryname="Internet Services" country="United States" sandbox="-" content-type="text/html"

2017:07:11-10:54:44 sg httpproxy[18755]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="xxx.xxx.xxx.xxx" dstip
="103.15.158.193" user="" group="" ad_domain="" statuscode="200" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPC
FFAction (Default content filter action)" size="42" request="0x900f800" url="p.rfihub.com/cm referer="htt
p://20501671p.rfihub.com/ca.html?rfiidc=1038150097342898428&rfiaid=a862cfd1dad943b1ace9b9b56b2537ce&ver=9&rb=3035&ca=20501671&pe=http%3A%2F%2Fwww.npr.org%2F2017%2F07%2F
10%2F536533586%2Fchristie-blasts-n-j-caller-i-love-getting-calls-from-communists-in-montclair&pf=&ra=7047619021825811" error="" authtime="0" dnstime="0" cattime="123" a
vscantime="1017" fullreqtime="213843" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safar
i/537.36" exceptions="" category="177" reputation="neutral" categoryname="Content Server" application="roketful" app-id="1003" sandbox="-" content-type="image/gif"

Please let me know if you need more information. 

Hi, 

www.google.com is just example for explaining. 

When i tested i tried to access www.npr.org. 

and i added a reject firewall rule for www.npr.org for blocking at the level of firewall. 

Thanks, 

You might want to reading my UTM architecture post in the Wiki section of this forum.  Short version:

When you bypass web filtering, the traffic is instead evaluated by the firewall rules, which is why the other reply said to ensure that the firewall allows traffic through it.

Other notes:

Many websites invoke content from other (seemingly unrelated) URLs.  So if your site does not load properly, you may need to check web filtering logs for dependent content blocks.

Your other option is to configure a webfilter exception which turns off specific rules without disabling webfiltering completely.

Note that www.example.com is not the same as example.com.   In the transparent host skiplist, you should specify example.com and www.example.com as DNS hosts for an exact match on host name.  If you want to exempt *.example.com, I think you need to specify a DNS group for example.com

Hello Degulas, 

Thanks for your help. 

I agree that what the most of main URL contains many other URLs to call the web page.

So I expected that some of contents were not displayed because of above reason.

But all contents were displayed when I accessed the www.npr.org.

Anyway...

I tried to test again with another simple web site (www.globaltelecom.co.kr)

It worked well when I added the url to the skip list.

(it was not generated the log in the http.log)

However, it is not blocked by the firewall rule after adding a firewall rule based on URL(DNS Group).

(Ultimately, I want to block certain IP ranges that contain web page using firewall rule.

This is a reason why i want to bypass for certain web page.)

And i can't see any relevant logs in the packetfilter.log.

When I turn off the web protection, i can see relevant logs in the packetfilter.log.

Because that traffics were controlled by the firewall rule.

As my result, If certain destination is bypassed on the web filtering, it will bypass firewall rule also.

Is it correct result?

Or do i have something wrong?

Thanks

Please show a Web Filtering log line with a block of something you want or a pass of something you wanted to block.

Cheers - Bob

Hello Bob, 

Any log were not generated after bypass for certain destination at the web filtering.

what i want to know is whether the certain destination is possible to block or not on the level of firewall after bypass the certain destination at the web filtering.

Yujin, If I understand what you are trying to ask, you are using the product in a very nonstandard way.

It may be better if you describe your ultimate goal, and we can tell you how to meet that goal.

Right now you've already decided on a solution and are asking for help in that solution but we're having trouble understanding what you are doing.

Note:  The transparent mode skip is in the Web section, however strictly speaking it is not a Web rule, it is a firewall rule.

When you configure the Web "Allowed networks" what you are really doing is creating a firewall rule that says "anything coming in on port 80 from this network going out to the internet, forward the packets to the Web Proxy".  When you create a destination skiplist what you are really doing is creating a firewall rule above that saying "anything going to port 80 to these IP addresses, do not forward the packet to the Web Proxy".  The skiplist must work on IPs (to UTM objects that resolve to one of more IPs).

Hello All,

Ultimately, my goal is that adding a firewall rule to block certain destination including web server at the level of firewall when web filtering is enabled.

== Scenario for my goal ==

• Issue: 1.1.1.10 is a web server. If I add a firewall reject rule for destination 1.1.1.0/24 , it is impossible to block 1.1.1.10 at the level of firewall because 1.1.1.10 is a web server.
• Goal: block for 1.1.1.0/24 at the level of firewall including web server
• SG firewall configuration: enable for firewall and web filtering.
• Expectation
  Adding a 1.1.1.0/24 to the transparent skiplist at web filter.
  I expected that if 1.1.1.0/24 is bypassed at the level of web filter, it might be possible to block for all 1.1.1.0/24 including 1.1.1.10 at the level of firewall.
• Result
  Clients behind the SG firewall can access the 1.1.1.10 even though it has a firewall reject rule for 1.1.1.0/24. Because web filter is enabled.
  That means web traffic is proxied.

I found a helpful article and then i could understand why i can't achieve my goal.
https://community.sophos.com/kb/en-us/115155 - Create A Basic Firewall (Packet Filter) Rule in Astaro Security Gateway

KB115155 explains the Proxied Services.
Based on the explanation of Proxied Service, it is impossible to control(allow or deny) web traffic at the level of firewall when web proxy is enabled.

Maybe, I tried to use nonstandard way like the Michael's mention.

Thank you very much for all for help me.