I would have expected to see something else in the logs.  What happens if you make an Exception for SSL scanning for ^https://www.hotel-icon.com/

Cheers - Bob

Hi Bob,

When add to Exception for SSL scanning, it is work!!! 

But the website "https://www.bensound.com" and https://web.whatsapp.com are not working in this "Exception" method, even it skips all checks.

The "bensound" website can't play the music and "web.whatsapp" website can't generate the QR CODE or connect when the "Decrypt and scan" is enabled.

(I already add a whitelist "^https?://[A-Za-z0-9.-]*\.whatsapp\.com/ws")

here is the whatsapp Log:

018:12:15-12:28:52 httpproxy[13789]: id="0002" severity="info" sys="SecureWeb" sub="http" name="web request blocked" action="block" method="GET" srcip="10.77.192.196" dstip="31.13.95.63" user="" ad_domain="" statuscode="504" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="2537" request="0xe1971000" url="https://web.whatsapp.com/ws" referer="" error="Timeout while reading response from Server" authtime="0" dnstime="0" cattime="116" avscantime="0" fullreqtime="61167990" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) WhatsApp/0.3.1649 Chrome/66.0.3359.181 Electron/3.0.0 Safari/537.36" exceptions="" reputation="trusted" category="122" reputation="trusted" categoryname="Instant Messaging" application="whatsapp" app-id="598"

When you see statuscode="504", if an exception for SSL Scanning doesn't solve the problem, you will have to skip the Proxy for the access.  fullreqtime="61167990" is another indication that whatsapp isn't "happy" being proxied with scanning active - we also see exceptions="", so there's something amiss with any Exception you created for this access.

Cheers - Bob

BAlfson said:

When you see statuscode="504", if an exception for SSL Scanning doesn't solve the problem, you will have to skip the Proxy for the access.  fullreqtime="61167990" is another indication that whatsapp isn't "happy" being proxied with scanning active - we also see exceptions="", so there's something amiss with any Exception you created for this access.

Cheers - Bob

 

Hi Bob, 

Where UTM sets to "skip the Proxy for the access"?

Following is my setting:

(Exceptions Setting)

Error:

(The QR Code can't be Generated)

Log (From 09:41:17 to 09:44:58, all related to whatsapp):

2018:12:17-09:41:17  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="169.60.79.31" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xca5e6000" url="w3.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="129" avscantime="0" fullreqtime="228510" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="neutral" category="122" reputation="neutral" categoryname="Instant Messaging"

2018:12:17-09:41:18  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="169.44.82.118" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xd3c34800" url="w4.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="88" avscantime="0" fullreqtime="184984" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="neutral" category="122" reputation="neutral" categoryname="Instant Messaging"

2018:12:17-09:41:22  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="158.85.224.171" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xe177d800" url="w5.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="114" avscantime="0" fullreqtime="217695" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="neutral" category="122" reputation="neutral" categoryname="Instant Messaging"

2018:12:17-09:41:36  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="169.60.79.31" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xdecb8800" url="w6.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="122" avscantime="0" fullreqtime="227099" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="trusted" category="122" reputation="trusted" categoryname="Instant Messaging"

2018:12:17-09:41:53  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="169.60.79.31" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xd20b0000" url="w7.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="75" avscantime="0" fullreqtime="215451" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="trusted" category="122" reputation="trusted" categoryname="Instant Messaging"

2018:12:17-09:42:22  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="169.60.79.31" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xe1faa000" url="w8.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="89" avscantime="0" fullreqtime="227029" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="trusted" category="122" reputation="trusted" categoryname="Instant Messaging"

2018:12:17-09:43:01  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="169.60.79.31" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xdb27c800" url="w6.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="143" avscantime="0" fullreqtime="227981" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="trusted" category="122" reputation="trusted" categoryname="Instant Messaging"

2018:12:17-09:43:03  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="169.60.79.31" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xe1ad1800" url="w7.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="81" avscantime="0" fullreqtime="253313" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="trusted" category="122" reputation="trusted" categoryname="Instant Messaging"

2018:12:17-09:43:08  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="169.60.79.31" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xdf874000" url="w8.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="81" avscantime="0" fullreqtime="281190" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="trusted" category="122" reputation="trusted" categoryname="Instant Messaging"

2018:12:17-09:43:37  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="158.85.224.174" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xdbe47000" url="w2.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="101" avscantime="0" fullreqtime="201452" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="trusted" category="122" reputation="trusted" categoryname="Instant Messaging"

2018:12:17-09:44:09  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="169.60.79.31" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xda442000" url="w3.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="147" avscantime="0" fullreqtime="215269" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="neutral" category="122" reputation="neutral" categoryname="Instant Messaging"

2018:12:17-09:44:58  httpproxy[13789]: id="0001" severity="info" sys="SecureWeb" sub="http" name="http access" action="pass" method="GET" srcip="10.77.192.196" dstip="169.44.82.118" user="" ad_domain="" statuscode="400" cached="0" profile="REF_DefaultHTTPProfile (Default Web Filter Profile)" filteraction="REF_DefaultHTTPCFFAction (Default content filter action)" size="0" request="0xddeb5000" url="w4.web.whatsapp.com/ws" referer="" error="" authtime="0" dnstime="0" cattime="145" avscantime="0" fullreqtime="202210" device="0" auth="0" ua="Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36" exceptions="ssl" reputation="neutral" category="122" reputation="neutral" categoryname="Instant Messaging"

Thanks!

Hi All,

I success now! 

After add all w(2-8).whatsapp.com to "Skip transparent mode destination hosts/nets". It will return to normal! Basically, I don't know Why!

But I want to add every website one by one when it has "This site can't be reached" warning. I will crazy!

Edit 2019-01-04: As darrellr points out below, this REGEX can be improved.  See MichaelDunn's post just below darrellr's for the preferred syntax. - Thanks, guys!

To skip SSL scanning for WhatsApp, Perry, the only thing you need in your Exceptions is the following:

^https?://[A-Za-z0-9.-]*whatsapp\.com

This means the match starts at the beginning (^), is either http or https (?) and has zero-or-more (*) combinations of all of the characters inside the brackets ([]).  That will include all of the individual ones you listed.

Cheers - Bob

Hi Bob,

Thank you for your teaching about the symbol. It is helpful for me.

UTM has multiple ways to specify an exception, and it is useful to learn all the options. 

I do not like to use Regular Expressions unless absolutely necessary.  It is too easy to include matches that you do not want, or to exclude matches that you do want.   

Most of my exceptions are at the organization level, as in this example, where you want to exclude *.whatsapp.com   For these situations, website exceptions can be used to eliminate the need for regular expressions:

Go to Web Protection.. Filtering Options... Websites.

• Click the [New] button
• To exclude organizations:  Enter one or more domain names, such as "whatsapp.com", and check the box for "Include Subdomains"
• To exclude specific host names:  Enter one or more FQDNs, and leave the box unchecked.
• In the Website Tags box, create a new tag for "HTTPS Scanning Bypass" and verify that it appears in your selected tags list.
• Save your changes.

(This will actually create one object for each entry in the list, but you get to create them all at once.)

Then Go to Web Protection... Exceptions...

• Create a new exception
• Check the box to exclude "SSL Scanning"
• At the bottom, choose "Going to websites tagged as".   Use the folder icon to select the "HTTPS Scanning Bypass" tag that you created earlier.

Hi DouglasFoster

   Thank you for sharing your experience.

Thanks again!

Perry

Hi DouglasFoster

   Thank you for sharing your experience.

Thanks again!

Perry

WhatsApp has had numerous problems over the years in both XG and UTM.  Search the forums for more.

https://community.sophos.com/products/xg-firewall/f/firewall-and-policies/100545/whatsapp-issue-with-web-protection/365232#365232

Issues with sites other than WhatsApp should be dealt with separately from other issues.

I did a quick test and had no issues with www.bensound.com or www.hotel-icon.com

I did note that in your log it had method CONNECT.  That is either a log from when Decrypt and Scan was turned off, or it is log line that the initial TLS connection / handshake is not working.

Quick question:  Are you using standard or transparent mode?