BUG - SSL/TLS Inspection seems to break Honeywell smart devices

I've noticed this behavior with both EAP 2 and now EAP 3. When SSL/TLS inspection is enabled Honeywell smart thermostats cannot connect to the internet. The only way they can connect is buy going the SSL/TLS inspection rules, SSL/TLS Inspection settings, advanced settings, and completely disabling the inspection (disable only when troubleshooting).

None of the default SSL/TLS inspection rules are configured to decrypt data.

Logs noticed in the webfilter when the issue is occurring is below.

2019-12-18 17:51:19Web filtermessageid="16002" log_type="Content Filtering" log_component="HTTP" log_subtype="Denied" status="" fw_rule_id="10" user="" user_group="" web_policy_id="13" web_policy="" category="" category_type="Acceptable" url="" content_type="" override_token="" response_code="" src_ip="x.x.x.x-internal IP" dst_ip="199.62.84.152" protocol="TCP" src_port="57393" dst_port="443" bytes_sent="0" bytes_received="0" domain="" exception="" activity_name="" reason="HTTP parsing error encountered." user_agent="" status_code="403" transaction_id="" referer="" download_file_name="" download_file_type="" upload_file_name="" upload_file_type="" con_id="0" app_name="" app_is_cloud="0" override_name="" override_authorizer="" used_quota="0"

  • In reply to Prism:

     if you make a firewall rule with LAN to LAN, disable everything, make an SSL rule to not decrypt, you will see the traffic counter increase on the SSL rule. This shows that snort is looking at the traffic. See below after a simple test.

     

    Maybe something is not right with my unit but snort is involved with my LAN to LAN traffic.

     

     

    EDIT: the second picture was taken before I saved the rule and shows decrypt as the action but the saved rule shows don't decrypt.

    Also  I am sorry man. I didn't mean to hijack this thread. I thought this was related, and it is in a way but it will be better handled in another thread.  let's move this over to https://community.sophos.com/products/xg-firewall/sfos-eap/sfos-v18-early-access-program/f/feedback-and-issues/117987/question---bug---dpi-appears-to-be-on-by-default as that is where Mike said it originated. 

  • In reply to MichaelBolton:

    But this is exactly what i said before, the new DPI engine is always running, since you have a LAN=>LAN Rule to not decrypt and also on Maximum compatibility, SSL/TLS Inspection will do nothing with the traffic, but of course It will still be logged.

    What I've told you before, If you DON'T want traffic to be SCANNED (IPS/AV/APP-WEB Policies being applied) on XG, all you need to do is have a Rule with no security profile on it (IPS/AV/App Classification.) and simply don't create a SSL/TLS Inspection rule to decrypt.

     

    As I've said  before, you can also do this test yourself, get a Web Server (Nginx), an certificate, and just throw a huge file to download over HTTPS (LAN=>LAN.)

    If there's no security profile on the Rule being used, and there's a "Do not decrypt, Maximum compatibility" SSL/TLS Inspection rule, you will pretty much see line-rate throughput with it, you can even SSH to XG and see it for yourself, Snort CPU usage won't even move without the security profiles being applied.

     

    Thanks!

  • In reply to Prism:

    Even without the SSL rule that shouldn't be doing anything, I can see the snort CPU usage increase when I transfer large files from LAN to LAN.

    Again, maybe something is not right with my unit, but if DPI is always on, that means snort is being used and snort cannot put up the numbers.

    I will default my test unit and test again.

  • In reply to MichaelBolton:

    Hi MichaelB,

    I think you and I are having trouble getting people to understand.

    See if you agree with this scenario.

    There should be 3 tropes of firewall rules:-

    1/. Proxy

    2/. DPI

    3/. no inspection  as per v17.5 firewall rules.

     

    Even though everyone talks about using the DPI default do not decrypt and maximum compatibility seems not to understand that SNORT still needs to intercept and read the connections to decide which SSL/TLS rule to use and therein lies the issue.

    Ian

  • In reply to rfcat_vk:

    rfcat_vk
    Even though everyone talks about using the DPI default do not decrypt and maximum compatibility seems not to understand that SNORT still needs to intercept and read the connections to decide which SSL/TLS rule to use and therein lies the issue.

    Well...

    1. Group A shall not be inspected. (IPS / Web filter / App Control / etc.)
    2. v18 has a problem with the DPI engine, so it is often impossible to communicate properly.
    3. Use Web Proxy to avoid problems with the DPI engine.
    4. Using Web Proxy violates policy of the Group A
    5. Evict the XG Firewall v18 appliance.

    Is that correct? Is there any other way?

  • In reply to FoW:

    Hi FOW,

    that is for external access, internal access you cannot use the proxy (LAN to LAN), but otherwise a nice summary.

    ian

  • In reply to rfcat_vk:

    ATP is a global on/off. When turned on it applies to all traffic on all ports.
    Bandwidth shaping and rate limiting are applied to all traffic.
    IPS is applied to various traffic.
    All of these things use snort to various extents. With the introduction of DPI they all use DPI to some extent.

    The reason that matching a firewall rule with no web policy and a TLS inspection rule with Do not Decrypt is not enough is because there are more parts to the system than that.

    So let says we have a problem where there is traffic that should go through the system and is not. That is a bug. There are two broad solutions to this: Fix the bug or exclude the traffic. As Rich has said, our approach is to fix the bug. Right now we are not looking at a way of excluding traffic completely from DPI. Right now we are focused on fixing bugs.

    That is partly why I have tried to separate the issue of "how do I bypass DPI" from "this device does not work" into separate threads, but I'm giving up now. You don't bypass DPI. Asking again how to bypass DPI is a dead end. Asking how you can help fix DPI so that the devices work is better. That is why EAP is so valuable to us. There are lots of really great and helpful people here who are sharing logs and tcpdumps and time and energy, and we really really thank you. And I know you are also sharing a feature request to have a way to bypass DPI completely but right now that feature is not going to happen. Rather than having a feature for a workaround we want to fix things so that you don't need to workaround.

    As for the counters proving that DPI is being used, you are semi-correct, and partly it gets down to terminology. Internally, all traffic gets looked at by snort. If the traffic looks like a TLS handshake then the TLS/SSL Inspection Rules are consulted and if a rule matches then the rule counter goes up. But even if no rules match and it falls to the bottom it is still looking at the traffic. You cannot say "don't look at the traffic if it matches these criteria, which you can only find out by looking at the traffic".  How can you apply a rule to Don't Decrypt traffic to microsoft.com until after you have looked in the SNI in the TLS Hello. So it is true if a TLS Inspection Rule counter goes up that proves that snort looked at the traffic and determined it was TLS. But even if none of the counters go up, snort is still looking at the traffic, so the proof is immaterial.  And if the traffic was not TLS, DPI is looking at the traffic and none of the TLS counters would change.

    Lets take a scenario where you have a Firewall Rule that says apply a web policy to block access to websites categorized as Phishing&Fraud. Lets say you have a TLS inspection rule that is Don't Decrypt, Maximum Compatibility. Lets say a client goes to HTTPS fakebank.com. DPI will see the TLS Hello and read the SNI in it, then decide to block the request based on the web policy. No TLS inspection rules counters went up. Lets say a client goes to HTTPS MalwareControlServer.com. DPI will see SNI and decide to block the request based on ATP. Even if the firewall rule had no Web policy, ATP is applied.

    Making sure that you hit a TLS Inspection Rule of Do not Decrypt does not mean that it isn't doing DPI, that it is not doing Deep Packet Inspection. It just means it won't decrypt.

    prism

    What I've told you before, If you DON'T want traffic to be SCANNED (IPS/AV/APP-WEB Policies being applied) on XG, all you need to do is have a Rule with no security profile on it (IPS/AV/App Classification.) and simply don't create a SSL/TLS Inspection rule to decrypt.


    Incorrect. If you don't want traffic to be scanned for web category, there is nothing you can do - if it is HTTP/HTTPS and there is a URL, we always look at the category of the URL.  We may not apply rules based on web category, we may not log it, but we will categorize every URL.

    If you don't want traffic to be scanned for viruses, turn off malware detection in the firewall rule.
    If you don't want traffic to have a web policy applied, use policy None or Allow All.
    If you don't want traffic to be scanned for ATP, turn off ATP globally.
    If you don't want traffic to be scanned for IPS, turn off IPS for it.
    If you don't want SSL/TLS to be decrypted, make sure it hits a Do Not Decrypt rule.
    If you don't want traffic shaping or bandwidth limited, turn off those rules.
    And probably more.
    There is no global way of saying don't scan for anything.  That is how it has been since XG began.  DPI means we are looking deeper into packets then before, but snort has always looked at packets.  Fact is, even when using the full blown web proxy, snort is still looking at the traffic.

     

    So, given that you cannot turn off DPI, the question is how can we make DPI work better.  We need helpful customers who are willing to tell us where it is broken, to share detailed information and logging, to test out solutions.  We have done our best to fix as many issues as we can for GA, we are going to do our best to fix as many as we can for MR1.  If you have an issue and you need it to work, then continue to use the web proxy.

  • In reply to Michael Dunn:

    Hi Michael,

    thank you for taking the time with the detailed explanation.

    I have tried to assist with this thread because I have an application that does not work LAN to LAN and I have not been able to find anything in the logs that I know how to access that would add further value to the thread. So, I ask how to create a similar situation as in V17 which you tell me I can't, so I am stuck with a application that doesn't work the way the application support people tell me it should. So to get around this issue I have permanently put one of my devices on my IoT network to provide local access. Shortly I will dig out an old android device and use that.

    Ian

  • In reply to rfcat_vk:

    rfcat_vk

    I have tried to assist with this thread because I have an application that does not work LAN to LAN and I have not been able to find anything in the logs that I know how to access that would add further value to the thread. So, I ask how to create a similar situation as in V17 which you tell me I can't, so I am stuck with a application that doesn't work the way the application support people tell me it should. So to get around this issue I have permanently put one of my devices on my IoT network to provide local access. Shortly I will dig out an old android device and use that.

    You can.  Just check the box "Use proxy instead of DPI mode".  Or maybe I am not understanding your problem.

    If you have a 17.5 box that is working and you upgrade that box to 18.0 and make no changes everything should continue to work.

    There are a few cases where this is not working - specifically when ports other than 80/443 are involved such as airprint.

    If that is not true - please let me know.  Everything that is being reported to us is about DPI mode.  If you are using proxy mode everything should just work.

    We have had almost no bug reports from anyone about proxy mode in v18.0.  If people upgrade, make no config changes, and stuff breaks that is a high priority.

  • In reply to Michael Dunn:

    Hi Michael,

    I don't have an issue with the proxy mode, but was advised in an earlier post that you cannot use proxy on internal LAN to LAN rules.

    Ian

  • In reply to rfcat_vk:

    rfcat_vk

    I don't have an issue with the proxy mode, but was advised in an earlier post that you cannot use proxy on internal LAN to LAN rules.

    What do you mean you cannot use the transparent mode proxy in LAN to LAN?

    AFAIK anything you can do in 17.5 you can do in 18.0.

  • In reply to Michael Dunn:

    But I didn't use the transparent proxy in v17 on LAN to LAN traffic. So what you are implying is that the proxy will work LAN to LAN, I will try again.

    Ian

     

    Extra, the application does not use http proxy ports.