Thread Info
  • State Verified Answer
  • +5 person also asked this people also asked this
  • Replies 19 replies
  • Answers 1 answer
  • Subscribers 34 subscribers
  • Views 58960 views
  • Users 0 members are here
Options
Suggested

Sophos Firewall v18: XStream - the new DPI Engine for web proxy explained

Michael Dunn

In our list of new features:
www.sophos.com/.../sophos-xg-firewall-key-new-features.pdf

Xstream Architecture
Sophos is pleased to introduce the new Xstream Architecture for XG Firewall, a new streaming packet processing
architecture that provides extreme levels of protection and performance. The new architecture includes:

1) Xstream SSL Inspection: Organizations can enable SSL inspection on their networks without
compromising network performance or user experience. It delivers high-performance, high connection
capacity support for TLS 1.3 and all modern cipher suites providing extreme SSL inspection performance
across all ports, protocols, and applications. It also comes equipped with enterprise-grade controls to optimize
security, privacy, and performance.

2) Xstream DPI Engine: Enables comprehensive threat protection in a single high-performance streaming
DPI engine with proxyless scanning of all traffic for AV, IPS, and web threats as well as providing Application
Control and SSL Inspection. Pattern matching on decrypted traffic makes patterns more effective and provides
increased protection from hash/pattern changing applications such as Psiphon proxy.

3) Xstream Network Flow FastPath: Provides the ultimate in performance by intelligently offloading traffic
processing to transfer trusted traffic at wire speeds. FastPath offloading can be controlled through policy to
accelerate important cloud application traffic, or intelligently by the DPI engine based on traffic characteristics.

 

So what does this mean?

One of the new features that is v18.0 is a new high performance way of handling web traffic, along with new high performance way of doing SSL/TLS decryption, and a lot of new options around enforcement of TLS/SSL rules. The web proxy from 17.5 is still present, and administrators have a choice which mode they want to use.

The following is an attempt to summarize the differences between the "proxy mode" and the new "DPI mode" (Deep Packet Inspection). Basically to explain 2) and the relevant parts of 1). But the overall feature is more than what I am covering.

It focuses on differences in web for the things you could do in 17.5, and do differently in 18.0.

 

  Mode Using the "web proxy" Using the "DPI Engine"
Summary Underlying process name awarrenhttp snort
  How does it work Terminates the connect from the client browser.
Creates a new connection to the web server.
Copies data from one connection to the other.
Delays traffic while it processes access control.
Delays traffic if using Batch AV scanning mode (default).
Can modify headers, changes destination IP, etc.
If blocking needed, displays block page.		 Allows connection straight from client browser to the web server.
Inspects the traffic from client to server.
Does little to modify, delay, or interrupt traffic.
Performs AV scanning in-line (like Real Time AV scanning mode).
If blocking needed and response is already started to sent it kills the connection.
If blocking needed and response is not sent it redirects browser to :8090 to display block page.
  Benefits More functionality.
Better block pages.		 Better Performance.
More port and TLS inspection options.
Configuration Creating a firewall rule for web traffic Firewall Rule:
Set the services to HTTP and HTTPS (port 80 and 443).
Select a Web Policy
Select Scan HTTP and decrypted HTTPS		 No difference.
  How to choose the web traffic proxy mode Firewall Rule:
Check "Use web proxy instead of DPI engine"		 Firewall Rule:
Uncheck "Use web proxy instead of DPI engine"
  How to configure HTTPS decryption Firewall Rule:
Check "Decrypt HTTPS during web proxy filtering"		 Go to Rules and policies >  SSL/TLS inspection rules and create an inspection rule
Set the services to HTTP and HTTPS (port 80 and 443).
Set the Action to Decrypt.
Select a Decryption Profile.
  How to exclude a site from HTTPS decryption Go to Web > Exceptions and create and exception.
Set URL pattern match to be website FQDN.  This is RegEx, see existing entries for proper syntax.
Select skip HTTPS decryption.		 Create a web exception (as in web proxy mode)
-OR-
Go to URL Groups and edit the Local TLS exclusion list.
Add the domain name.  This is plaintext not RegEx.
Make sure the default SSL/TLS inspection rule "Exclusions by website" is enabled.
-OR-
Create an SSL/TLS inspection rule that is set to action "Do not decrypt".  Create your own URL group or custom category and use it in the rule.
  How to exclude a source from HTTPS decryption Go to Web > Exceptions and create and exception.
Set Source IP or IP range.
Select skip HTTPS decryption.		 Create a web exception (as in web proxy mode)
-OR-
Create an SSL/TLS inspection rule that is set to action "Do not decrypt".  Base it on source IP or IP range or on user.
Port and TLS support Transparent Mode Ports Port 80/443 Any port
  Standard/Explicit/Direct Mode Supported (Default port 3128) Not supported.  Will direct to proxy.
  Enforce TLS and certificate restrictions
for port 80/443 traffic		 Basic on/off flags that covers common settings:
Web > General Settings > Block unrecognized SSL protocols
Web > General Settings > Block invalid certificates		 Detailed control:
Profiles > Decryption Profiles
Rules and policies >  SSL/TLS inspection rules > SSL/TLS inspection settings
  Enforce TLS and certificate restrictions for other ports Not supported Supported
Feature Support HTTPS signing Certificate Authority You can select a single CA that will be used for all traffic. You can select different CAs for signing traffic where the web server is using an RSA or EC certificate.
You can select different CAs based on source or destination.
  AV scanning mode Batch (holds entire file while it scans, displays block page if there is a virus)
Real Time (delivers file as it arrives, holds last bit of file while it scans, kills connection if there is a virus)		 Real Time (delivers file as it arrives, holds last bit of file while it scans, kills connection if there is a virus).  If downloaded a second time soon after, redirect to a block page.
  Pharming Protection Supported Not supported
  Caching Supported Not supported
  Enforce SafeSearch Supported Not supported
  Enforce YouTube restrictions Supported Not supported
  Restrict login domains for Google apps Supported Not supported


Edited Title
[edited by: emmosophos at 10:16 PM (GMT -8) on 24 Nov 2022]
  • 0 in reply to Michael Dunn

    Hello Michael,

    I apologize, but I feel more and more that you are building the Doge's Palace in Venice ( https://en.wikipedia.org/wiki/Doge%27s_Palace ). Beautiful gothic palace, which unfortunately stands on very shaky foundations on the water. By this I mean the simultaneous implementation of NAT rules and their link to firewall rules.


    I would be very much in favor of reinforcing the foundations, that is, a completely redesigned implementation of NAT rules and links to firewall rules.

    I think you wouldn't want your palace to fall on your head.

    Regards

    alda

  • 0 in reply to alda

    Hi Alda,

    I am on the web development team for web proxy, DPI, and sandstorm.  I have no involvement or knowledge in the NAT rules.  I would like to maintain this thread only for discussion of the DPI engine.  Do you mind starting a new thread for discussion of NAT rules and deleting the above, so as to keep this thread focused?

  • 0

    Could someone post a tutorial on how to step by step implement a DPI traffic scan? Maybe I'm doing something wrong, because after enabling "decrypt" in TLS / SSL rules some used applications stop working. They work after restarting the device, but if one of the household members leaves the house and comes back - applications cannot connect (for ex. Facebook, Facebook Messenger, Gadu Gadu Messenger, Youtube and Youtube Go App).

    Thank you

    darnoK

  • 0 in reply to alda

    Alda,

    I understand and share your frustration. We are trying to get our voice listened but please keep posting on proper thread. In a big sw development project, teams are split into UNITS. For the moment devs are not taking part on community. Michael is an exception. I already shared with Sophos that devs must be on the community as communication must be direct between end-user and devs. More people are in the middle and more information is lost. Something is already changing on v18. You can see how many devs are there but they are still not interacting as Michael does.

    Regards

  • 0 in reply to lferrara

    Hello Luk,

    of course you're right. I just let my frustration out of the quality of developing NAT rules.
    I'm sorry, it won't happen again.

    Regards

    alda

  • +2 in reply to darnoK

    A brief step by step guide:

    I would start by doing things incrementally, to build out knowledge and experience. By far the biggest and most common usage will be in HTTP/HTTPS so start there. Most customers will already be using the web proxy, this assumes that you have some knowledge.

    Next is to determine if you are going to decrypt traffic or not. The cost and the reasons are the same as they are with the web proxy. Decrypting traffic means that you will need to deploy the appliance CA to all your devices. Regardless of whether you choose to do so in the long term, incrementally I would start by not decrypting. Make sure everything is stable without decryption before enabling.

    Apply the workaround listed about so that TLS 1.3 traffic is downgraded to 1.2.

    So create a a firewall rule with appropriate source and destination, service is HTTP and HTTPS. Choose a web policy, scan for malware, don't "use the web proxy".
    On the SSL/TLS inspection rules, just have the top two rules active. Don't create (or enable) others.

    Run this for a few hours. You should see logs in the web filter log. You can test your policy works by going to HTTP and HTTPS sites that should be blocked due to category. If you do not have the CA installed, when you go to an HTTPS site that is blocked, you will first get a browser warning message. You should see in the firewall rules your new rule bandwidth goes up.

    If everything is working, you can turn on HTTPS decryption. First deploy the CA to your devices.

    Now you can create a SSL/TLS Inspection Rule. Use the same source and destination as your HTTPS firewall rule, HTTPS service only. Set action to decrypt. Set the profile to maximum compatibility.

    Run like that for a while. See what works and what does not. If there is one particular device that is not working, check the CA on it. If there is one particular application that is not working, check the logs and see if there is anything interesting. Note what domains it is using. Some applications use their own certificate stores.

    The 'just get it working' approach to HTTPS sites that don't work is to add them to the TLS exclusion list. Go to Web \ URL Groups and edit "Local TLS exclusion list" to add the domains. Then test again.
    Now when we (developers of this feature) find stuff that does not work, rather than adding to the exclusion list, we investigate to see why it is not working and if there is anything that the product could do
    differently.

  • 0 in reply to Michael Dunn

    OK, I already know. Not everything can be decrypted even using a certificate. Gradually, I will update individual rules as instructed, step by step.

    Michael and Stuart, thank you so much for your help.

Unfiltered HTML