Firewall FQDN fail to match if domain match web exception
Create FQDN Firewall rule. Surf to FQDN and watch traffic hit rule.
Add a web exception with Matching URL regex and skip all.
Surface to FQDN again and watch it the firewall rule, confirm with firewall logs.
Actually, if i can follow your request, this is not a Bug at all.
Firewall Rules have nothing to do with a Web Exception.
Thank you for contacting the Sophos Community!
Just to be clear on your duplication steps:
1.- Create a Firewall rule with destination set as a FQDN, for example abc.com
2.- Create a web exception ^([A-Za-z0-9.-]*\.)?abc\.com\.?/ skip all the checks
3.- Try accessing the FQDN, and the Firewall rule with the FQDN will not be matched?
Yes, I am finding that there are a few sites that do not like the DPI engine no matter what settings I use. So I created a separate rule with destination set as FQDN locations to send those to proxy instead. If there are any matching rules in web exclusion, the traffic would take the old firewall rule path. If I disable the matching web exception with the toggle button, I can see the traffic switch to the new FQDN rule. I see this in the firewall log viewer by firewall rule ID.
Sites so far are smartthings/samsung, ring, and netflix.
Rule #51 with web exception enabled (DPI).
Rule #2 with web exception disabled. (can see it is heading to proxy at dst_trans_ip = 192.168.8.1)
Basically the Web Module exception will only be called, if the web proxy is enabled for this firewall rule. The firewall rule will hit, no matter what (first match).
So maybe i do not understand your issue, it would be better to explain it with more screenshots.
Firewall rules are just not getting hitting based on URL see (ring.com)
Firewall log shows Rule #51. Should hitting Rule #2 which comes first. I am finding lots of sites that just don't work with xStream, even when fast-pathed (all rules disabled). I may have to switch back to v17 for the second time now until v19 comes out.
An according to the documentation, web exceptions also apply to the DPI engine, not just the web proxy.
Do some more experiments, I found that even if I have no inspections selected in the firewall rule, Example Rule #2 below. Certain sites are still not able to connect (all streaming sites). I found that I have to add the URLs in the Local Exclusion group in SSL/TLS rules before the site would connect. This must be because of the first line below "SSL/TLS inspection rules are applied independently of firewall rules".
SSL/TLS inspection rules are applied independently of firewall rules. Inspection rules continue to enforce the specified exclusions even if you don't select a web policy in firewall rules.
You can use both web exceptions and SSL/TLS exclusion rules to stop connections from being decrypted. For details of how they differ in enforcing HTTPS decryption-related exceptions, see the table below:
SSL/TLS exclusion list
Processes you can exclude
HTTPS certificate and protocol enforcement
HTTPS certificate validation
Malware and content scanning
Web policy checks
Applies in this mode
Web Exclusions will be applied to DPI, if possible. Currently we think around 90% of the web traffic can be applied to the exclusion.
The reason is, any product can only decide, if it want to break the TLS within the first packet of TLS. Thats the TLS Syn in the handshake. See: https://tls.ulfheim.net/
The Client "can" put a SNI into the helo. So to speak, you can say, give me the connection of ring.com. But you do not have to do this, its optional.
So if the client is not doing this, we cannot apply the web exception. Because later in the stream, you cannot go and undo the decryption. Thats your only chance to apply decryption based on this first packet.
So lets paint the picture: Your Connection will come and hit, based on the first matching firewall rule. Seems like this can be #2, as it has some bytes on it. This firewall rule will forward this connection to DPI to decrypt or not. DPI may not be able to find a exclusion, as the SNI may not be there. So it will start to decrypt, which leads to a faulty connection.
Does this make sense?
Yes, Thanks for details explaination.