SSL/TLS inspection | bridge mode | multiple local subnets | SSL connections time out

To all of you following this topic:

New additional workaround:

set ips ac_atp exception fwrules <add at most eight rule IDs, comma separated>

See: Sophos Firewall: Bypass a specific firewall rule for application classification and ATP:

https://support.sophos.com/support/s/article/KB-000038900?language=en_US

Does anyone exactly know what this exception does? And why it could help here?

Regards, Nicolai

The reason this workaround (disabling ATP) is working, is because this feature was the only remaining feature that was also interested in the initial part of TLS connections (you have everything else disabled on rule 12 already, like IPS, Web, AV, etc.). Even if not decrypting, there is data in the handshake that is used for threat protection. With ATP (and the rest of the features) disabled, packets on the TLS connection do not have to be sent into the DPI and TLS processing logic anymore. This is effectively the same as completely disabling SSL/TLS, that you already found to be a workaround too.

In this setup, with the external gateway directing traffic back through the XG/S, the TLS connection is effectively "picked up" twice. From client to router, and then again (the same data) from the router to the server. Resulting in the issue that you are experiencing.

If the XG/S were to make the routing decision, or the client and server were not routed through the same XG/S, this issue would of course not be seen (the connection would not traverse the TLS inspection logic twice).

We will have some more discussion on this internally.

Thanks for the detailed info that you have provided. Based on it, we were able to determine the cause of what you are experiencing.

Best,
Elardus

Hi Elardus,

thanks for these technical informations.

I had already guessed that the fact, that the TLS connection is effectively "picked up" twice, could be the problem.

For the moment we can live with the 'set ips ac_atp exception fwrules'-solution.

Does this setting survive firmware updates?

How do you suggest to carry on? Can we hope for an adjustment in one oft the next firmware versions?

Regards, Nicolai

Yes, that setting is saved and retained across upgrades.

I understand your interest in this going forward. We are still to discuss and finalize the course of action here. I will update you once there is progress.

Best,
Elardus

Yes, that setting is saved and retained across upgrades.

I understand your interest in this going forward. We are still to discuss and finalize the course of action here. I will update you once there is progress.

Best,
Elardus

Hi Elardus,

Thanks a lot!!

I look forward to hear again from you!

Best, Nicolai

Hi Nicolai,

So, it turns out this hairpin type of deployment could have ramifications to multiple features. E.g. the proxy would also fail, normal conntrack could become “unstable” in the presence of retransmits and speed differences between the various links involved, double accounting of traffic, and so on.

In this case, the recommended solution is to configure masquerading on the router. In that way, the connection between the client and router, will be distinct from the one between the router and server. And that should get everything working as expected.

We plan to create a KB article for this. Hopefully helping others that run into the same thing in the future.

Thanks for your time on this.
Elardus

Hello Elardus,
please excuse my late reaction. I'm not entirely sure I understood that correctly. Is your recommendation not to use the 'set ips ac_atp exception fwrules' solution? Can you explain why in more detail? Masquerading on the router simply has the disadvantage that we can no longer differentiate between source IPs and source ports.

Regards, Nicolai

Hi Nicolai,

The ATP exception is a workaround to get past the immediate issue that you experienced. The issue was in the TLS inspection path, due to the hairpin deployment. The ATP exception, in addition to disabling all other features on the firewall rule, means that these connections will not be sent for TLS decryption and inspection anymore. If you are happy with this, then you can leave it as is.

The problem with hairpin type deployments, however, is larger than this alone, and you may experience other problems. As mentioned in the past, it can cause all sorts of trouble. Functionally and functions/features that rely on traffic accounting per user - the XG/S effectively sees the traffic with the same src/dst details, ingressing (and egressing) the firewall twice.

That is why masquerading on the router is recommended in these types of scenarios. The firewall will essentially see two different flows, and then you can turn TLS inspection back on if you want, by removing the ATP exception and enabling more features on the firewall rule if desirable. Even though it will be seen as two flows, only one of them will be inspected - so you will not use double the resources to inspect the traffic.

We are in the process of writing a Knowledge Base Article on this, which may contain a few more details. I will send it along once it is done.

Hope it helps.

Regards, Elardus