This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Dropped Connections during Pattern Updates

Since installing multiple XG Firewalls in a multi-site environment, we have been plagued with "random" outages that last between 30-90 seconds.

I have finally correlated this with Pattern updates for either ATP, AV or IPS.  During the time of the definition updates all connectivity to the XG firewall is lost.  This actually brings down our Wide Area network and causes VoIP phones to restart looking for the phone server.

I have an open support ticket with Sophos but I'm awaiting their response.

I have changed the updates to happen less frequently (Daily), however when there are updates it still brings down the connection (albeit less often now).

Is there a way to still have automatic updates turned on but do them on a time schedule?  I find it utterly ridiculous that the system cannot do pattern updates without bringing down the entire network.

If this is "expected" behavior what have others done as workarounds?  I cannot have 30-90 seconds of downtime every other day for pattern updates. 



This thread was automatically locked due to age.
Parents
  • Thanks Bill.  I agree and have seen this article as well.

    But there is currently no fix and no workaround other than to turn off automatic pattern updates?  How can we have a firewall device that drops all connections during pattern updates?  How can I recommend to enterprise?  How do I get more visibility to this?  I've also seen the Sophos Idea to give more control over scheduling these updates which I have upvoted, but frankly, I don't want to lose connection, EVER.

    I'm awaiting Sophos support to get back to me on my questions above as well, but I just can't fathom how this is acceptable on any level.

    I feel like now I am forced to choose between consistent connectivity by turning off automatic pattern updates and security.

  • Yes, I am seeing this behaviour on firewalls that have fastpath disabled (which seems to be the default for XG's that are in a cluster).

    It would make more sense if this was the other way around, surely? if fastpath routes 'trusted' traffic directly without IPS checking it, it shouldn't be affected by the IPS service restarting? Where as if fastpath is disabled, and traffic cannot be checked as IPS was restarting, then the traffic would be dropped?

  • Virtual Fastpath is a component, which uses Snort as well. Therefore if Snort uses a update, it could drop the session as well, but certainly not in each and every case. 

    VFP is per default enabled on all appliances (And HA). But was disabled pre V18.0 MR4. It will not get enabled after an upgrade, instead you can change your config and enable it. 

    __________________________________________________________________________________________________________________

  • some interesting facts are coming up here. Any reason for the default disabled VFP setting in MR4? Is this only for fresh installations on MR4? What is this with migrations over MR4 to MR5. We went from 17.5 MR12 over 18 MR1,->4,->5 where we re-imaged our appliances when going to MR4, then imported the config.

    VFP was enabled when checking it recently but has now been disabled because asked by support for some kind of issue without fxing the issue by the disabled setting.

     can you provide some steps how you measured the time of connection loss?

    I'd like to review this with our XG430s HA.

    I know we lost traffic for some seconds when disabling VFP.

  • Sophos is not enabling most settings after a firmware upgrade to avoid issues within the network after a firmware update. V18.0 MR4 enabled VFP option on HAs. Customers coming from a older version, had this disabled and can enable it, if they want. This option will be likely be enabled with a future release. 

    A new installation without backup/restore will have VFP enabled per default. 

    __________________________________________________________________________________________________________________

  • Is there a command to show if it is enabled (rather than enable/disable it)?

  • console> system firewall-acceleration show
    Firewall Acceleration is Enabled in Configuration.

    __________________________________________________________________________________________________________________

  • Just from seeing the issue a few times, we'd typically notice that a MS Teams call would stop responding, then i'd try a web browser and see that it was a generic 'page cannot be displayed'. Give it around 20 seconds and then it works again as expected. But normally the delay is long enough that you'll get dropped from your Teams call and need to dial back in again. Super frustrating.

    I may try the workaround by and reboot our firewall in the early hours, hoping that the pattern updates will take place 24 hours again after that (e.g. out of hours).

  • We use a program called PingPlotter. We run pings to several external addresses (to avoid false positives) and the XG IP as well. It maintains logs of all the connections and we can check those to see at the time of an update, the ping to the XG is fine but all the other pings are blocked.

    You can use PingPlotter free but if you want to run it as a service, you can either run a 14 day trial or buy it.

  • PS: Keep in mind, Ping is not a TCP/UDP connection. There is another Bug ID related to Pings in virtual fastpath, as ICMP seems to behavior differently. As there are no real indication of session in ICMP, it cannot remain the session. Therefore if the ping packet is lost, its lost. TCP/UDP can work with retransmission and there pickup the same session. So a Ping lost does not have to result into a lost session within the network. 

    __________________________________________________________________________________________________________________

  • thanks for your replies. seems hard enough to even create a valid test scenario...

Reply Children
No Data