Sophos Firewall: v19.0 GA: Feedback and experiences

We are facing an issue due a rollout of an XGS4500 v19 GA Cluster these days. We set al lot of static routes (around 40) through the GUI. After goLive we figured out that around 30% of the routes in the GUI are not avaliable on the backend. These routes are shown in gui, but never in the cli "ip route". It picks up randomly routes, so it have nothing to do with the order when you put them in.

We have tested to boot the primary and then the aux device, without any change in the situation.

After we figured out the packets are leaving on the WAN interface instead of the ifc where they should go, we found a workaround. (we followed relayed dhcp traffic)

-> Just reopen the non deployed routes and click simply save again -> the will appear in the routing table a moment later.

Due this is reall weird, we opened a case 05295087 and will see what is the reason.

This might be helpful for all of you, to doublecheck the entered & the real applied routes on the CLI to avoid our problem ;)

Network level RDP connections over SSL VPN Client no longer works after 19 update.  Roll back to 18.5 mr3 works fine.

So far everything is working well except for the depreciation of the old SSL VPN client along with the lack of update for the Sophos Connect client as users are having issues with characters in their passwords. Seems like an oversight to have not released Sophos Connect 2.2 prior to the update. 

https://support.sophos.com/support/s/article/KB-000044122?language=en_US

Dead firewalls everywhere after scheduling FW updates after the first 20 appliances upgraded without issue. Why can't the upgrade process detect that the certificate is not valid and advise instead of proceeding to kill the unit completely?

The problem was discovered recently and only affects Cyberoam Appliances or old installations coming from Cyberoam.

Sophos always recommend to read the Release notes carefully: https://docs.sophos.com/releasenotes/index.html?productGroupID=nsg&productID=xg&versionID=19.0

The Upgrade and known issue section is explaining the issue. 

This will be addressed in the next version.

One option is to put it in the release notes and rely of the user reading them before the upgrade. Fair for most people on this forum, we're a bit geeky, not sure about the standard business user who just wants to upgrade their firmware, and receives no prompt on the GUI interface to read the release notes first. Also bear in mind Sophos often push out emails saying "Make sure you're running the latest firmware or you could be at risk / unsupported"

Another option may be a re-release of the existing v19 with a built in check - (if coming from Cyberoam, do not apply, and prompt with message) or equivalent.

I guess what I'm implying is, lets not try to move too much of the emphasis to the end user when trying to offer a user friendly product.

It is not that easy to re release a product / firmware from such perspective. There are many implication on doing this. Not only technical, legal etc. 

ahhh I see, okay cool

Ryan summaries my concerns better than I have.

For us, we manage over 100 Sophos units. We upgraded 1 without issue, waited a few weeks, then did another 5 and again no issues, so push out the update to the rest of the units centrally. Then all of a sudden we had 12 clients dead in the water with no internet access. And because upgrades to critical infrastructure has to happen out of hours, the failures occurred at night and we have to wait until the morning when suddenly 12 different units are down and staff can't work. It's fine for an MSP to perform the steps to rectify, but you can't get a client to log in to advanced shell and run those commands.

I get the issues with reissuing firmware to resolve a bug, but what I don't understand is that if the Sophos knows it's doing a firmware update and then it ends up in Failsafe mode, it clearly knows the process has failed. Why can't it mark that FW image as faulty so that next the time appliance boots it goes back to the known good FW image? That way we could just tell the client to power cycle the unit and they'd be back online and log in remotely to fix. This was one of my biggest issues with Cyberoam previously as well. If a FW fails to boot, next time boot to the other one. That way the client is not dead in the water.

First of all, a SIG/ISO build is a finish state. You are using a build XYZ version. In this version, the issue was not discovered. So to "flag this as a failed image" the code is not there to flag in, as the issue was not known to the firewall at this point.

Sophos could revoke the entire build. This means pull back build XYZ for all customers. Re release a new version (build ABC with the fix). This takes a long time to go through all cycles (QA checks, legal, release infrastructure etc.). 

Failsafe is some issue, which is actually quite rare in the latest releases. I guess the last time i saw failsafe was 2 years ago. So this is not a common issue. If the migration does not work, it can pull back to the old version. But fail safe is some sort of kernel panic mode. So the entire firewall stopped at this point. 

There is a lot of background work currently to get a state of this cannot happen again. 

And again, we do not talk about "every customer is affected". It seems like only a customer base with a old configuration coming from cyberoam.