Sophos Firewall: v18.5 MR4: Feedback and experiences

Since I can't wait for Sophos to release v19 MR1 and v18.5 MR4 won't fix the SASI hassle that's going on, I decided to downgrade to 18.5 MR2. However, this seems to be a problem sometimes.

I have a XGS126 that was upgraded from 18.5 MR2 to 19 GA. I could downgrade it back to 18.5 MR2 without any problem.

After that I tried to downgrade a XG210 that was upgraded from 18.5 MR3 to 19 GA. When uploading 18.5 MR2 or MR4 to the firewall it said "The firmware will boot the device with factory default configuration. Are you sure you want to continue?". Since I can't stand all the spam, I did it anyway and booted the uploaded 18.5 MR2. It indeed came up with factory defaults loaded. I tried to restore the latest config backup from 19 GA, but it just said that the firmware is not matching, so I restored an old backup from a time the firewall had installed 18.5 MR2, which worked. After that I just booted up the still present 19 GA firmware and the firewall came up with the restored MR2 config. I restored the latest 19 GA backup and booted the 18.5 MR2 firmware. I expected the firewall to come up with factory defaults again, but it still has the latest config running.

So thats the processes that work:
18.5 MR2 > 19 GA > 18.5 MR2/MR4 without factory reset
18.5 MR3 > 19 GA > 18.5 MR2/MR4 with factory reset  > 19 GA > restore latest config from 19 GA > 18.5 MR2/MR4 without factory reset

Why do I need a factory reset if 18.5 MR3 was previously installed, but not if I skipped it? In both scenarios I go 19 GA > 18.5 MR2 in the end.

Since I can't wait for Sophos to release v19 MR1 and v18.5 MR4 won't fix the SASI hassle that's going on, I decided to downgrade to 18.5 MR2. However, this seems to be a problem sometimes.

I have a XGS126 that was upgraded from 18.5 MR2 to 19 GA. I could downgrade it back to 18.5 MR2 without any problem.

After that I tried to downgrade a XG210 that was upgraded from 18.5 MR3 to 19 GA. When uploading 18.5 MR2 or MR4 to the firewall it said "The firmware will boot the device with factory default configuration. Are you sure you want to continue?". Since I can't stand all the spam, I did it anyway and booted the uploaded 18.5 MR2. It indeed came up with factory defaults loaded. I tried to restore the latest config backup from 19 GA, but it just said that the firmware is not matching, so I restored an old backup from a time the firewall had installed 18.5 MR2, which worked. After that I just booted up the still present 19 GA firmware and the firewall came up with the restored MR2 config. I restored the latest 19 GA backup and booted the 18.5 MR2 firmware. I expected the firewall to come up with factory defaults again, but it still has the latest config running.

So thats the processes that work:
18.5 MR2 > 19 GA > 18.5 MR2/MR4 without factory reset
18.5 MR3 > 19 GA > 18.5 MR2/MR4 with factory reset  > 19 GA > restore latest config from 19 GA > 18.5 MR2/MR4 without factory reset

Why do I need a factory reset if 18.5 MR3 was previously installed, but not if I skipped it? In both scenarios I go 19 GA > 18.5 MR2 in the end.

I do not have the answer to this point but MR2 will go End of Life soon. Per definition of the lifecycle policy: https://support.sophos.com/support/s/article/KB-000035279?language=en_US#xgfirewallsoftware

https://community.sophos.com/sophos-xg-firewall/b/blog/posts/sophos-lifecycle-policy-changes-end-of-life-announcement-for-older-sfos-versions

So i assume, it is not smart to go back in time to MR2. 

At least Anti-Spam is working with MR2. I can confirm that with 18.5 MR4 Anti-Spam is still not working! After over 3 months, the problem is still not fixed, how do you explain that to your customers?

Do you have a support case for this not working? What do you mean by not working? Is the engine not loaded or what does not work in your picture? Because i actually do not receive any complain by my customers about this in the last month. It was fixed by a hotfix and the engine should work for now. 

Like this user is reporting: 

The only thing I see is that a lot of very obvious spam mails are still getting through, which is not the case with MR2, so I hope that this is not as good as it gets

Check the Logs for those Emails and see, what the engine reports. 

I see mails in GUI Mail log that are not present in the smtpd_main.log, is that by design?

The mail log can eventually rotate. See: https://docs.sophos.com/nsg/sophos-firewall/18.0/Help/en-us/webhelp/onlinehelp/AdministratorHelp/Logs/LogFileDetails/index.html

In Webadmin the logs are a database, which can eventually store much more meta data compared to the plain text data. 

I found the last two obvious spam mails that got through:

MSG   Jun 21 03:58:24Z [1o3V1z-0005pO-SF]: spam scanning result: 'NonSpam'
MSG   Jun 21 03:58:45Z [1o3V2K-0005qt-IP]: spam scanning result: 'NonSpam'

So I guess the SASI engine is just bad then.

SASI is being used in UTM, Central Email, Email Appliance, PureMessage etc. 

It is now in the retrospective hard to compare those, as you do not have the comparison for this exact email. 

But: Do you use IP Reputation? 

That's not correct, I indeed can compare both engines for the very same email. I just send one of the two mails I posted above to an exchange server behind a XGS I downgraded to MR2, guess what the result was?

MSG   Jun 21 09:15:06Z [1o3ZyT-0006wW-Mn]: spam scanning result: 'Confirmed spam'