Failed to upgrade from v19.5 to v20 (Home license)

Hi cm00001,

Thank you for reaching out to Sophos Community.

Kindly try updating via SFLoader, please refer to the following KB.

• Load firmware using SFLoader

Hello Erick,  thanks for the reply. Today I tried something different. I noticed the v19.5 MR4 update was posted. I decided to install the update, and it got updated without issues.

Later today, I tried to upload the v20.0 GA update on top, and got an interesting message and screen.

Incompatible firmware

  The firmware you're trying to install is incompatible with your current firmware version. When you restart the firewall with this firmware, the firewall will start with the factory configuration.

Make sure you take a backup of your current configuration if you want to continue. For compatible versions, see the release notes.

    I understand that when I restart the firewall with this firmware, I'll lose my current configuration.  

The upload failed.

From this point:

• Do I downgrade to 19.5 MR3 and perform the v20 CLI install?
• Or try the v20 CLI install with 19.5 MR4 in place?

Thanks!

Hi cm00001,

I've just tested upgrading my V19.5 MR3 to V20, and it went well without any issues.

Also, V20  isn’t supported from V19.5 MR4.

doc.sophos.com/.../index.html

Kindly try to do V19.5 MR3 and please check the following workaround, which might assist.

 (Missing Forum Thread) 

Thanks Erick.

I didn't realize MR4 wasn't in the list of supported upgrades to v20.  I'll try again from MR3 and report back. 

Update..  Following this guide (Load firmware using SFLoader - Sophos Firewall) didn't work.  (I went back to 19.5 MR3).

Any other place where I can check for logs to see where is the failure?

Sophos Firewall: Automatic firmware rollback in case of configuration migration failure

Hi cm00001,

Have you checked migration.log and migrationhash.log?

Hello Erick, here are the logs I found (in bold what I believe are errors, something about a WAF Security Profile). Let me know if you see something that might cause the firmware update to fail. 

Thanks! 

Migration.Log

2024-01-27 21:26:48.071 GMT starting old version corporate db
Starting conf database
374 2024-01-27 21:26:48.589 GMTLOG: could not connect socket for statistics collector: Network is unreachable
374 2024-01-27 21:26:48.589 GMTLOG: disabling statistics collector for lack of working socket
376 2024-01-27 21:26:48.590 GMTLOG: database system was shut down at 2024-01-27 21:26:32 GMT
374 2024-01-27 21:26:48.604 GMTLOG: database system is ready to accept connections
2024-01-27 21:26:50.402 GMT
2024-01-27 21:26:50.424 GMT
2024-01-27 21:26:50.426 GMT : Database started after 0 seconds
DROP SCHEMA
UPDATE 3
Stopping database
374 2024-01-27 21:26:51.906 GMTLOG: received fast shutdown request
374 2024-01-27 21:26:51.906 GMTLOG: aborting any active transactions
377 2024-01-27 21:26:51.906 GMTLOG: shutting down
377 2024-01-27 21:26:52.068 GMTLOG: database system is shut down
2024-01-27 21:26:52.927 GMT : Database stopped after 1 seconds
/sdisk/oldpgconfdump.sql is created
Starting conf database
426 2024-01-27 21:26:53.907 GMTLOG: could not connect socket for statistics collector: Network is unreachable
426 2024-01-27 21:26:53.907 GMTLOG: disabling statistics collector for lack of working socket
428 2024-01-27 21:26:53.907 GMTLOG: database system was shut down at 2023-11-01 07:11:05 GMT
426 2024-01-27 21:26:53.921 GMTLOG: database system is ready to accept connections
2024-01-27 21:26:55.873 GMT
2024-01-27 21:26:55.875 GMT
2024-01-27 21:26:55.877 GMT : Database started after 0 seconds
DROP SCHEMA config CASCADE
DROP SCHEMA
DROP SCHEMA public CASCADE
DROP SCHEMA
DROP PROCEDURAL LANGUAGE plpgsql
442 2024-01-27 21:26:56.609 GMTERROR: cannot drop language plpgsql because extension plpgsql requires it
442 2024-01-27 21:26:56.609 GMTHINT: You can drop extension plpgsql instead.
442 2024-01-27 21:26:56.609 GMTSTATEMENT: DROP PROCEDURAL LANGUAGE plpgsql
ERROR: cannot drop language plpgsql because extension plpgsql requires it
HINT: You can drop extension plpgsql instead.
CREATE SCHEMA public
CREATE SCHEMA
psql:/sdisk/oldpgconfdump.sql:19504: WARNING: column "senderemail" has type "unknown"
DETAIL: Proceeding with relation creation anyway.
psql:/sdisk/oldpgconfdump.sql:19504: WARNING: column "receipientemail" has type "unknown"DETAIL: Proceeding with relation creation anyway.
setval
--------
5404
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
1
(1 row)

setval
--------
5
(1 row)

setval
--------
68060
(1 row)

setval
--------
1
(1 row)

429 2024-01-27 21:27:00.414 GMTLOG: checkpoints are occurring too frequently (7 seconds apart)
429 2024-01-27 21:27:00.414 GMTHINT: Consider increasing the configuration parameter "checkpoint_segments".
Stopping database
426 2024-01-27 21:27:08.984 GMTLOG: received fast shutdown request
426 2024-01-27 21:27:08.984 GMTLOG: aborting any active transactions
429 2024-01-27 21:27:10.489 GMTLOG: shutting down
429 2024-01-27 21:27:10.864 GMTLOG: database system is shut down
2024-01-27 21:27:11.026 GMT : Database stopped after 2 seconds
old conf to new conf migrated with return value :: 0
nvram_del failed with -32
nvram_del failed with -32
nvram_del failed with -32
nvram_del failed with -32
2024-01-27 21:27:11.240 GMT starting migration log
Starting conf database
512 2024-01-27 21:27:11.368 GMTLOG: could not connect socket for statistics collector: Network is unreachable
512 2024-01-27 21:27:11.368 GMTLOG: disabling statistics collector for lack of working socket
514 2024-01-27 21:27:11.371 GMTLOG: database system was shut down at 2024-01-27 21:27:10 GMT
512 2024-01-27 21:27:11.382 GMTLOG: database system is ready to accept connections
2024-01-27 21:27:13.326 GMT
2024-01-27 21:27:13.329 GMT
2024-01-27 21:27:13.330 GMT : Database started after 0 seconds
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
INSERT 0 0
UPDATE 1
INSERT 0 0
UPDATE 4
INSERT 0 0
UPDATE 1
nvram_get failed with -16
Old version is 19.506 and currentversion is 20.002
Database is upgrading to dbv20.000
Check migration for version dbv20.000
Applying migration for version dbv20.000
add_column
------------

(1 row)

add_column
------------

(1 row)

add_column
------------

(1 row)

add_new_waf_columns
---------------------

(1 row)

waf_cipher_config_changes
---------------------------

(1 row)

1542 2024-01-27 21:27:15.615 GMTERROR: duplicate key value violates unique constraint "tblwafsecurityprofile_name_key"
1542 2024-01-27 21:27:15.615 GMTDETAIL: Key (name)=(Microsoft RD Web) already exists.
1542 2024-01-27 21:27:15.615 GMTSTATEMENT: UPDATE tblwafsecurityprofile SET name = 'Microsoft RD Web', comment = 'Microsoft RD Web' WHERE name = 'Microsoft RD Web 2008';
psql:/_conf/DB/dbv20.000/corporate.sql:451: ERROR: duplicate key value violates unique constraint "tblwafsecurityprofile_name_key"
DETAIL: Key (name)=(Microsoft RD Web) already exists.
/bin/psql -1 -p 5432 -U pgroot -q -d corporate -f /_conf//DB/dbv20.000/corporate.sql Failed
/bin/sh /_conf//DB/dbv20.000/migration.sh Failed

UPDATE 1
Stopping database
512 2024-01-27 21:27:16.692 GMTLOG: received fast shutdown request
512 2024-01-27 21:27:16.692 GMTLOG: aborting any active transactions
515 2024-01-27 21:27:16.693 GMTLOG: shutting down
515 2024-01-27 21:27:17.254 GMTLOG: database system is shut down
2024-01-27 21:27:17.711 GMT : Database stopped after 1 seconds
applymigration.sh exited with 1

MigrationHash.Log

Thu Jan 25 22:20:21 2024Z Migration/Legacy-restore process is called

Thu Jan 25 22:20:21 2024Z Retrieving diff for the version :/_conf//DB/dbv19.506/

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :adsserver

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :advancedsmtpsetting

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :backuprestore

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :dkimsigning

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :dynamicdns

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :edirserver

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :guestusers

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :haconfiguration

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :hbcloudregistration

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :hotspotuserportal

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :hotspotvoucheruserportal

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :idpssoserver

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :interface

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :ipsecconnection

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :ldapserver

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :notification

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :otptokens

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :parentproxyv4

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :parentproxyv6

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :radiusserver

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :reddevice

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :routingauth

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :smarthostsetting

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :snmpcommunity

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :snmpv3users

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :ssoradiusaccount

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :tacacsserver

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :user

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :waf_advanced_config

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :wirelessaccesspoints

Thu Jan 25 22:20:21 2024Z Diff processing started for entity :wirelessnetworks

Thu Jan 25 22:20:21 2024Z Secure data migration done successfully

Hi cm00001,

can you further share/DM the csc debug logs and syslog if possible?

There is a default out-of-the-box WAF security policy called "Microsoft RD Web 2008".  The upgrade is trying to rename it to "Microsoft RD Web" but is failing because you already have a policy with that name.

Can you delete or rename your existing "Microsoft RD Web"

That did it. Thanks Michael! I should have tried that out before submitting the logs. 

Thanks Erick, for pointing me to pull those 2 logs!