Weird UTM freezes randomly approximately once a day ...

Hello there!

Good day and thanks for reaching out to Sophos Community and hope you are well. 

Wanted to check if this freeze happens often at night? or just a single occurence? 

If ths happens often I may recommend you to open a support ticket to have this further checked, if just a single occurence we may want to observe and see if this would persist in the future. 

Have a nice day ahead and thank you for choosing Sophos

Cheers,

Unfortunately the reload using asg-9.714-4.1.iso instead of ssi-9.714-4.1.iso did not solve the freezing, I used the 64 bit kernel.

I have performed a disk check and memory check and it passes without any error.
When the unit is frozen, you can't ping the LAN interfaces, and USB keyboard doesn't respond in order to login.
The freezing seems to happen randomly, it looks like it happens every day ...

I am including all fail, error, invalid messages from boot log, I do not know if these are useful at all:

2023:03:19-14:09:12 escape75 kernel: [    0.000000] ACPI BIOS Warning (bug): FADT (revision 6) is longer than ACPI 5.0 version, truncating length 276 to 268 (20130725/tbfadt-311)
2023:03:19-14:09:12 escape75 kernel: [    0.000000] ACPI Error: Gpe0Block - 32-bit FADT register is too long (32 bytes, 256 bits) to convert to GAS struct - 255 bits max, truncating (20130725/tbfadt-202)

2023:03:19-14:09:12 escape75 kernel: [    1.785222] TCP: TFO aes cipher alloc error: -2

2023:03:19-14:09:12 escape75 kernel: [    5.318546] i801_smbus 0000:00:1f.1: can't derive routing for PCI INT A
2023:03:19-14:09:12 escape75 kernel: [    5.318554] i801_smbus 0000:00:1f.1: PCI INT A: no GSI
2023:03:19-14:09:12 escape75 kernel: [    5.318591] i801_smbus 0000:00:1f.1: Failed to allocate irq 255: -22
2023:03:19-14:09:12 escape75 kernel: [    5.318598] i801_smbus: probe of 0000:00:1f.1 failed with error -22

2023:03:19-14:09:12 escape75 kernel: [    0.004000] tsc: Fast TSC calibration failed
2023:03:19-14:09:12 escape75 kernel: [    0.012000] tsc: PIT calibration matches HPET. 1 loops
2023:03:19-14:09:12 escape75 kernel: [    0.012000] tsc: Detected 1592.856 MHz processor

2023:03:19-14:09:12 escape75 kernel: [    0.199653] acpi PNP0A08:00: ACPI _OSC support notification failed, disabling PCIe ASPM
2023:03:19-14:09:12 escape75 kernel: [    0.199661] acpi PNP0A08:00: Unable to request _OSC control (_OSC support mask: 0x08)

2023:03:19-20:09:37 escape75 [daemon:notice] rrdcached[3786]:  queue_thread_main: rrd_update_r (/var/log/reporting/rrd/ips.rrd) failed with status -1. (/var/log/reporting/rrd/ips.rrd: illegal attempt to update using time 1679255404 when last update time is 1679256005 (minimum one second step))

2023:03:19-14:09:12 escape75 kernel: [    0.199505] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
2023:03:19-14:09:12 escape75 kernel: [    0.199579] \_SB_.PCI0:_OSC invalid UUID
2023:03:19-14:09:12 escape75 kernel: [    0.199582] _OSC request data:1 8 0 
2023:03:19-14:09:12 escape75 kernel: [    0.199643] \_SB_.PCI0:_OSC invalid UUID

Are you able to install some kind of windows or linux OS to the device?  Maybe run something like prime95 or equiv to load it up.  Could be cooling issues (or lack there of) causes instability.

I don't know, it's a regular XG115 R3 from Sophos ...

I would expect that there would be some kind of troubleshooting path, as this appears caused by my config possibly ...

I wonder if I could load SFOS HW Firmware and get a trial license and would be able to tell within a few days if it freezes ...

Why would you need a trial license?  Just install it and let it run without importing your config.  I would start from scratch and see how that works, if it's still freezing.  If it's not after a few days, it might be something in the config.  If it freezes, it's probably hardware related. 

I didn't know I could run it without a license ...

Well UTM you can just use your existing license file or it comes with a 30-day trial, and XG assigns you one when you download it, and you can see it under Administration > Licensing. 

Perfect, that's what I'm doing right now, UTM hardware ISO using the 30 day trial, and a basic setup, this will test my XG115 R3.

I have also re-loaded my XG115 R2 just to see if the other box exhibits the same issues, this should give me some more clues.

I tried XG115 R2 re-install and use the same config file I've had issues with on my R3, and it's been fine for over 24 hours.

Then I tried a basic setup with R3, eth1 - WAN, and eth0, eth2, eth3 - LAN (br0) and it went down in about 15 minutes!

Temperatures seem ok, and memory tested fine, and storage is also OK ... I'm at a loss!

loginuser@escape75:/home/login > sensors
acpitz-virtual-0
Adapter: Virtual device
temp1: +36.0&***;C (crit = +125.0&***;C)

coretemp-isa-0000
Adapter: ISA adapter
Physical id 0: +36.0&***;C (high = +110.0&***;C, crit = +110.0&***;C)
Core 0: +35.0&***;C (high = +110.0&***;C, crit = +110.0&***;C)
Core 1: +35.0&***;C (high = +110.0&***;C, crit = +110.0&***;C)
Core 2: +35.0&***;C (high = +110.0&***;C, crit = +110.0&***;C)
Core 3: +35.0&***;C (high = +110.0&***;C, crit = +110.0&***;C)

I don't believe this is a configuration issue, and it could be either a software and/or hardware issue.

Did you by chance run a smart test on the disk via SSH?

smartctl -a /dev/sda

Yes, smart test both short and long passes, as well as memory test, as well as temperatures seem ok.
I've been monitoring temperatures as the box has been up for over an hour, so I'm just checking what I can.

Yes, smart test both short and long passes, as well as memory test, as well as temperatures seem ok.
I've been monitoring temperatures as the box has been up for over an hour, so I'm just checking what I can.

Can you paste the SMART test information here? 

Of course, here it is:

escape75:/home/login # smartctl -d ata --all /dev/sda
smartctl 6.3 2015-02-08 r4039 [x86_64-linux-3.12.74-0.424574463.ge309b77.rb7-smp64] (SUSE RPM)
Copyright (C) 2002-14, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     ADATA_IM2S3134N-064GM
Serial Number:    2I3920032044
LU WWN Device Id: 5 707c18 1006e2fd0
Firmware Version: 6.8E
User Capacity:    64,023,257,088 bytes [64.0 GB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-2 (minor revision not indicated)
SATA Version is:  SATA 3.1, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Tue Mar 21 06:31:32 2023 CET
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x00) Offline data collection activity
                                        was never started.
                                        Auto Offline Data Collection: Disabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                (   32) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        (   1) minutes.
SCT capabilities:              (0x0039) SCT Status supported.
                                        SCT Error Recovery Control supported.
                                        SCT Feature Control supported.
                                        SCT Data Table supported.

SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000a   100   100   000    Old_age   Always       -       0
  2 Throughput_Performance  0x0005   100   100   050    Pre-fail  Offline      -       0
  3 Spin_Up_Time            0x0007   100   100   050    Pre-fail  Always       -       0
  5 Reallocated_Sector_Ct   0x0013   100   100   050    Pre-fail  Always       -       0
  7 Unknown_SSD_Attribute   0x000b   100   100   050    Pre-fail  Always       -       0
  8 Unknown_SSD_Attribute   0x0005   100   100   050    Pre-fail  Offline      -       0
  9 Power_On_Hours          0x0012   100   100   000    Old_age   Always       -       20706
 10 Unknown_SSD_Attribute   0x0013   100   100   050    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0012   100   100   000    Old_age   Always       -       147
167 Unknown_Attribute       0x0022   100   100   000    Old_age   Always       -       0
168 Unknown_Attribute       0x0012   100   100   000    Old_age   Always       -       0
169 Unknown_Attribute       0x0013   100   100   010    Pre-fail  Always       -       196611
170 Unknown_Attribute       0x0013   100   100   010    Pre-fail  Always       -       0
173 Unknown_Attribute       0x0012   128   128   000    Old_age   Always       -       4365747135
175 Program_Fail_Count_Chip 0x0013   100   100   010    Pre-fail  Always       -       0
180 Unused_Rsvd_Blk_Cnt_Tot 0x0033   100   100   020    Pre-fail  Always       -       553
192 Power-Off_Retract_Count 0x0012   100   100   000    Old_age   Always       -       116
194 Temperature_Celsius     0x0022   052   052   030    Old_age   Always       -       48 (Min/Max 44/49)
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
231 Temperature_Celsius     0x0033   069   069   005    Pre-fail  Always       -       31
233 Media_Wearout_Indicator 0x0032   100   100   000    Old_age   Always       -       127659751936
234 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       192211142656
240 Unknown_SSD_Attribute   0x0013   100   100   050    Pre-fail  Always       -       0
241 Total_LBAs_Written      0x0032   100   100   000    Old_age   Always       -       23934369819
242 Total_LBAs_Read         0x0032   100   100   000    Old_age   Always       -       5005309732

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed without error       00%     20667         -
# 2  Short offline       Completed without error       00%     20633         -

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

Oh God, thank you for formatting that, lol. 

Yeah that's a very healthy disk.  I was wanting the values specifically, as most people just read the pass/fail line and don't worry about anything else.  All the values look really good.