Thread Info
  • State Not Answered
  • Locked Locked
  • Replies 2 replies
  • Subscribers 16 subscribers
  • Views 2960 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Updates of Sophos Network Threat Protection Module causing outages

LHerzog

Hi,

everytime Intercept X Agents managed by Central update this component:

Sophos Network Threat Protection

it causes a network interruption of seconds to minutes of this device, client or server (Windows 10 x64, 1809 and 2004, Server 2012 R2 and 2016 x64)

These updates happen throughout the day at randome times. Even if we have selected a time schedule for this on the client computers group.

We've noticed at least one Server 2012 R2 machine that hung completely after such an update and could only be put back in production by a hard-reset.

Every machine loses Heartbeat to the XG firewalls during that updates once or more often causing firewall blocks until the heartbeat is re-established.

We've noticed, this component is updated quite often and so also causes some trouble often.

Also the update creates an error log in event viewer:

Fehler   04.03.2021 09:00:03       Application Error             1000       (100)
Name der fehlerhaften Anwendung: MsiExec.exe, Version: 5.0.17763.404, Zeitstempel: 0x07240266
Name des fehlerhaften Moduls: MSIDFC7.tmp, Version: 1.11.194.0, Zeitstempel: 0x5fbbebe2
Ausnahmecode: 0xc0000005
Fehleroffset: 0x000000000000ea8d
ID des fehlerhaften Prozesses: 0x3b14
Startzeit der fehlerhaften Anwendung: 0x01d710cc3b5ea29f
Pfad der fehlerhaften Anwendung: C:\Windows\System32\MsiExec.exe
Pfad des fehlerhaften Moduls: C:\Windows\Installer\MSIDFC7.tmp
Berichtskennung: fe55c4ce-702b-4b39-8d39-fa66d6417c2c
Vollständiger Name des fehlerhaften Pakets: 
Anwendungs-ID, die relativ zum fehlerhaften Paket ist:

Is it the same as on XG - updating IPS patterns causing network interruptions each time?

msiexec log file extract

...
...
MSI (s) (C0:84) [08:59:00:795]: Executing op: CustomActionSchedule(Action=UnregisterHbtManagementAdapter,ActionType=1089,Source=BinaryData,Target=UnregisterHbtManagementAdapter,)
MSI (s) (C0:84) [08:59:00:799]: Creating MSIHANDLE (93) of type 790536 for thread 10628
MSI (s) (C0:9C) [08:59:00:800]: Invoking remote custom action. DLL: C:\Windows\Installer\MSIF0C1.tmp, Entrypoint: UnregisterHbtManagementAdapter
MSI (s) (C0!40) [08:59:00:813]: Creating MSIHANDLE (94) of type 790531 for thread 14656
MSI (s) (C0!40) [08:59:00:814]: Closing MSIHANDLE (94) of type 790531 for thread 14656
MSI (s) (C0!40) [08:59:00:814]: Creating MSIHANDLE (95) of type 790531 for thread 14656
UnregisterManagementAdapter:  UnregisterManagementAdapter called
MSI (s) (C0!40) [08:59:00:814]: Closing MSIHANDLE (95) of type 790531 for thread 14656
MSI (s) (C0!40) [08:59:00:815]: Creating MSIHANDLE (96) of type 790531 for thread 14656
UnregisterManagementAdapter:  Adapter = HBT
MSI (s) (C0!40) [08:59:00:815]: Closing MSIHANDLE (96) of type 790531 for thread 14656
MSI (s) (C0!40) [08:59:00:815]: Creating MSIHANDLE (97) of type 790531 for thread 14656
UnregisterManagementAdapter:  Management adapter successfully unregistered
MSI (s) (C0!40) [08:59:00:816]: Closing MSIHANDLE (97) of type 790531 for thread 14656
MSI (s) (C0!40) [08:59:00:816]: Creating MSIHANDLE (98) of type 790531 for thread 14656
UnregisterManagementAdapter:  Waiting for adapter to be unloaded
MSI (s) (C0!40) [08:59:00:816]: Closing MSIHANDLE (98) of type 790531 for thread 14656
MSI (s) (C0!40) [09:00:00:893]: Creating MSIHANDLE (99) of type 790531 for thread 14656
UnregisterManagementAdapter:  Adapter path=C:\Program Files\Sophos\Sophos Network Threat Protection\HbtAdapter.dll
MSI (s) (C0!40) [09:00:00:894]: Closing MSIHANDLE (99) of type 790531 for thread 14656
MSI (s) (C0:9C) [09:00:00:895]: Closing MSIHANDLE (93) of type 790536 for thread 10628
MSI (s) (C0:84) [09:00:00:896]: Executing op: ActionStart(Name=UnregisterManagementAdapterRollback,,)
UnregisterManagementAdapter:  Failed to unload the adapter - continuing.
MSI (s) (C0:84) [09:00:00:897]: Executing op: CustomActionSchedule(Action=UnregisterManagementAdapterRollback,ActionType=1345,Source=BinaryData,Target=RegisterManagementAdapter,CustomActionData=C:\Program Files\Sophos\Sophos Network Threat Protection\NTPAdapter.dll)
MSI (s) (C0:84) [09:00:00:916]: Executing op: ActionStart(Name=UnregisterManagementAdapter,,)
MSI (s) (C0:84) [09:00:00:917]: Executing op: CustomActionSchedule(Action=UnregisterManagementAdapter,ActionType=1089,Source=BinaryData,Target=UnregisterManagementAdapter,)
MSI (s) (C0:84) [09:00:00:921]: Creating MSIHANDLE (100) of type 790536 for thread 10628
MSI (s) (C0:3C) [09:00:00:923]: Invoking remote custom action. DLL: C:\Windows\Installer\MSIDB9F.tmp, Entrypoint: UnregisterManagementAdapter
MSI (s) (C0!08) [09:00:00:943]: Creating MSIHANDLE (101) of type 790531 for thread 14344
MSI (s) (C0!08) [09:00:00:944]: Closing MSIHANDLE (101) of type 790531 for thread 14344
MSI (s) (C0!08) [09:00:00:944]: Creating MSIHANDLE (102) of type 790531 for thread 14344
UnregisterManagementAdapter:  UnregisterManagementAdapter called
MSI (s) (C0!08) [09:00:00:945]: Closing MSIHANDLE (102) of type 790531 for thread 14344
MSI (s) (C0!08) [09:00:00:945]: Creating MSIHANDLE (103) of type 790531 for thread 14344
UnregisterManagementAdapter:  Adapter = NTP
MSI (s) (C0!08) [09:00:00:946]: Closing MSIHANDLE (103) of type 790531 for thread 14344
MSI (s) (C0!08) [09:00:00:946]: Creating MSIHANDLE (104) of type 790531 for thread 14344
UnregisterManagementAdapter:  Management adapter successfully unregistered
MSI (s) (C0!08) [09:00:00:947]: Closing MSIHANDLE (104) of type 790531 for thread 14344
MSI (s) (C0!08) [09:00:00:947]: Creating MSIHANDLE (105) of type 790531 for thread 14344
UnregisterManagementAdapter:  Waiting for adapter to be unloaded
MSI (s) (C0!08) [09:00:00:948]: Closing MSIHANDLE (105) of type 790531 for thread 14344
MSI (s) (C0!08) [09:00:00:974]: Creating MSIHANDLE (106) of type 790531 for thread 14344
UnregisterManagementAdapter:  Adapter path=C:\Program Files\Sophos\Sophos Network Threat Protection\NTPAdapter.dll
MSI (s) (C0!08) [09:00:00:975]: Closing MSIHANDLE (106) of type 790531 for thread 14344
MSI (s) (C0:3C) [09:00:00:976]: Closing MSIHANDLE (100) of type 790536 for thread 10628
MSI (s) (C0:84) [09:00:00:978]: Executing op: ActionStart(Name=StopServices,Description=Dienste werden beendet.,Template=Dienst: [1])
UnregisterManagementAdapter:  Management adapter has been unloaded.
MSI (s) (C0:84) [09:00:00:978]: Executing op: ProgressTotal(Total=1,Type=1,ByteEquivalent=1300000)
MSI (s) (C0:84) [09:00:00:978]: Executing op: ServiceControl(,Name=SntpService,Action=2,Wait=1,)
MSI (s) (C0:84) [09:00:01:982]: Executing op: ActionStart(Name=UninstallSophosNTPLWF,,)
MSI (s) (C0:84) [09:00:01:983]: Executing op: CustomActionSchedule(Action=UninstallSophosNTPLWF,ActionType=1089,Source=BinaryData,Target=UninstallNetCfg,)
MSI (s) (C0:84) [09:00:01:986]: Creating MSIHANDLE (107) of type 790536 for thread 10628
MSI (s) (C0:F0) [09:00:01:988]: Invoking remote custom action. DLL: C:\Windows\Installer\MSIDFC7.tmp, Entrypoint: UninstallNetCfg
MSI (s) (C0:F0) [09:00:05:386]: Closing MSIHANDLE (107) of type 790536 for thread 10628
MSI (s) (C0:84) [09:00:05:387]: Executing op: ActionStart(Name=UnregisterSntpEventManifestRollback,,)
CustomAction UninstallSophosNTPLWF returned actual error code 1603 but will be translated to success due to continue marking
...
...

Property(S): INSTALLLEVEL = 1
Property(S): SOURCEDIR = C:\ProgramData\Sophos\AutoUpdate\Cache\decoded\ntp64\
Property(S): SourcedirProduct = {2D2A1891-4657-4E6F-9373-BFCE4C9AC5BA}
MSI (s) (C0:84) [09:00:09:400]: Note: 1: 1728 
MSI (s) (C0:84) [09:00:09:400]: Transforming table Error.

MSI (s) (C0:84) [09:00:09:400]: Transforming table Error.

MSI (s) (C0:84) [09:00:09:400]: Produkt: Sophos Network Threat Protection -- Die Konfiguration wurde abgeschlossen.

MSI (s) (C0:84) [09:00:09:401]: Das Produkt wurde durch Windows Installer neu konfiguriert. Produktname: Sophos Network Threat Protection. Produktversion: 1.11.194.0. Produktsprache: 1031. Hersteller: Sophos Limited. Erfolg- bzw. Fehlerstatus der neuen Konfiguration: 0.

MSI (s) (C0:84) [09:00:09:401]: Closing MSIHANDLE (1) of type 790542 for thread 10628
MSI (s) (C0:84) [09:00:09:414]: Deferring clean up of packages/files, if any exist
MSI (s) (C0:84) [09:00:09:415]: MainEngineThread is returning 0
MSI (s) (C0:10) [09:00:09:415]: No System Restore sequence number for this installation.
=== Protokollierung beendet: 04.03.2021  09:00:09 ===
MSI (s) (C0:10) [09:00:09:431]: User policy value 'DisableRollback' is 0
MSI (s) (C0:10) [09:00:09:431]: Machine policy value 'DisableRollback' is 0
MSI (s) (C0:10) [09:00:09:431]: Incrementing counter to disable shutdown. Counter after increment: 0
MSI (s) (C0:10) [09:00:09:431]: Note: 1: 1402 2: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Installer\Rollback\Scripts 3: 2 
MSI (s) (C0:10) [09:00:09:435]: Note: 1: 2265 2:  3: -2147287035 
MSI (s) (C0:10) [09:00:09:441]: Note: 1: 1402 2: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Installer\Rollback\Scripts 3: 2 
MSI (s) (C0:10) [09:00:09:442]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied.  Counter after decrement: -1
MSI (s) (C0:10) [09:00:09:443]: Destroying RemoteAPI object.
MSI (s) (C0:54) [09:00:09:443]: Custom Action Manager thread ending.
MSI (c) (E4:F0) [09:00:09:445]: Decrementing counter to disable shutdown. If counter >= 0, shutdown will be denied.  Counter after decrement: -1
MSI (c) (E4:F0) [09:00:09:445]: MainEngineThread is returning 0
=== Verbose logging stopped: 04.03.2021  09:00:09 ===

setup::MsiInstaller::installOrUpgrade: Install/upgrade returned 0
`anonymous-namespace'::setResult: installation successful
ProductSetup::InstUninstEntry: Using IPS rules: C:\ProgramData\Sophos\AutoUpdate\Cache\decoded\ntp64\2021030301.ips
ProductSetup::InstUninstEntry: Copy IPS rules file: C:\ProgramData\Sophos\AutoUpdate\Cache\decoded\ntp64\2021030301.ips to: "C:\\ProgramData\\Sophos\\Sophos Network Threat Protection\\IPS\\2021030301.ips"
ProductSetup::InstUninstEntry: Update IpsCurrentRules to: "C:\\ProgramData\\Sophos\\Sophos Network Threat Protection\\IPS\\2021030301.ips"
setup::TamperProtectionControl::enable: Registered tamper protection integrity.dat for NTP
setup::TamperProtectionControl::enable: Enabled tamper protection for NTP
ProductSetup::~ProductSetup: End product setup

This update-process does'nt look like it's really ready for enterprise use.



This thread was automatically locked due to age.
  • 0

    Hi

    When you say that "quite often", what is the time duration you are seeing this issue? Could you also share Heartbeat logs(C:\ProgramData\Sophos\Heartbeat\Logs) to check if the connection is established successfully. 

    Shweta

    Community Support Engineer | Sophos Technical Support
    Are you a Sophos Partner? | Product Documentation | @SophosSupport | Sign up for SMS Alerts
    If a post solves your question use the 'Verify Answer' link.
    The New Home of Sophos Support Videos! - Visit Sophos Techvids
  • 0

    That Application Error doesn't cause a problem and I've been told it is fixed in the installer for the next release.  Essentially some code is running that doesn't need to run on that platform.

    From my computer:

    NTP install log:

    MSI (s) (04:A8) [08:25:55:159]: Executing op: ActionStart(Name=UninstallSophosNTPLWF,,)
    MSI (s) (04:A8) [08:25:55:160]: Executing op: CustomActionSchedule(Action=UninstallSophosNTPLWF,ActionType=1089,Source=BinaryData,Target=UninstallNetCfg,)
    MSI (s) (04:A8) [08:25:55:167]: Creating MSIHANDLE (107) of type 790536 for thread 16040
    MSI (s) (04:7C) [08:25:55:170]: Invoking remote custom action. DLL: C:\WINDOWS\Installer\MSIC81E.tmp, Entrypoint: UninstallNetCfg
    MSI (s) (04:7C) [08:25:57:070]: Closing MSIHANDLE (107) of type 790536 for thread 16040
    MSI (s) (04:A8) [08:25:57:071]: Executing op: ActionStart(Name=UnregisterSntpEventManifestRollback,,)
    CustomAction UninstallSophosNTPLWF returned actual error code 1603 but will be translated to success due to continue marking

    App Event Log entry:

    Faulting application name: MsiExec.exe, version: 5.0.21322.1000, time stamp: 0xfa1b1bbd
    Faulting module name: MSIC81E.tmp, version: 1.11.194.0, time stamp: 0x5fbbebe2
    Exception code: 0xc0000005
    Fault offset: 0x000000000000ea8d
    Faulting process ID: 0xd8
    Faulting application start time: 0x01d710cfd7a6ede5
    Faulting application path: C:\Windows\System32\MsiExec.exe
    Faulting module path: C:\WINDOWS\Installer\MSIC81E.tmp
    Report ID: f8b43006-ee72-48d9-af53-388c560ff963
    Faulting package full name:
    Faulting package-relative application ID:

    Essentially the custom action UninstallSophosNTPLWF doesn't need to be called on platforms without this driver as it's only used on older platforms.

    I'm not sure if it would cause an issue.  As a test, you could force it to run to observe it.  To do so:

    1. Disable Tamper if enabled.

    2. Launch Notepad as admin and open SophosUpdateStatus.xml from C:\ProgramData\Sophos\AutoUpdate\data\status\

    3. Turn on Wordwrap

    4. Find the NTP section by searching for ntp64, e.g.:

    <ComponentState lineId="8087796B-2289-4897-98A5-58FF23DAAFD0" installedThumbprint="c28a3ee4c48ad91ff33deca678c7f1192238614a045eb654bbb09ada0d34c924" downloadedThumbprint="c28a3ee4c48ad91ff33deca678c7f1192238614a045eb654bbb09ada0d34c924" installedVersion="1.11.194.0" downloadedVersion="1.11.194.0" defaultHomeFolder="ntp64" name="ntp64" installable="1"/>

    4. Clear the installedThumbprint value so it reads
    <ComponentState lineId="8087796B-2289-4897-98A5-58FF23DAAFD0" installedThumbprint="" downloadedThumbprint="c28a3ee4c48ad91ff33deca678c7f1192238614a045eb654bbb09ada0d34c924" installedVersion="1.11.194.0" downloadedVersion="1.11.194.0" defaultHomeFolder="ntp64" name="ntp64" installable="1"/>

    5. Open the UI of the Sophos Agent - About - Update now.

    6. Monitor \windows\temp\ for the NTP install log/Event log entry.

    I guess you could perform some sort of ongoing network test during this time?

    Download a large file.  Ping a host, etc..  For me I don't see an issue.

    Hope it helps.

Unfiltered HTML