Thread Info
  • State Verified Answer
  • Locked Locked
  • Replies 6 replies
  • Subscribers 14 subscribers
  • Views 13358 views
  • Users 0 members are here
Options
Suggested
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Health service not running and clients not updating

Paul Hutchings

So at the moment it's simple, I have never used an A/V product that requires the amount of handholding as Sophos Central/Cloud.

The latest issue is that we have a batch of new installs and for quite a few I'm seeing the clients aren't updating, usually with the Sophos Health Service stopped and crashing/failing to start.

Updates don't happen and the client usually shows the status cannot be reported.

Yes they have internet access...

I see an email today mentioning this article which may address this but something seems badly badly wrong here.  community.sophos.com/.../127758

Typical log excerpt -

Seeing lots of failures to update along the lines of the below.

2017-12-15T11:24:58.639Z [ 5560] INFO WinMain =========================
2017-12-15T11:24:58.639Z [ 5560] INFO WinMain SophosUpdate is starting.
2017-12-15T11:24:58.640Z [ 5560] INFO WinMain AutoUpdate version : 5.8.407
2017-12-15T11:24:58.640Z [ 5560] INFO WinMain SophosUpdate version : 5.8.259
2017-12-15T11:24:58.640Z [ 5560] INFO WinMain Build : 206630
2017-12-15T11:24:58.640Z [ 5560] INFO WinMain =========================
2017-12-15T11:24:58.640Z [ 5560] INFO Environment::Print Platform ID: WIN_7_X64
2017-12-15T11:24:58.640Z [ 5560] INFO Environment::Print Platform upgraded: 0
2017-12-15T11:24:58.640Z [ 5560] INFO Environment::Print Subscription: WindowsCloudNextGen RECOMMENDED 11
2017-12-15T11:24:58.640Z [ 5560] INFO Environment::Print Subscription: WindowsCloudAV RECOMMENDED 11
2017-12-15T11:24:58.640Z [ 5560] INFO Environment::Print Subscription: WindowsCloudHitmanProAlert RECOMMENDED 1
2017-12-15T11:24:58.640Z [ 5560] INFO Environment::Print Subscriptions changed: 0
2017-12-15T11:24:58.640Z [ 5560] INFO Environment::Print Features: APPCNTRL AV CORE DLP DVCCNTRL EFW HBT NTP SAV SDU WEBCNTRL
2017-12-15T11:24:58.640Z [ 5560] INFO Environment::Print Features changed: 0
2017-12-15T11:24:58.640Z [ 5560] INFO WinMain Set process security
2017-12-15T11:24:58.640Z [ 5560] INFO WinMain Initialise COM.
2017-12-15T11:24:58.641Z [ 5560] INFO WinMain Load config.
2017-12-15T11:24:58.642Z [ 5560] INFO WinMain Create registry reporter.
2017-12-15T11:24:58.642Z [ 5560] INFO WinMain Create platform reporter.
2017-12-15T11:24:58.642Z [ 5560] INFO WinMain Create features reporter.
2017-12-15T11:24:58.642Z [ 5560] INFO WinMain Create subscription reporter.
2017-12-15T11:24:58.642Z [ 5560] INFO WinMain Create version persister.
2017-12-15T11:24:58.642Z [ 5560] INFO WinMain Load state.
2017-12-15T11:24:58.642Z [ 5560] INFO StatePersister::Load Loading state file C:\ProgramData\Sophos\AutoUpdate\data\status\SophosUpdateStatus.xml
2017-12-15T11:24:58.642Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for 1129226C-32AB-4B72-85E1-A9CC8DFBC859 to 36419441480e30e3dd94b61c98186c272aa6f1af4592b80617ee332fc8be619f.
2017-12-15T11:24:58.642Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for 1129226C-32AB-4B72-85E1-A9CC8DFBC859 to 36419441480e30e3dd94b61c98186c272aa6f1af4592b80617ee332fc8be619f.
2017-12-15T11:24:58.642Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for 1FE3E7DF-EFFA-408A-A1B0-89F15BA61F31 to f3312bd72b209c9978b5cd8aebd90526b0c1265cc5b21f98a7b62bc67a9e5797.
2017-12-15T11:24:58.642Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for 1FE3E7DF-EFFA-408A-A1B0-89F15BA61F31 to f3312bd72b209c9978b5cd8aebd90526b0c1265cc5b21f98a7b62bc67a9e5797.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for 243DECCD-8080-410D-A45F-77F2182715EE to ffe311c18b242937189804b642fea8a91a3882d61802acb9f95028ac8ad6d49e.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for 243DECCD-8080-410D-A45F-77F2182715EE to ffe311c18b242937189804b642fea8a91a3882d61802acb9f95028ac8ad6d49e.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for 3CE954A1-0F41-4D9B-B2F0-58AA75334DFD to 2703d0c5f37fa5560437bede240ba60033f7c02b17d8552b9100ffcb876ce6a5.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for 3CE954A1-0F41-4D9B-B2F0-58AA75334DFD to 2703d0c5f37fa5560437bede240ba60033f7c02b17d8552b9100ffcb876ce6a5.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for 5CD1A7B6-812E-47A1-A986-3A6D5D5C19F5 to 9b3a1ebd9f3cbca634a6ed39cef16c66c10138cb4df0f4d5ebccfb02e1c57f93.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for 5CD1A7B6-812E-47A1-A986-3A6D5D5C19F5 to 9b3a1ebd9f3cbca634a6ed39cef16c66c10138cb4df0f4d5ebccfb02e1c57f93.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for 6DA9E48A-5756-47E9-BAA3-F0286CF945BF to 43a5e2774510ce4150551360441e212bf20f80433d7b5491b57664ee76065146.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for 6DA9E48A-5756-47E9-BAA3-F0286CF945BF to 43a5e2774510ce4150551360441e212bf20f80433d7b5491b57664ee76065146.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for 7F682906-6E49-481B-89C5-2DCA36720F4F to 98773119601f54506c5f01b303b72e286103705279f934e5fef0ebc232ef3fdd.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for 7F682906-6E49-481B-89C5-2DCA36720F4F to 98773119601f54506c5f01b303b72e286103705279f934e5fef0ebc232ef3fdd.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for 8087796B-2289-4897-98A5-58FF23DAAFD0 to 398e6004f93af9e1d32ed97d35cb81bb14da520094a8a13051435eb450edf2d5.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for 8087796B-2289-4897-98A5-58FF23DAAFD0 to 398e6004f93af9e1d32ed97d35cb81bb14da520094a8a13051435eb450edf2d5.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for 8580B6E7-AD8F-4c42-8085-ABD5765C98C5 to 5e2243765ec656a7aa0b0cc7d7f0079c7c9581a0e49d986c7fdc649ea4484b3c.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for 8580B6E7-AD8F-4c42-8085-ABD5765C98C5 to 5e2243765ec656a7aa0b0cc7d7f0079c7c9581a0e49d986c7fdc649ea4484b3c.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for 8E360AC3-A63E-48C8-AAC0-B02090ED484C to 03b183a824be1519f4ef72ef474666014b036173aaf3109f2083b87b4920a272.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for 8E360AC3-A63E-48C8-AAC0-B02090ED484C to 03b183a824be1519f4ef72ef474666014b036173aaf3109f2083b87b4920a272.
2017-12-15T11:24:58.643Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for CD297D6B-58A5-474F-8A0D-0A15803B8B50 to 3015394914c4647456200a8d086d5890be3a20bbe17778628beac93f20fed774.
2017-12-15T11:24:58.644Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for CD297D6B-58A5-474F-8A0D-0A15803B8B50 to 3015394914c4647456200a8d086d5890be3a20bbe17778628beac93f20fed774.
2017-12-15T11:24:58.644Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for E17FE03B-0501-4aaa-BC69-0129D965F311 to 08ab0968fcac81294fa0831e30a9674c2486c671740e754650d10a406d93b59c.
2017-12-15T11:24:58.644Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for E17FE03B-0501-4aaa-BC69-0129D965F311 to 08ab0968fcac81294fa0831e30a9674c2486c671740e754650d10a406d93b59c.
2017-12-15T11:24:58.644Z [ 5560] INFO ComponentState::SetInstalledThumbprint Set installed thumbprint for SDU to 65bb116d82afc057d50b84d0f99085960d45ae1f94cee4ec8cd50b065e1119a6.
2017-12-15T11:24:58.644Z [ 5560] INFO ComponentState::SetDownloadedThumbprint Set downloaded thumbprint for SDU to 65bb116d82afc057d50b84d0f99085960d45ae1f94cee4ec8cd50b065e1119a6.
2017-12-15T11:24:58.644Z [ 5560] INFO WinMain Create progress reporter.
2017-12-15T11:24:58.651Z [ 5560] INFO WinMain Create language neutral logger.
2017-12-15T11:24:58.652Z [ 5560] INFO WinMain Create Update cache evaluator.
2017-12-15T11:24:58.653Z [ 5560] INFO WinMain Create downloader.
2017-12-15T11:24:58.657Z [ 5560] INFO WinMain Create Health event manager.
2017-12-15T11:24:58.657Z [ 5560] INFO WinMain Create Cloud reboot manager.
2017-12-15T11:24:58.657Z [ 5560] INFO CloudRebootManager::SignalRebootIfRequired No key found for RebootRequiredByCloudInstaller
2017-12-15T11:24:58.657Z [ 5560] INFO WinMain Create installer.
2017-12-15T11:24:58.657Z [ 5560] INFO WinMain Create adapter writer.
2017-12-15T11:24:58.657Z [ 5560] INFO IPCBase::IPCBase IPCBase::IPCBase: Connected to shared memory A32951C539924a12B3C8F2FDA5A268E4
2017-12-15T11:24:58.657Z [ 5560] INFO WinMain Create completion reporter.
2017-12-15T11:24:58.657Z [ 5560] INFO WinMain Create telemetry submitter.
2017-12-15T11:24:58.657Z [ 5560] INFO WinMain Create fallback updating logic.
2017-12-15T11:24:58.657Z [ 5560] INFO WinMain Create update logic.
2017-12-15T11:24:58.657Z [ 7972] INFO `anonymous-namespace'::SenderThreadFn::operator() Sender thread started.
2017-12-15T11:24:58.657Z [ 5560] INFO WinMain Performing update.
2017-12-15T11:24:58.657Z [ 7972] INFO IPCSender::ProcessSend IPCSender::ProcessSend started
2017-12-15T11:24:58.657Z [ 5560] INFO UpdateLogic::Update Reporting update start.
2017-12-15T11:24:58.657Z [ 7972] INFO IPCSender::ProcessSend IPCSender::ProcessSend: No messages in queue, starting to wait
2017-12-15T11:24:58.658Z [ 5560] INFO IPCSender::Write IPCSender::Write: Writing message: <?xml version="1.0" encoding="utf-8" ?><Config type="RMSStartUpdate" />
2017-12-15T11:24:58.658Z [ 7972] INFO IPCSender::ProcessSend IPCSender::ProcessSend: Send message: <?xml version="1.0" encoding="utf-8" ?><Config type="RMSStartUpdate" />
2017-12-15T11:24:58.658Z [ 7972] INFO IPCSender::ProcessSend IPCSender::ProcessSend: No messages in queue, starting to wait
2017-12-15T11:24:58.660Z [ 5560] INFO UpdateLogic::SyncAndInstall Performing standard update.
2017-12-15T11:24:58.660Z [ 5560] INFO UpdateLogic::SyncAndInstall Syncing products.
2017-12-15T11:24:58.660Z [ 5560] INFO SourceSelector::getCandidateProxies No manually configured proxy.
2017-12-15T11:24:58.660Z [ 5560] INFO WindowsProxyDiscoveryWrapper::GetDefaultProxyConfiguration WinHttp default proxy not set
2017-12-15T11:24:58.686Z [ 5556] WARN WindowsProxyDiscoveryWrapper::GetProxyForUrl Failed to get the automatic proxy configuration. The error code was 12180.
2017-12-15T11:24:58.686Z [ 5560] INFO SourceSelector::evaluate Trying update location: dci.sophosupd.com/update with proxy: <direct; no proxy>
2017-12-15T11:24:59.007Z [ 5560] INFO SourceSelector::analyze Selected update location: dci.sophosupd.com/update with proxy: <direct; no proxy>
2017-12-15T11:24:59.008Z [ 5560] INFO SDDSDownloader::SyncInternal Username: AJHDFDDOGF
2017-12-15T11:24:59.008Z [ 5560] INFO SDDSDownloader::SyncInternal Filename: 54cfa77f5dcb0d5beeea2bf6f8e94f42
2017-12-15T11:24:59.012Z [ 5560] INFO CacheEvaluator::Evaluate Analyzing whether to update from Sophos CDN or update cache
2017-12-15T11:24:59.013Z [ 5560] INFO CacheEvaluator::EvaluateCache Checking access to update cache: updatecache.domain:8191
2017-12-15T11:24:59.087Z [ 5560] INFO CacheEvaluator::EvaluateCache Successfully connected to cache: updatecache.domain:8191
2017-12-15T11:24:59.087Z [ 5560] INFO CacheEvaluator::Evaluate Analysis complete - Using update cache: updatecache.domain:8191
2017-12-15T11:24:59.087Z [ 5560] INFO SDDSDownloader::SyncInternal Updating from cache: updatecache.domain:8191
2017-12-15T11:24:59.089Z [ 5560] ERROR SDDSDownloader::ReportSyncFailure Failed to set data=2a5f463f-0a2c-4795-9bf0-7964d907ef96 for value=UpdateSource
2017-12-15T11:24:59.089Z [ 5560] INFO UpdateLogic::SyncAndInstall Saving state.
2017-12-15T11:24:59.090Z [ 5560] ERROR RegistryVersionPersister::Save Error writing version for line ID WindowsCloudNextGen, error: 5
2017-12-15T11:24:59.091Z [ 5560] ERROR RegistryVersionPersister::Save Error writing version for line ID WindowsCloudHitmanProAlert, error: 5
2017-12-15T11:24:59.091Z [ 5560] ERROR RegistryVersionPersister::Save Error writing version for line ID WindowsCloudAV, error: 5
2017-12-15T11:24:59.091Z [ 5560] ERROR RegistryVersionPersister::Save Error deleting line ID subkey WindowsCloudAV, error: 5
2017-12-15T11:24:59.092Z [ 5560] ERROR RegistryVersionPersister::Save Error deleting line ID subkey WindowsCloudHitmanProAlert, error: 5
2017-12-15T11:24:59.092Z [ 5560] ERROR RegistryVersionPersister::Save Error deleting line ID subkey WindowsCloudNextGen, error: 5
2017-12-15T11:24:59.092Z [ 5560] INFO StatePersister::Save Overwriting state file C:\ProgramData\Sophos\AutoUpdate\data\status\SophosUpdateStatus.xml
2017-12-15T11:24:59.093Z [ 5560] INFO UpdateLogic::SyncAndInstall Skipping product install as Sync failed.
2017-12-15T11:24:59.095Z [ 5560] ERROR RegistryReporter::SetUpdateStatus SetUpdateStatus: Failed to write LastUpdateTime:5
2017-12-15T11:24:59.095Z [ 5560] ERROR RegistryReporter::SetUpdateStatus SetUpdateStatus: Failed to write Result:5
2017-12-15T11:24:59.095Z [ 5560] ERROR RegistryReporter::SetUpdateStatus SetUpdateStatus: Failed to write FallbackInUse: 5
2017-12-15T11:24:59.096Z [ 5560] ERROR RegistryReporter::SetUpdateStatus SetUpdateStatus: Failed to write FirstFailedUpdateTime: 5
2017-12-15T11:24:59.097Z [ 5560] INFO IPCSender::Write IPCSender::Write: Writing message: <?xml version="1.0" encoding="utf-8" ?><Config type="RMSEndUpdate"><ErrorMessage><ID>SDDSDownloadFailed</ID><StringID>107</StringID><Sender>SophosUpdate</Sender><Insert>WindowsCloudNextGen</Insert><Insert>dci.sophosupd.com/.../ErrorMessage><ReadableMessage>ERROR: Download of WindowsCloudNextGen failed from server dci.sophosupd.com/.../Config>
2017-12-15T11:24:59.097Z [ 5560] INFO WinMain SophosUpdate has completed with the result 2.
2017-12-15T11:24:59.097Z [ 7972] INFO IPCSender::ProcessSend IPCSender::ProcessSend: Send message: <?xml version="1.0" encoding="utf-8" ?><Config type="RMSEndUpdate"><ErrorMessage><ID>SDDSDownloadFailed</ID><StringID>107</StringID><Sender>SophosUpdate</Sender><Insert>WindowsCloudNextGen</Insert><Insert>dci.sophosupd.com/.../ErrorMessage><ReadableMessage>ERROR: Download of WindowsCloudNextGen failed from server dci.sophosupd.com/.../Config>
2017-12-15T11:24:59.097Z [ 7972] INFO IPCSender::ProcessSend IPCSender::ProcessSend: No messages in queue, starting to wait
2017-12-15T11:25:00.097Z [ 7972] INFO IPCSender::ProcessSend IPCSender::ProcessSend exiting
2017-12-15T11:25:00.097Z [ 7972] INFO `anonymous-namespace'::SenderThreadFn::operator() Sender thread finished.
2017-12-15T11:25:00.100Z [ 5560] ERROR RegistryVersionPersister::Save Error writing version for line ID WindowsCloudNextGen, error: 5
2017-12-15T11:25:00.101Z [ 5560] ERROR RegistryVersionPersister::Save Error writing version for line ID WindowsCloudHitmanProAlert, error: 5
2017-12-15T11:25:00.102Z [ 5560] ERROR RegistryVersionPersister::Save Error writing version for line ID WindowsCloudAV, error: 5
2017-12-15T11:25:00.103Z [ 5560] ERROR RegistryVersionPersister::Save Error deleting line ID subkey WindowsCloudAV, error: 5
2017-12-15T11:25:00.103Z [ 5560] ERROR RegistryVersionPersister::Save Error deleting line ID subkey WindowsCloudHitmanProAlert, error: 5
2017-12-15T11:25:00.104Z [ 5560] ERROR RegistryVersionPersister::Save Error deleting line ID subkey WindowsCloudNextGen, error: 5
2017-12-15T11:25:00.105Z [ 5560] INFO StatePersister::Save Overwriting state file C:\ProgramData\Sophos\AutoUpdate\data\status\SophosUpdateStatus.xml



This thread was automatically locked due to age.
  • 0

    Given that you are getting:

    error: 5

    For registry operations, it makes me wonder if Sophos Endpoint Defense is causing issues here as that is access denied.  I'm sure Process Monitor would back that up.

    You should have management of these computer via MCS, if you disable tamper protection for one of these clients. Does the next update succeed?

    I wonder if you're seeing an issue such as this:

    https://community.sophos.com/kb/en-us/127757

    Regards,

    Jak

     

  • 0 in reply to jak

    It doesn't seem to check in fully i.e. I can turn off tamper through the web console but it doesn't kick in.

    I can enter the override code on the client and turn off tamper but it just turns it back on immediately.

    Which I believe means I need the computer on a desk in front of me to boot into Safe Mode.  Which is great in a mobile workforce.

    I think the PCs were imaged but Sophos was definitely not part of the image.

    Sorry Jak, I'm not sure whether you work for Sophos so if you do this isn't aimed at you personally as I know you're trying to help, and if you don't work for Sophos I do appreciate you taking the time to try to help, but right now I'm seriously regretting ever going with Sophos - AV products that manage to lock themselves out of keys and files that they need to update you couldn't even make that up.

    I would say I'm looking forward to uninstalling it but given the dogs mess of an uninstall process I'm really not :(

  • +1 in reply to Paul Hutchings

    It does sound like Tamper Protection is not behaving correctly.

    Imaged or not, did you check if the volume "number" referenced in the value of SystemDrive under this key:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\Paths\

    Is the same as the value reported running in a command prompt:

    fltmc volumes

    For example, on my computer in an Admin command prompt:

    fltmc volumes | find "C:"
    returns:
    C: \Device\HarddiskVolume3 NTFS

    and 
    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\Paths!SystemDrive 

    contains:
    \Device\HarddiskVolume3

    So these match.  Is your computer with the issue OK?

    Otherwise, as you say, you would have to boot into Safe-mode, maybe just rename sophosed.sys under \windows\system32\drivers\ to sophosed.sys.rename and reboot to disable Endpoint Defense.  With this done, does the computer then update as expected?

  • 0 in reply to jak

    It was the HarddiskVolume* issue, thank you very much.

    Do you have any idea how that can happen?  I don't get involved in imaging/installing our PCs but Sophos is installed once it's on the desk so it's 100% not part of any kind of image, though the PC's are imaged prior to coming onsite.

  • 0 in reply to jak

    Hi Jak,

     

    In my case 

     

    fltmc volumes | find "C:"

     

    C:                              \Device\HarddiskVolume3                  NTFS

     

    and 
    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Sophos Endpoint Defense\Paths!SystemDrive 

    contains:
    \Device\HarddiskVolume5

     

    Now how to solve the issue. Please help. Thanks

  • 0 in reply to Adamshaffi Shaik

    Is that an endpoint or server?

    If you can boot to safe mode you can manually update the registry key paths to ensure the paths are correct.

    I think this is a resolved issue in the latest SED driver.  I believe this is currently just starting to roll out to servers and is in EAP.  I think endpoints have a version of SED with the fix. Hence the question what platform this is on.

    Regards,

    Jak

Unfiltered HTML