Some Sophos services are not running/missing

I have at least 20% of the computers on my network now reporting "Some Sophos services are not running/missing"

This was not the case 2 weeks ago, and invariably it is "Sophos System Protection Service is not started".

This involves a visit to that computer, pushing the user out of the way, running "Services" as administrator, which then involves typing the admin password under a user's nose.

I am sure this is not how Sophos Central is meant to work. It is highly inconvenient, and at worst liable for a security breach.

How do I start these services remotely ?

I could log in through Remote Desktop, but this would Log Out the User

  • In reply to sabdul:

    Agree! We'll be moving away once out contact is up too!! Only another 2 years to go!

  • In reply to Jay Parmar:

    I just installed Sophos on a brand new machine and what you know it worked for 5 minutes and then I got an email

     

    Sophos Central Event Details for ---------

    What happened: The service has stopped running.

    Where it happened: 4483

    User associated with device: --------------

    How severe it is: High

    What Sophos has done so far: n/a

    What you need to do: Start the service.

  • In reply to sabdul:

    Did it complete the install OK? The computer was then restarted and a service failed to start? Which service or services do you have issues with on this computer?

  • In reply to jak:

    Yes it installed fine. Computer restarted and then services stopped. 

    Security Health

     
     
    No malware or potentially unwanted applications
    •  Sophos MCS Agent
    •  Sophos Web Intelligence Service
    •  Sophos Anti-Virus
    •  Sophos Anti-Virus Status Reporter
    •  Missing: Sophos AutoUpdate Service
    •  Sophos MCS Client
    •  Sophos Device Control Service
    •  Missing: Sophos Heartbeat
    •  Sophos System Protection Service
    •  Sophos Web Control Service
    •  Sophos Endpoint Defense
    •  Sophos Network Threat Protection
  • In reply to sabdul:

    I don't believe the install was 100% successful based on that service list and it's actually quite an odd state.

    1) There is no longer a "Sophos Heartbeat" service since version 2 so very odd it's down as missing.  There isn't even an installed component called Heartbeat any more as it's part of the Sophos Network Threat Protection component.

    2) AutoUpdate is the last component to get installed when you run the Central installer and I think it is also gets uninstalled when you run the installer over the top on an existing installation I believe.

    Did AutoUpdate install correctly? 
    How is the installer (SophosSetup.exe) being run, could it have been re-run and therefore started to remove components?
    Was this a migration from on-premise Enterprise Console to Sophos Central?

    When you run the Central installer it logs to:
    C:\ProgramData\Sophos\CloudInstaller\Logs\SophosCloudInstaller_date_time.log
    You should see it log each component being installed at the bottom of the log, E.g.

    ...
    2018-05-19T14:05:59.8193379Z INFO : Installing Component: Sophos Diagnostic Utility
    2018-05-19T14:05:59.8247964Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:00.3600192Z INFO : Installed Sophos Diagnostic Utility: 0 (reboot code: 0)
    2018-05-19T14:06:01.0147385Z INFO : Installing Component: Sophos Endpoint Uninstaller for Windows (64-bit)
    2018-05-19T14:06:01.0172184Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:01.0182103Z INFO : Trying IProductSetup interface.
    2018-05-19T14:06:01.2726580Z INFO : Installed Sophos Endpoint Uninstaller for Windows (64-bit): 0 (reboot code: 0)
    2018-05-19T14:06:01.6536743Z INFO : Installing Component: Sophos Endpoint Defense for Windows (64-bit)
    2018-05-19T14:06:01.6561532Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:11.7984697Z INFO : Installed Sophos Endpoint Defense for Windows (64-bit): 0 (reboot code: 0)
    2018-05-19T14:06:11.9620928Z INFO : Installing Component: Sophos Management Communication System for Windows
    2018-05-19T14:06:11.9640618Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:12.7037342Z INFO : Installed Sophos Management Communication System for Windows: 0 (reboot code: 0)
    2018-05-19T14:06:13.4497178Z INFO : Installing Component: Sophos Standalone Engine (64-bit)
    2018-05-19T14:06:13.4521982Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:16.5112608Z INFO : Installed Sophos Standalone Engine (64-bit): 0 (reboot code: 0)
    2018-05-19T14:06:16.7811181Z INFO : Installing Component: Sophos File Scanner (64-bit)
    2018-05-19T14:06:16.7840942Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:20.0799969Z INFO : Installed Sophos File Scanner (64-bit): 0 (reboot code: 0)
    2018-05-19T14:06:20.2328786Z INFO : Installing Component: Sophos Clean for Windows (64-bit)
    2018-05-19T14:06:20.2353582Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:23.1036579Z INFO : Installed Sophos Clean for Windows (64-bit): 0 (reboot code: 0)
    2018-05-19T14:06:23.2166723Z INFO : Installing Component: Sophos Endpoint Self Help Installer (64-bit)
    2018-05-19T14:06:23.2186561Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:24.2929500Z INFO : Installed Sophos Endpoint Self Help Installer (64-bit): 0 (reboot code: 0)
    2018-05-19T14:06:24.4298444Z INFO : Installing Component: Sophos Health
    2018-05-19T14:06:24.4323250Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:26.2609754Z INFO : Installed Sophos Health: 0 (reboot code: 0)
    2018-05-19T14:06:26.4483437Z INFO : Installing Component: Sophos UI (64-bit)
    2018-05-19T14:06:26.4508247Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:27.4459752Z INFO : Installed Sophos UI (64-bit): 0 (reboot code: 0)
    2018-05-19T14:06:27.6651664Z INFO : Installing Component: Sophos Endpoint Firewall Management (64-bit)
    2018-05-19T14:06:27.6805426Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:28.2603598Z INFO : Installed Sophos Endpoint Firewall Management (64-bit): 0 (reboot code: 0)
    2018-05-19T14:06:32.0192587Z INFO : Installing Component: Sophos Anti-Virus for Windows
    2018-05-19T14:06:32.0311630Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:06:32.0326853Z INFO : Trying IProductSetup interface.
    2018-05-19T14:07:16.6480405Z INFO : Installed Sophos Anti-Virus for Windows: 0 (reboot code: 1)
    2018-05-19T14:07:20.4865822Z INFO : Installing Component: Sophos Device Encryption
    2018-05-19T14:07:20.4890621Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:07:24.3563720Z INFO : Installed Sophos Device Encryption: 0 (reboot code: 0)
    2018-05-19T14:07:24.4560685Z INFO : Installing Component: Sophos ML Engine (64-bit)
    2018-05-19T14:07:24.4595407Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:07:24.6931568Z INFO : Installed Sophos ML Engine (64-bit): 0 (reboot code: 0)
    2018-05-19T14:07:25.7694754Z INFO : Installing Component: Malicious Traffic Detection (64-bit)
    2018-05-19T14:07:25.7724515Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:07:31.0150061Z INFO : Installed Malicious Traffic Detection (64-bit): 0 (reboot code: 0)
    2018-05-19T14:07:31.1583498Z INFO : Installing Component: Sophos HitmanPro Alert (64 bit)
    2018-05-19T14:07:31.1613266Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:07:33.6522977Z INFO : Installed Sophos HitmanPro Alert (64 bit): 0 (reboot code: 1)
    2018-05-19T14:07:33.7648914Z INFO : Installing Component: Sophos AutoUpdate XG
    2018-05-19T14:07:33.7678654Z INFO : Trying IProductSetup2 interface.
    2018-05-19T14:07:33.7683629Z INFO : Trying IProductSetup interface.
    2018-05-19T14:07:36.8227273Z INFO : Installed Sophos AutoUpdate XG: 0 (reboot code: 0)
    2018-05-19T14:07:36.8242160Z INFO : Command 'SetupPlugin' completed with success with reboot code '1' and error message ''.

    Here you can see that the last component to be installed was "Sophos AutoUpdate XG" and it was successful.  I would therefore check this log from the install to be sure.

    I don't see how you would get the missing Heartbeat service state without it being a install over the top of an existing install of version 1.  I don't think even having paused updated would prevent you getting V2 now.

    I would suggest the first thing to do is check if you have Heartbeat installed as a MSI and remove it if this is the case. Given this KBA https://community.sophos.com/kb/en-us/122126, it has the GUID for the Heartbeat component down as follows so the following should remove it if it is installed:

    MsiExec.exe /qn /X{DFFA9361-3625-4219-82C2-9EF011E433B1} REBOOT=ReallySuppress

    But you may need to check under: HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Microsoft\Windows\CurrentVersion\Uninstall for "Heartbeat" and if you do find it, use the GUID in the UninstallString.

    Regards,
    Jak

  • In reply to jak:

    Jak, and all others, I had this problem with a customer too. So far, what did the trick was this KB article: https://community.sophos.com/kb/en-us/125467

    However, it's unacceptable, considering that our customer has around a hundred PCs with these kind of issues, to do it manually at all the machines. My question then, is there a way to turn this procedure in to a bat? I figured it wouldn't be so hard, but I'm missing some basics on my command-line knowledge. Something like this:

     

     - Is there a way to disable the tamper for a lot of machines at once? -

    net stop "Sophos AutoUpdate Service"

    ren C:\ProgramData\Sophos\AutoUpdate\Cache\decoded C:\ProgramData\Sophos\AutoUpdate\Cache\decoded-old

    ren C:\ProgramData\Sophos\AutoUpdate\data\warehouse C:\ProgramData\Sophos\AutoUpdate\data\warehouse-old

    del C:\ProgramData\Sophos\AutoUpdate\data\status\SophosUpdateStatus.xml

    net start "Sophos AutoUpdate Service"

    And finally, "cd" to the folder where the Auto Update exec is located and run it.

     

    Is it possible? Thanks in advance!

  • In reply to Marco Ramos:

    Does anyone have a simple reliable way to completely uninstall Central please?

    Something that can be run as SYSTEM as a scheduled/GPO initiated task ideally.

    I would hope/assume Sophos have a removal tool.

    I found this https://community.sophos.com/kb/en-us/122126 but it's a year old so probably out of date, and we frequently seem to have removal issues when clients get broken.

    We need to be able to uninstall remotely on hundreds of endpoints so need something as reliable as possible.

  • In reply to Paul Hutchings:

    Hi,

    Apologies for the misunderstanding, except the known issue regarding the Device Encryption Service randomly not starting/stopping we do not have cases created for multiple services being hampered on large scale. We are constantly trying to work with you to take your cases on priority. It will help us if someone can reach us with an active case so we can prioritize the investigation.

  • In reply to Gowtham Mani:

    Hi Gowtham Mani,

    >> It will help us if someone can reach us with an active case so we can prioritize the investigation.

    Okay use our active case #8143748. Please escalate, investigate and resolve this problem.

  • In reply to FritsHeemstra:

    Hi FritsHeemstra,

    Thanks for the case number, I will have it followed up internally regarding the case.

  • In reply to FritsHeemstra:

    Hi,

    It might be worth checking if the problematic services are marked for deletion, i.e. pending a system restart.

    When the Service Control Manager (SCM) of Windows tries to delete a service, if a handle it held open to the service, which could be by any process, the service will be marked for deletion rather than removed there and then.  Evidence of this is a delete flag for the service:

    HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\[servicename]\
    DeleteFlag = 1 (DWORD)

    If you restart the computer, then when the computer starts back up the SCM (services.exe) will be able remove the service. Which would allow for it to be re-created.

    Note: if the process that has opened a handle to the service that is marked for deletion terminates or eventually closes the handle it should be removed there and then so a restart may not be required.

    If this is the case, then you could just restart the computer.  SAV (if a missing service belongs to SAV, e.g. Sophos Web Intelligence, Sophos Web Filter, etc) then on the next IDE or product update it will detect the service is missing and re-install it by forcing a major update.

    So it may just be a restart and the next update will fix your issue, which would probably be the next ide update in a couple of hours.

    The reason I know this is that I had an issue where the Sophos Web Intelligence service was missing.  It turned out that an inventory agent (enumerating services) wasn't closing the handle it had to the services.  As a result on a Sophos update of SAV, the Sophos Web Intelligence service was removed as expected but had to be marked for deletion, the SAV installer couldn't then add it back because you can't create a service with the same name as a "marked for deletion" on.  So the service was still present but marked for deletion.  On the next restart the service was removed by the SCM so it was lost and reported as missing.  It was only on the next update of Sophos was the service re-added.  The thing is it's quite tricky to know who (which process) has opened a handle to a service.  I used API monitor monitor the API calls that open the service control manager (msdn.microsoft.com/.../ms684323(v=vs.85).aspx) and open service (msdn.microsoft.com/.../ms684330(v=vs.85).aspx) to work it out.  In the end the inventory agent software was updated to ensure it closed its handles to services when done.

    I hope this past problem I had is helpful.

    Regards,
    Jak

  • In reply to FritsHeemstra:

    Hi FritsHeemstra,

    I believe one of our Product management team members reached you regarding the mentioned case and updated you regarding the fix. I will be following up internally regarding this with the product team and periodically update this thread with further developments.

  • In reply to sabdul:

    Here we are once again.....OMG.

    Some services missing or not running. Heavy Heavy Sigh!!!

     

     

    This is really really depressing.

    More administrative time eaten up because of this foolish software.

    Perhaps this wouldn't be so bad if Sophos could actually uninstall itself. However, I have Disabled every single Sophos Service. Rebooted.

    I go to uninstall and it says: Update is currently in progress. Please wait for the update to finish. How is this even possible if I have ALL services disabled??

    So I leave it for 30 min, go back and then its says "Attempting to stop the service: Sophos AutoUpdate Service." The SERVICE is DISABLED you morons.

    Uninstall failed. Update is currently in progress. Please wait for the update to finish. 

    HOW DO I STOP THIS UPDATE!!!!

    SOPHOS.....GIVE ME A TOOL THAT RIPS THIS SOFTWARE OUT NO MATTER WHAT THE %^&$ IT IS DOING.

     

    HERE IS SOPHOS AT ITS FINEST, 100% unable to manage malware. 100% useless crap.

    OHHHH LOOKY LOOK.....ALL SOPHOS CRAP DISABLED, YET THIS POS STILL THINKS AUTOUPDATE IS RUNNING. ARRGGGGG....

    FAIL FAIL FAIL FAIL ON EVERY LEVEL.

    SO HOW DO I UNINSTALL THIS?? NOW, NOT LATER...NOW!!

    Ok..I -edited- give up. Another -edited- service call to the pundits at Sophos because this useless POS will not uninstall. -edited- .

    Why am I swearing...because my end user hasn't been able to work all -edited- day because of Sophos crap. Three hours I have been at this and I am still no further ahead than I was three -edited- hours ago.

    Rebooted at least 6 times already. Still cannot uninstall this crap.

    Where do I send my Invoice for troubleshooting your fucking product. You owe me MIN $500.00 and thats just for today.

     

     

     

     

     

     

  • In reply to Howiedog:

    Hello Howiedog,

    Sorry about the issues that you are experiencing with our product. However, in order to further assist you I am going to request that you please refrain from swearing. I have edited your post to remove profanity. Other than that, I understand that you are frustrated since the product is not behaving as it should, and I am going to try my best to help you.

    If you are still receiving "Update is currently in progress. Please wait for the update to finish" despite having disabled all the services, please give this a try:

    - Make a full backup of your Registry
    - Turn off tamper protection on the affected system
    - Open regedit and navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Sophos\Autoupdate\UpdateStatus
    - Look for a key named IsUpdating
    - Delete the key
    - Re-attemp uninstalling Sophos.

    You can also try running a batch file to force the uninstallation

    Please let me know the outcome.

    Per your last response, it seems you have contacted support already. If you want to send me the ticket number, I will have a look at it. 

    I will be following up with you via DM so that I can gather more information and escalate your concerns accordingly.