Issue: Sophos Central Admin – US-West region - Delays with the enforcement of Central policies on managed endpoints.

**Update 9** Root cause analysis KBA has been published: see knowledge base article for the latest.

**Update 8** As part of a routine database maintenance task customers may notice a few intermittent install and policy rendering failures. Please retry before contacting support. 7/17/2017 8:00 AM PST

**UPDATE 7** Some customers may notice a few intermittent install failures, please retry before contacting Sophos Support. 7/14/2017 2:00 PM PST

**UPDATE 6** Installations are being processed normally, service is restored. Please re-download installer from Central. 7/14/2017 9:00 AM PST

**UPDATE 5** Installations are now working as of July 13, 2017 19:00 UTC-5. See knowledge base article for the latest.

**UPDATE 4** New installs likely to still fail. http://centralstatus.sophos.com/#!/ has latest update. 

**UPDATE 3** System is now processing backlogs. Please see last updates here.

**UPDATE 2** Issue is ongoing, apologies. Impacts all areas within Central that rely on MCS communication between client and Central. 7/13/2017 8:00 AM PST

**UPDATE** Development has identified root cause and is working on a fix. 

Hello,

We are seeing delays with policy changes and enforcement in Sophos Central (US-West region) as well as installation failures due to inability of new endpoint installations to initially register. Our engineers are working to restore latency. Please note your endpoints remain protected. Updates will be provided on this thread.

KBA: https://community.sophos.com/kb/en-us/126477

Thank you,

Bob

  • Any update when this will be resolved. I was in the middle of a massive deployment and haven't been able to get any done today!

  • In reply to Matthew Nasholm:

    I am also not able to deploy new installation... 3 failed so far... and I won't attempt any further.

  • In reply to Simon Ng:

    We are unable to Unlock the hosted server on central Sophos console from yesterday, there is a delay in unlocking the server, its taking hours to get unlock, what exactly is the issue ? 

  • In reply to Matthew Nasholm:

    Posting update now.

  • Hi Bob.  This is extremely critical as it puts all new installations on hold.  Do we have an ETA on the repair?

    Thank you,

    Keith

  • In reply to K_M:

    Hi Keith,

    ASAP. Best case scenario by eod.

    Thank you,

    Bob

  • What is Sophos standard operating procedure for outages as far as updates are concerned?  Thus far, I've had to piece together Twitter, and this message thread, and the Sophos status page to receive status changes to this issue.  

    When there are status changes they seem to be a bit misleading, for example telling us soon when it won't be does not help our business stakeholders.

    SaaS products will have outages, I understand.  What I don't understand is why the SaaS provider does not have a clear method to communicate updates, and when the service will be restored.  Perhaps updates every hour, even if the update is "nothing new to report" as it provides a timed mechanism to check.  Currently I have to obsessively refresh this thread to receive status changes.

  • In reply to Bianson:

    If I've received an error of 136 for a new installation, showing that it registered but couldn't install due to lack of internet connection, will it automatically finish it's install/download once the issue is resolved? Or will I need to do some sort of uninstall/manual update?

    Thanks,

    Geoff T.

  • In reply to Bianson:

    Bianson
    Best case scenario by eod.

    Hi Bob,

    What does Sophos consider end of day? 5PM eastern time? 5PM pacific?

  • In reply to Christopher Curwood:

    EST. It is unlikely fix will be available today, Central development team continues to investigate.

  • In reply to Bianson:

    Bianson
    Central development team continues to investigate.

    Does the earlier statement that root cause was identified still hold true, or is root cause still being investigated and unknown?

  • In reply to Bianson:

    so what do we do in the meantime, with clients that are unprotected? EC-managed client has been removed and cannot be reinstalled -- blocked and removed by the bits of Central client that did get installed.

  • In reply to Dan Orth:

    Hi Dan,

    The Sophos Central Status page is linked in the KBA. http://centralstatus.sophos.com/#!/

    Thank you,

    Bob

  • In reply to Christopher Curwood:

    Earlier statement holds true, sorry if I wasn't clear.

  • If this is only affecting US-West region why can't you failover to US-Central region or US-East region or do those not exist? How about sending us to someware else in the world even if it is slow it would al least work, is there no redundancy built into the network and services.

    Also where is the failover for this situation?

    I think there should be some details should be given out to the customers so we can evaluate the reason and what will be done in the future to keep this from happening again.

    I am currently on a business trip an can not complete my work.