This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Most Clients Shown As 'Disconnected' in SEC 5.5.0

Hi folks,

We are running Sophos Enterprise Console (SEC) 5.5.0 on a Windows 2008 R2 Enterprise (64-bit) Server.

I have recently noticed that more than 50% of our client PCs to which Sophos Endpoint Security & Control has been deployed are shown as 'disconnected' in SEC. I have carried out a ping-sweep of the network and can confirm that most, if not all, of these PCs are actually powered on, connected to the network and working fine.

Only after I restart the Sophos Message Router Service on the client PCs do they then change their status to 'connected' in SEC. I have no wish to carry this task out on several hundred client PCs individually as you can imagine, so I'm hoping someone can possibly shed some light on what may be happening here and suggest a solution to this issue?

Many thanks,

John P



This thread was automatically locked due to age.
Parents
  • I'm sure there will be others offering advice, but from my experience, it's most likely that the Remote Management System (RMS) that cannot communicate on the required ports.  You can try with the telnet command from the server to the endpoint and vice versa on the required ports.

    You may like to watch the video below on setting up a GPO to allow the required ports (this means you don't have to go round to each computer).  Watch from the 9 minute mark...

    There is also the deployment guide which mentions the ports. http://www.sophos.com/deployment - click the 'Allowing computers to report' link on the right-hand rail.

     - - - - - - - - - - - -

    Communities Moderator, SOPHOS
    Knowledge Base  |  @SophosSupport  |  Video tutorials
    Remember to like a post.  If a post (on a question thread) solves your question use the 'This helped me' link.

  • Hi guys,

    Thank you for your prompt and helpful replies.

    Christian, I have to admit that I may be a bit lax in checking my SEC installation. Unfortunately, my duties dictate that I cannot spend as much time as I'd like (or indeed, need) to monitor our SEC installation. Wearing too many hats at times methinks!!

    19.09.2017 08:28:19 0B28 I SOF: C:\ProgramData/Sophos/Remote Management System/3/Router/Logs/Router-20170919-072819.log
    19.09.2017 08:28:19 0B28 I Sophos Messaging Router 4.1.1.127 starting...
    19.09.2017 08:28:19 0B28 I Setting ACE_FD_SETSIZE to 138
    19.09.2017 08:28:19 0B28 I Initializing CORBA...
    19.09.2017 08:28:19 0B28 I Connection cache limit is 10
    19.09.2017 08:28:20 0B28 I Router::ConfigureSslContext: keeping legacy compatibility of TLS 1 and TLS 1.1.
    19.09.2017 08:28:20 0B28 I Creating ORB runner with 4 threads
    19.09.2017 08:28:20 0B28 I Compliant certificate hashing algorithm.
    19.09.2017 08:28:20 0B28 I This computer is part of the domain SECRAT
    19.09.2017 08:28:20 0B28 I This router's IOR:
    IOR:010000002600000049444c3a536f70686f734d6573736167696e672f4d657373616765526f757465723a312e300000000100000000000000a0000000010102000a0000003132372e302e302e310001204100000014010f004e5550000000210000000001000000526f6f74504f4100526f7574657250657273697374656e740003000000010000004d657373616765526f7574657200000003000000000000000800000001000e01004f4154010000001800000001000e01010001000100000001000105090101000000000014000000080000000100a60086000220
    19.09.2017 08:28:20 0B28 E Localhost address (e.g. 127/8) found in the IOR
    19.09.2017 08:28:20 0B28 E This router's IOR is invalid
    19.09.2017 08:28:20 0B28 I This computer is part of the domain ****
    19.09.2017 08:28:20 0B28 I Reading router table file
    19.09.2017 08:28:20 0B28 I Host name: DNIA16807
    19.09.2017 08:28:20 0B28 I Local IP addresses: 10.63.14.118 
    19.09.2017 08:28:20 0B28 I Resolved name: DNIA16807.****.****.****
    19.09.2017 08:28:20 0B28 I Resolved alias/es: 
    19.09.2017 08:28:20 0B28 I Resolved IP addresses: 127.0.0.1 
    19.09.2017 08:28:20 0B28 I Resolved reverse names/aliases: DNIA16807.****.****.**** 
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01AE532F, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01AE8D1F, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01AFA2D6, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01AFDE91, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0B28 I Waiting for messages...
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01B0F5D5, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01B13019, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01B245FF, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01B28197, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01B63A35, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01B67608, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01B78F73, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01B7C79E, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01B8DE7B, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01B91916, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01BA3121, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01BA6A99, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01BB8324, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01BBBBFE, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01BF7130, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-GetStatus-Reply
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01BF7254, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01BF726D, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-GetStatus-Reply
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01BF9C9C, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-GetStatus-Reply
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01BFB08B, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:28:20 0CB8 I Routing to parent: id=01BFC6D0, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-GetStatus-Reply
    19.09.2017 08:28:20 0CCC W Delivery failed(Timeout) for message type EM-GetStatus-Reply, originator Router$DNIA16807:450391.Agent
    19.09.2017 08:28:20 0CCC W Delivery failed(Timeout) for message type EM-GetStatus-Reply, originator Router$DNIA16807:450391.Agent
    19.09.2017 08:28:20 0CCC W Delivery failed(Timeout) for message type EM-GetStatus-Reply, originator Router$DNIA16807:450391.Agent
    19.09.2017 08:28:20 0B28 I RouterSystemCheck::onInfoPortsUsed() - number of user ports 3, max number of user ports 15360
    19.09.2017 08:28:20 0CCC W Delivery failed(Timeout) for message type EM-GetStatus-Reply, originator Router$DNIA16807:450391.Agent
    19.09.2017 08:28:22 0C94 I Client::LogonPushPush() successfully called back to client
    19.09.2017 08:28:22 0C94 I Logged on Agent as a client
    19.09.2017 08:28:22 0CB8 I Routing to Agent: id=03C0C716, origin=Router$DNIA16807:450391, dest=Router$DNIA16807:450391.Agent, type=EM-ClientLogon
    19.09.2017 08:28:22 0CAC I Sent message (id=03C0C716) to Agent
    19.09.2017 08:28:22 0CB8 I Received message for this router
    19.09.2017 08:28:22 0CB8 I EM-NotifyClientUpdates originator Router$DNIA16807:450391.Agent
    19.09.2017 08:28:22 0CB8 I Routing to Agent: id=07C0C716, origin=Router$DNIA16807:450391, dest=Router$DNIA16807:450391.Agent, type=EM-NotifyClientUpdates-Reply
    19.09.2017 08:28:22 0CB0 I Sent message (id=07C0C716) to Agent
    19.09.2017 08:28:57 0CB8 I Routing to parent: id=01C0C739, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-GetStatus-Reply
    19.09.2017 08:33:48 0CB8 I Routing to parent: id=01C0C85C, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 08:34:10 0CCC W Delivery failed(Timeout) for message type EM-EntityEvent, originator Router$DNIA16807:450391.Agent
    19.09.2017 08:34:14 0CB8 I Routing to parent: id=01C0C876, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-GetStatus-Reply
    19.09.2017 09:28:20 0B28 I RouterSystemCheck::onInfoPortsUsed() - number of user ports 5, max number of user ports 15360
    19.09.2017 10:28:20 0B28 I RouterSystemCheck::onInfoPortsUsed() - number of user ports 5, max number of user ports 15360
    19.09.2017 11:28:21 0B28 I RouterSystemCheck::onInfoPortsUsed() - number of user ports 5, max number of user ports 15360
    19.09.2017 12:28:21 0B28 I RouterSystemCheck::onInfoPortsUsed() - number of user ports 5, max number of user ports 15360
    19.09.2017 12:40:21 0CCC W Delivery failed(Timeout) for message type EM-EntityEvent, originator Router$DNIA16807:450391.Agent
    19.09.2017 12:40:50 0CB8 I Routing to parent: id=01C10242, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:28:21 0B28 I RouterSystemCheck::onInfoPortsUsed() - number of user ports 5, max number of user ports 15360
    
    
    
    19.09.2017 13:36:47 1AA4 I SOF: C:\ProgramData/Sophos/Remote Management System/3/Router/Logs/Router-20170919-123647.log
    19.09.2017 13:36:47 1AA4 I Sophos Messaging Router 4.1.1.127 starting...
    19.09.2017 13:36:47 1AA4 I Setting ACE_FD_SETSIZE to 138
    19.09.2017 13:36:47 1AA4 I Initializing CORBA...
    19.09.2017 13:36:47 1AA4 I Connection cache limit is 10
    19.09.2017 13:36:48 1AA4 I Router::ConfigureSslContext: keeping legacy compatibility of TLS 1 and TLS 1.1.
    19.09.2017 13:36:48 1AA4 I Creating ORB runner with 4 threads
    19.09.2017 13:36:48 1AA4 I Compliant certificate hashing algorithm.
    19.09.2017 13:36:48 1AA4 I This computer is part of the domain ****
    19.09.2017 13:36:48 1AA4 I This router's IOR:
    IOR:010000002600000049444c3a536f70686f734d6573736167696e672f4d657373616765526f757465723a312e300000000100000000000000a4000000010102000d00000031302e36332e31342e313138000001204100000014010f004e5550000000210000000001000000526f6f74504f4100526f7574657250657273697374656e740003000000010000004d657373616765526f7574657200000003000000000000000800000001001d01004f4154010000001800000001001d01010001000100000001000105090101000000000014000000080000000100a60086000220
    19.09.2017 13:36:48 1AA4 I Successfully validated this router's IOR
    19.09.2017 13:36:48 1AA4 I Reading router table file
    19.09.2017 13:36:48 1AA4 I Host name: DNIA16807
    19.09.2017 13:36:48 1AA4 I Local IP addresses: 10.63.14.118 
    19.09.2017 13:36:48 1AA4 I Resolved name: DNIA16807.****.****.****
    19.09.2017 13:36:48 1AA4 I Resolved alias/es: 
    19.09.2017 13:36:48 1AA4 I Resolved IP addresses: 10.63.14.118 
    19.09.2017 13:36:48 1AA4 I Resolved reverse names/aliases: DNIA16807.****.****.**** 
    19.09.2017 13:36:48 1AA4 I Waiting for messages...
    19.09.2017 13:36:48 1988 I Routing to parent: id=01AFA2D6, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1AA4 I RouterSystemCheck::onInfoPortsUsed() - number of user ports 4, max number of user ports 15360
    19.09.2017 13:36:48 1988 I Routing to parent: id=01AFDE91, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01B0F5D5, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01B13019, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01B245FF, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01B28197, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1B2C I Getting parent router IOR from 10.63.20.72:8192
    19.09.2017 13:36:48 1988 I Routing to parent: id=01B63A35, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01B67608, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01B78F73, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01B7C79E, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01B8DE7B, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01B91916, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01BA3121, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01BA6A99, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01BB8324, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01BBBBFE, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01BF7254, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01BFB08B, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01C0C739, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-GetStatus-Reply
    19.09.2017 13:36:48 1988 I Routing to parent: id=01C0C85C, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1988 I Routing to parent: id=01C0C876, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-GetStatus-Reply
    19.09.2017 13:36:48 1988 I Routing to parent: id=01C10242, origin=Router$DNIA16807:450391.Agent, dest=EM, type=EM-EntityEvent
    19.09.2017 13:36:48 1B2C I Received parent router's IOR:
    IOR:010000002600000049444c3a536f70686f734d6573736167696e672f4d657373616765526f757465723a312e300000000100000000000000a4000000010102000c00000031302e36332e32302e3732004fc000004100000014010f004e5550000000210000000001000000526f6f74504f4100526f7574657250657273697374656e740003000000010000004d657373616765526f757465720000000300000000000000080000000100b700004f415401000000180000000100b700010001000100000001000105090101000000000014000000080000000100a60086000220
    19.09.2017 13:36:48 1B2C I Successfully validated parent router's IOR
    19.09.2017 13:36:48 1B2C I Accessing parent
    19.09.2017 13:36:48 1B2C I SSL handshake done, local IP address = 10.63.14.118
    19.09.2017 13:36:48 1B2C I Parent is Router$SV-AV-01
    19.09.2017 13:36:48 1B2C I RouterTableEntry::LogonToParentRouter() - logging on as active consumer
    19.09.2017 13:36:48 1B0C I SSL handshake done, local IP address = 10.63.14.118
    19.09.2017 13:36:48 1B2C I RouterTableEntry state (router, logging on): Router$SV-AV-01 is passive consumer, passive supplier
    19.09.2017 13:36:48 1B2C I Logged on to parent router as Router$DNIA16807:450391
    19.09.2017 13:36:48 1B2C I This computer is part of the domain SECRAT
    19.09.2017 13:36:48 1C14 I Sent message (id=01AFA2D6) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01AFDE91) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01B0F5D5) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01B13019) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01B245FF) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01B28197) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01B63A35) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01B67608) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01B78F73) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01B7C79E) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01B8DE7B) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01B91916) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01BA3121) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01BA6A99) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01BB8324) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01BBBBFE) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01BF7254) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01BFB08B) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01C0C739) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01C0C85C) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01C0C876) to Router$SV-AV-01
    19.09.2017 13:36:48 1C14 I Sent message (id=01C10242) to Router$SV-AV-01
    
    
    

    Anyway, attached (I hope) is a copy of the client message router logs showing the situation before and after the Sophos Message Router service restart.

    Hope this helps.

    Ruckus, many thanks for your input. I will review the material you suggested and will keep you posted of any developments.

    Best regards,

    John P

    2 x SG450 (Version 9.714-4)

    HA = Active-Passive

  • Hello Ian,

    64-bit systems
    strange, RMS is a 32bit application, but apparently it works (when it works) regardless.

    Christian

  • Hi John,

    We have been using the netstat -ano | findstr :8194 command you provided.  The Level 2 Escalation engineer contacted us this morning after waiting for the GES\DEV team to put a mentoring note on the case and advises the issue seems to be with the IOR string not finding the IP address of the server, it is finding the loop back address. 127.0.0.1.  This would suggest they haven't been following this thread.  The engineer provided a link to http://sophos.com/kb/17268 and requested we add an entry to the hosts file on the server for 127.0.0.1  This we have done to no avail.  The article referenced the Sophos Network Communications Report.  The report which can be accessed via Start > All Programs > Sophos > Sophos Endpoint Security and Control > View Sophos Network Communications Report lists the current state of communications with Enterprise Console and attempts to identify any problems. 

    Ian.

  • Hello all,

    I wonder if I'm the idiot here ...
    This is IMO a ridiculous answer. I can only imagine that this came about because only partial information has been passed on with an extra big amount of self assurance.

    add an entry to the hosts file
    yeah, for thousands of endpoints, and yes, assigning static addresses for all hosts and adapters in DHCP? And maybe I'm misinterpreting the logs but there's clearly a Local IP addresses: 10.63.14.118 entry before it spits out 127.0.0.1. You can't achieve more than that with etc\hosts.

    Whatever Level 2 is these days, I'd like to know what information was passed to GES/DEV and what they have put into the note.   

    Christian

  • Either I am looking in the wrong place or I cant see 10.63.*.*, from the log I can see. The loopback suggestion was 1 of 3 suggestions to be passed to the customer on what he could do to get around the issue however was also advised it was probably not the best suggestion to use.

    So the customer has given us a router log and in this we can see the below.

    25.09.2017 08:56:49 0D88 T IPAddressSet::InitialiseWithHost() called
    25.09.2017 08:56:49 0D88 T Added host network address:172.30.*.*:0
    25.09.2017 08:56:49 0D88 T Added host network address:127.0.0.1:0
    25.09.2017 08:56:49 0D88 T IPAddressSet::InitialiseWithHost() returns
    25.09.2017 08:56:49 0D88 I Local IP addresses: 172.30.*.*
    25.09.2017 08:56:49 0D88 I Resolved name: scopat-*.*.local
    25.09.2017 08:56:49 0D88 I Resolved alias/es:
    25.09.2017 08:56:49 0D88 I Resolved IP addresses: 127.0.0.1

    so scopat-*.*.local is resolving to loopback not 172.30.*.*, as far as I can see this is causing the IOR to show loopback also which obviously is why they are seeing issues.

    IOR:010000002600000049444c3a536f70686f734d6573736167696e672f4d657373616765526f757465723a312e300000000100000000000000a0000000010102000a0000003132372e302e302e310001204100000014010f004e5550000000210000000001000000526f6f74504f4100526f7574657250657273697374656e740003000000010000004d657373616765526f757465720000000300000000000000080000000100e001004f415401000000180000000100e001010001000100000001000105090101000000000014000000080000000100a60086000220

    If we parse this we see:

    _IIOP_ParseCDR: byte order LittleEndian, repository id <IDL:SophosMessaging/MessageRouter:1.0>, 1 profile
    _IIOP_ParseCDR: profile 1 is 160 bytes, tag 0 (INTERNET), LittleEndian byte order
    (iiop.c:parse_IIOP_Profile): bo=LittleEndian, version=1.2, hostname=127.0.0.1, port=8193, object_key=<....NUP...!........RootPOA.RouterPersistent.........MessageRouter>
    (iiop.c:parse_IIOP_Profile): encoded object key is <%14%01%0F%00NUP%00%00%00%21%00%00%00%00%01%00%00%00RootPOA%00RouterPersistent%00%03%00%00%00%01%00%00%00MessageRouter>
    (iiop.c:parse_IIOP_Profile): non-native cinfo is <iiop_1_2_1_%2514%2501%250F%2500NUP%2500%2500%2500%2521%2500%2500%2500%2500%2501%2500%2500%2500RootPOA%2500RouterPersistent%2500%2503%2500%2500%2500%2501%2500%2500%2500MessageRouter@tcp_127.0.0.1_8193>
    object key is <#14#01#0F#00NUP#00#00#00!#00#00#00#00#01#00#00#00RootPOA#00RouterPersistent#00#03#00#00#00#01#00#00#00MessageRouter>;
    no trustworthy most-specific-type info; unrecognized ORB type;
    reachable with IIOP 1.2 at host "127.0.0.1", port 8193

    The IOR Response should be giving us the IP Address 172.30.*.* not 127.0.0.1

    I believe this is what support are getting at here and this is also why it is believed to be an environmental issue as covered in https://community.sophos.com/kb/en-us/17268

  • Hello WomboCombo,

    I'm not directly involved, just know-it-all.

    an environmental issue
    perhaps. That it seems to happen after certain Windows updates suggests that it's not RMS' fault in the first place.

    Resolved IP addresses: 127.0.0.1
    obviously
    not what it should be. But we (the customers) have no idea what to check. I bet everything looks normal when you inspect it on the endpoint (even if you could run a trace at this early stage). We don't know what API the Router uses or which methods it calls. So we can't check if this is perhaps noticeable somewhere else.
    The IOR is perhaps built by some Windows function, but the process is not under control of the customer.

    But then comes the part where Sophos should be able to help (even if it's Microsoft's fault in the first place):
    As everything works correctly "a little bit later" and occurs only after boot it's likely related to the initialization of the networking stack. The Router recognizes the invalid IOR and wait for an adapter change notification - which it doesn't get though. Could be a race condition. But none of the What to do points in 17268 apply. It would resolve itself were the Router to check again after, say, a minute.

    We (no, they) have already tested some workarounds like Delayed Start, adapter dis-/en-able, Router restart - all this shows it's a rather short-lived situation. Admittedly it's likely something that hasn't been seen before and this is not covered in the Router's logic. Sophos should at least acknowledge this issue and comment on the workarounds - and not suggest actions that can't work.

    Just may two cents
    Christian

  • Hello WomboCombo,

    Your post states, "The loopback suggestion was 1 of 3 suggestions to be passed to the customer on what he could do to get around the issue however was also advised it was probably not the best suggestion to use."  JohnP initiated this thread.  I don't know whether he was the recipient of the three suggestions of what to do and that the loopback solution was probably not the best solution.  The email we received included:

    As a test could you to try the following please.
    1. Add an entry in the hosts file for 127.0.0.1 to translate to the IP of this server.
    2. Then see if this resolve the issue.

    Are you, or John perhaps, aware of what the other two proposed solutions comprised?

    Ian.

  • Hi guys,

    Thank you all for your continued input. I appear to be having more of a return on the forum than from Sophos Support. As of yet I have received no recommendations from them on how this issue may be resolved.

    As for the '3 suggestions', I was working under the impression that they were recommendations made by Christian earlier in this discussion and were: Delayed Start on Sophos Message Router service, Restart Sophos Message Router service or adapter disable/enable.

    We amended Group Policy to delay starting the Sophos Message Router service as it was the least path of resistance and easily implemented. However, it has proven not to be the cure for our current ill. We are still seeing PCs as 'disconnected' in SEC when they are actually online.

    Ultimately, I admit, this issue may not lie at the feet of Sophos but I would expect that their Support Team would (as Christian quite rightly states) "acknowledge this issue and comment on the workarounds - and not suggest actions that can't work".

    I'm also looking into possible network issues, cabling etc. to see if there may be an issue at that end which may have contributed to this current situation. I'm making slight headway, but it's too early to say if our network is at fault here. Suffice to say that it appears (so far) it is only Windows 10 Enterprise 2015 LTSB PCs with a Broadcom NexXtreme Gigabit Ethernet adapter (driver version 16.8.1.0) which are affected by this issue. I will update this post if I find anything awry.

    Many thanks,

    John

    2 x SG450 (Version 9.714-4)

    HA = Active-Passive

  • Hello John,

    still seeing PCs as 'disconnected' in SEC
    from the
    descriptions and logs posted here it seems that "this issue" manifests itself at Router startup. The telltale 127.0.0.1 with the invalid IOR should either be in the first lines of the current Router log or not there at all. And of course - did the service actually start delayed? Endpoints might not yet have applied the GPO.

    Thanks for the soft- and hardware details.

    Christian

  • Hi Christian,

    I selected a few 'problem' PCs and can confirm that the Group Policy update did apply. The service did indeed start, approximately 2 minutes after the PC booted up. Needless to say, the 'invalid IOR' entry was still in the log files. and the PCs shown as 'disconnected'.

    The quest goes on!!

    Many thanks,

    John

    2 x SG450 (Version 9.714-4)

    HA = Active-Passive

  • Hi Ian,

    Have you had any further feedback from Sophos on this issue?

    Despite submitting outputs from the Sophos Diagnostic Utility last week, I haven't heard anything back from Support on this problem.

    Many thanks,

    John

    2 x SG450 (Version 9.714-4)

    HA = Active-Passive

Reply
  • Hi Ian,

    Have you had any further feedback from Sophos on this issue?

    Despite submitting outputs from the Sophos Diagnostic Utility last week, I haven't heard anything back from Support on this problem.

    Many thanks,

    John

    2 x SG450 (Version 9.714-4)

    HA = Active-Passive

Children
  • Hello John,

    I am working on this issue currently and one thing we have noticed is that a telnet hostname 8192 returns the loopback address in the IOR until a restart of the Sophos Message Router is completed. We are currently looking deeper into ORB logging to try and identify the root cause of this. If you could message me your case reference I will ensure both cases are added to the open development ticket I am creating.

    If you can enable the below and also send us in new router logs after the below has been done it would help in our investigations.

    On rare occasions it may be necessary to enable additional "orb debug logging" for the 'Sophos Message Router' service, to do so:

    1. Stop the 'Sophos Message Router' service.
    2. Browse to: 
      • HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Sophos Message Router
    3. Modify the ImagePath value by adding the following switches to the end of the image path value:
      -ORBDebug -ORBDebugLevel 10 -ORBVerboseLogging 2
    4. Start the 'Sophos Message Router' service.
    5. Reboot the machine
    6. Collect the logs located in C:\ProgramData\Sophos\Remote Management System\3\Router\Logs\
  • Hi WomboCombo,

    I have messaged you with the details you requested.

    Many thanks for your assistance in this matter.

    Best regards,

    John P

    2 x SG450 (Version 9.714-4)

    HA = Active-Passive

  • Hello John & WomboCombo,

    Just a quick update... an obviously competent Sophos engineer logged onto one of our affected laptops yesterday afternoon.  Over a two hour period he performed a wide and thorough range of tests that included telnet connectivity to the SEC and packet inspection using RawCap along with an extensive review of log files.  All to no avail I'm afraid.  He is going to contact the 'Corba' people.  I believe this is a result of the router logs which report: Initializing CORBA; Creating an ORB runner with 4 threats, ultimately identifying the router's IOR and the correct local IP address of the workstation and the incorrect resolved IP of 127.0.0.1.

    More to this than meets the eye.  I shall keep you posted.

    Kind regards,

    Ian.

  • Hi Ian,

    Many thanks for that update. It does indeed look like the issue goes much deeper than originally thought.

    I, at last, heard back from Sophos Support who informed me that may call has been escalated and will be addressed by a next level engineer.......when one becomes available.

    Many thanks,

    John

    2 x SG450 (Version 9.714-4)

    HA = Active-Passive

  • Carrying on carrying on.  Earlier this week we received an email informing us that the engineer was still awaiting a response and that the JIRA ticket had not been updated.

    Today we were requested to create a new software subscription updating policy (subscribing to a previous recommended subscription) together with associated groups, updating and anti-virus and HIPS policies. We applied this to two affected workstations.

    The original version was 10.7.2 VE3.69.2.  The two affected workstation are now running 10.6.4 VE3.67.3. After the policies were applied and rebooting when prompted the workstation are showing connected.

    I'll keep you posted.

    Ian.

  • Hello Ian,

    I assume you've rebooted more than once.
    As far as I can see there were some changes in RMS, it's 4.1.0.140 vs. 4.1.1.127.

    Thanks for the update
    Christian

  • Hi guys!

    Any news regarding this issue?

    Thanks,

    Rok

  • We understand that Sophos are unable to replicate the fault.  Last week we were contacted to review our DNS settings.  Straws and clutching came to mind.  We provided the requested  information, stated we do not have any DNS issues and reminded them that, following their instructions, we have some affected users setup in the console who are running v10.6 without issue.

    Regards,

    Ian.

     

  • I thought that this issue is related to CORBA as it was mentioned here and not DNS. The fact is that RMS was upgraded and the only workable workaround is to run scheduled powershell script that restarts rms service every morning on all clients because only restarted or shutdown/power on machines are affected.

     

    Regards, Rok

  • Hi,

     

    We are experiencing same problems, so I assume it is on going problem. What is the latest response from Sophos?

    Thanks,

     

    Regards,