This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

Heartbeat Stops Reporting Knocking PC Off Network

Good morning all, I am beginning to have a serious issue with our PC's since moving over to Sophos Intercept X Advanced. I am beginning to have PC's stop sending their Heartbeat, Sophos Central shows the PC as no longer sending a heartbeat and I believe what happens is the system goes into a protected mode, preventing the PC from having any network access, actually preventing its IP so my network icon immediately changes from connected to no connection. Most of the time the PC's are bouncing back and resume sending the heartbeat without any intervention on my part.

However some are not bouncing back and left in this disconnected state until, I manually assign an IP address to get the heartbeat back then I can simply reset the network card back to DHCP. This is beginning to be a serious issue, as yesterday this happened to 3 PC's and today I just back from our other plant because of the same issue on another PC.

Any users experienced something similar? Intercept X is managed through Sophos Central as well we do have a Sophos XG firewall that ties everything together. I have reached out to Sophos support but in the meantime if anyone has some suggestions to possible look at, please let me know, I'm at a loss as to why this is happening, if it's a bug, I'm not sure.

Thank you



This thread was automatically locked due to age.
Parents
  • Oh I should also add before someone asks, when Sophos Central reports the device as no longer sending a heartbeat, I do acknowledge the event to allow the PC status to go back to green because my firewall rule does have the Security heart enabled on the PC must at least have an amber status or greater to maintain its network connection.

  • FormerMember
    0 FormerMember in reply to SophosNewby

    Heartbeat can periodically not be sent when services are starting up/recycling or any other network interruption. Also, at reboot, the services don't start in a set order so the network elements could start up before Health does and therefore there is no Heartbeat to send. 

    In general, the endpoint itself isn't stopping the traffic - it is the firewall that is dropping the traffic. Is this traffic bound for an external resource or an internal one?

  • Thank you for the reply, all traffic is blocked, the PC no longer has a network connection period, it can't be pinged, restarting the device does nothing, it stays in a blocked state completely. Until I manually assign another IP address does the PC become pingable again. Once it notices the heartbeat, all traffic is resumed. These PC's have been on this firewall now for 4 years, with another anti-virus product, zero issues. Since moving the PC to Intercept X, allowing the PC to basically self quarantine itself in the policies and changing the heartbeat requirements on the main firewall rule has this become an issue so it is definitely since making these changes. Had another PC today that its heartbeat stopped sending on Friday at 5.30pm, I had to go over to the plant with the PC today, manually assign its IP, allow the heartbeat the resume, and put back on DHCP to get it communicating on the network again, it does not bounce back  on it's own.

  • FormerMember
    0 FormerMember in reply to SophosNewby

    Okay, that isn't Heartbeat. Do you have Device Isolation turned on in your Endpoint policy?

  • FormerMember
    0 FormerMember in reply to FormerMember

    unless your DHCP server is beyond the firewall from the machine. What is the network topography?

  • Thank you RichardP , I do have Device Isolation turned on under the Threat Protection Policy on the Endpoint.

  • Our topography would be considered the standard star topography. The firewall, dhcp server, PC's all communicate amongst themselves.

  • I have turned off Device Isolation to monitor over the next few days, I assumed it was necessary to have this enabled, in case of a PC possibly being compromised and then preventing any lateral movement through the network.

Reply Children
  • FormerMember
    0 FormerMember in reply to SophosNewby

    yes - that is exactly what it does. The balance comes though on how much your prioritize that protection over network availability. With Isolation turned on - you will see service interruptions from time to time. 

  • Thank you, I do have my firewall LAN rule requiring an amber heartbeat so if it something is compromised and changes the status threshold, that should still provide me some sort of reliable security laterally but I shall see.

  • FormerMember
    0 FormerMember in reply to SophosNewby

    Think of it this way:

    SFOS will protect from infections spreading between network segments that have to transverse it to talk to each other.

    Isolation protects from infections spreading inside a network segment.

  • However just a quick note, in theory with device isolation, if the administrator acknowledges the device has gone into this red status, should that now allow the PC to resume communication after its status has been reviewed and deemed no threat? Is that not the purpose of reviewing an issue and then acknowledging it? Because on Sophos Central the device then changes from red status to green status, the software on the device changes from red to green but yet it still never recovers, seems to be a bug no?

  • FormerMember
    0 FormerMember in reply to SophosNewby

    Isolation only goes into effect when the Health on the machine is RED. So, once you complete the action that turns it to GREEN, the isolation will be removed on that machine. You can confirm in the registry: Computer\HKEY_LOCAL_MACHINE\SOFTWARE\WOW6432Node\Sophos\Health\Status
    The admin, health, service, and threat values are what you care about. 1 is good. 3 is red. So, you can check what the machine thinks it is.

    Since you have two layers that can be affecting network traffic you might be encountering a collision. So, endpoint turns RED, it isolates and reports to the SFOS that it has RED health. SFOS also blocks. If the SFOS is safeguarding the comms to the DHCP server too - this could be a problem. 

    Do you have local DHCP servers in each segment? Or are they all beyond the SFOS from the perspective of the clients? Like you have the DHCP server in segment A and all machines have to route through the SFOS to get there?

  • Everything on site is LOCAL, the XG being part of the local LAN network along with the DHCP server. XG is our gateway/firewall device, if it were removed the devices would simply not have a way out to the internet, that is it. Should this occur again with device isolation off I'll be sure to check the registry key status.

  • I have turned off Device Isolation to monitor over the next few days, I assumed it was necessary to have this enabled, in case of a PC possibly being compromised and then preventing any lateral movement through the network.

    I suggest you leave this enabled.This is a big security plus but of course this isolation knocks you out in such cases where you have trouble with endpoints getting isolated for no visible reason.

  • I just had a thought, do you think the issue is related to Synchronized User ID Authentication? Reason I ask is in my logs under Authentication I have dozens and dozens of entries, example below.

    2021-06-21 13:16:26 Authentication messageid="17702" log_type="Event" log_component="Firewall Authentication" log_subtype="Authentication" status="Failed" user="USERNAME" user_group="" client_used="Heartbeat" auth_mechanism="Local" reason="wrong credentials" src_ip="X.X.X.X" message="User USERNAME failed to login to Firewall through Local authentication mechanism from X.X.X.X because of wrong credentials" name="" src_mac=""

    There are no credentials with my users, and synchronized user ID authentication is enabled by default. 

    This is a statement directly on how this system works.

    "If the client Heartbeat is lost or missing, the heartbeat daemon will logout the user from the firewall as a Synchronized ID user, however other client authentication mechanisms may still apply."