Sophos Heartbeat At risk

Hi,

We’re having frequent issues with a number of Lenovo ThinkPad T14s laptops showing red heartbeat (at risk) status in the XG dashboard. Firewall rules with heartbeat restrictions are also blocking traffic since the status is red. In Sophos Central and on the endpoint the status is green, and all services are running and healthy. Heartbeat status is usually stuck as red and does not change to green or missing. Older laptops such as ThinkPad 490s/T480s are not affected by this.

Heartbeat.log on the endpoints shows it is sending “health:3” as health status which I understand is red health. The log also shows “Connection closed (network error)” after every heartbeat communication to 52.5.76.173 on port 8347.

XG is on the latest firmware 18.0.5-MR5-Build586 and the endpoints on the latest version and rebooted.

Is this a known issue?

Heartbeat.log

a 2021-05-24T07:50:42.217Z [6284:22556] - Received request to enable enhanced application control
a 2021-05-24T07:50:42.217Z [6284:22556] - Sending endpoint state list request
a 2021-05-24T07:50:42.217Z [6284:22556] - Sending login status.
a 2021-05-24T07:50:42.218Z [6284:22556] - Received response to endpoint state list request, size: 1
a 2021-05-24T07:50:52.160Z [6284:22556] - Sending health status: {"health":3}
a 2021-05-24T07:50:52.163Z [6284:22556] - Received notification of endpoint state changes, size: 1
a 2021-05-24T07:50:52.483Z [6284:22556] - Received request to disable enhanced application control for C:\program files (x86)\microsoft\edge\application\msedge.exe
a 2021-05-24T07:50:52.486Z [6284:22556] - Received request to disable enhanced application control for C:\program files (x86)\microsoft\edge\application\msedge.exe
a 2021-05-24T07:55:19.836Z [6284:22556] - Received request to disable enhanced application control for C:\program files (x86)\internet explorer\iexplore.exe
a 2021-05-24T07:55:20.224Z [6284:22556] - Received request to disable enhanced application control for C:\program files\internet explorer\iexplore.exe
a 2021-05-24T08:05:12.930Z [6284:22556] - Sending network status. Active Interfaces:
MAC: 34:2E:B7:0E:6C:A8 - INET: x.x.x.107 - INET6: fe80::1c3a:a66e:1766:8a9e
MAC: 54:05:DB:25:FF:E5 - INET: x.x.x.109 - INET6: fe80::3c8f:c24:ebad:c943
a 2021-05-24T08:05:12.935Z [6284:22556] - Connection closed (network error).
a 2021-05-24T08:05:13.993Z [6284:22556] - Connection succeeded.
a 2021-05-24T08:05:13.993Z [6284:22556] - Connected to 'ec13ffbf-c542-41b2-8ff6-dc070df936d9' at IP address 52.5.76.173 on port 8347
a 2021-05-24T08:05:14.027Z [6284:22556] - Sending network status. Active Interfaces:
MAC: 34:2E:B7:0E:6C:A8 - INET: x.x.x.107 - INET6: fe80::1c3a:a66e:1766:8a9e
MAC: 54:05:DB:25:FF:E5 - INET: x.x.x.109 - INET6: fe80::3c8f:c24:ebad:c943
a 2021-05-24T08:05:14.038Z [6284:22556] - Received request to enable enhanced application control
a 2021-05-24T08:05:14.038Z [6284:22556] - Sending endpoint state list request
a 2021-05-24T08:05:14.039Z [6284:22556] - Sending login status.
a 2021-05-24T08:05:14.039Z [6284:22556] - Received response to endpoint state list request, size: 0
a 2021-05-24T08:05:15.439Z [6284:22556] - Received request to disable enhanced application control for C:\program files (x86)\microsoft\edge\application\msedge.exe
a 2021-05-24T08:05:25.367Z [6284:22556] - Sending health status: {"health":3}
a 2021-05-24T08:05:27.935Z [6284:22556] - Received notification of endpoint state changes, size: 1
a 2021-05-24T08:06:25.227Z [6284:22556] - Connection closed (network error).
a 2021-05-24T08:06:27.312Z [6284:22556] - Connection succeeded.
a 2021-05-24T08:06:27.312Z [6284:22556] - Connected to 'ec13ffbf-c542-41b2-8ff6-dc070df936d9' at IP address 52.5.76.173 on port 8347
a 2021-05-24T08:06:27.314Z [6284:22556] - Connection closed (network error).

Parents Reply Children
  • We are experiencing a similar issue. Our devices are Lenovo ThinkPad 14 & Thinkbook 13s, stuck each day for hours "at risk".

    Was there any update to this issue?

    a 2021-05-25T07:01:09.147Z [5168:6608] - Connection closed (network error).
    a 2021-05-25T07:01:10.180Z [5168:6608] - Connection succeeded.
    a 2021-05-25T07:01:10.180Z [5168:6608] - Connected to 'd5340c68-99e5-439b-8429-95cd57919e88' at IP address 52.5.76.173 on port 8347
    a 2021-05-25T07:01:10.192Z [5168:6608] - Sending network status. Active Interfaces:
    MAC: 48:2A:E3:9A:70:44 - INET: 192.168.96.131 - INET6:
    a 2021-05-25T07:01:10.229Z [5168:6608] - Received request to enable enhanced application control
    a 2021-05-25T07:01:10.230Z [5168:6608] - Sending login status.
    a 2021-05-25T07:01:13.690Z [5168:6608] - Sending health status: {"admin":1, "health":1, "service":1, "threat":1}
    a 2021-05-25T07:03:59.780Z [5168:6608] - Received request to disable enhanced application control for C:\program files (x86)\microsoft\edge\application\msedge.exe
    a 2021-05-25T07:54:17.028Z [5168:5656] - ----------------------------------------------------------------------------------------------------
    a 2021-05-25T07:54:17.031Z [5168:5656] - Stopped Heartbeat
    a 2021-05-25T07:54:17.031Z [5168:5656] - ----------------------------------------------------------------------------------------------------

    a 2021-05-25T07:54:19.858Z [13220:14304] - Starting Heartbeat version 1.11.194.0
    a 2021-05-25T07:54:19.858Z [13220:14304] - ----------------------------------------------------------------------------------------------------
    a 2021-05-25T07:54:19.882Z [13220:3992] - Connection succeeded.
    a 2021-05-25T07:54:19.882Z [13220:3992] - Connected to 'd5340c68-99e5-439b-8429-95cd57919e88' at IP address 52.5.76.173 on port 8347
    a 2021-05-25T07:54:19.901Z [13220:3992] - Sending network status. Active Interfaces:
    MAC: 48:2A:E3:9A:70:44 - INET: 192.168.96.131 - INET6:
    a 2021-05-25T07:54:19.926Z [13220:3992] - Received request to enable enhanced application control
    a 2021-05-25T07:54:19.927Z [13220:3992] - Sending login status.
    a 2021-05-25T07:54:19.937Z [13220:3992] - Sending health status: {"health":3}
    a 2021-05-25T09:19:55.924Z [13220:3992] - Received request to disable enhanced application control for C:\program files (x86)\microsoft\edge\application\msedge.exe

  • Thank you, I'll keep an eye on this ticket. Please PM me if you need anything else. 

    Thanks,

    Yashraj Singha

    Community Team Lead, Support & Services| Sophos Technical Support
    Support Videos | Product Documentation@SophosSupport | Sign up for SMS Alerts
    If a post solves your question use the 'Verify Answer' button.

  • Hi ,

    Sending health status: {"admin":1, "health":1, "service":1, "threat":1}

    This suggests that there was a threat in your system. Are there are any outstanding alerts on Sophos Central? 

    What is the current status of the endpoint is on the device and Sophos Central? Are you facing a similar issue as mentioned in this thread where the device is reported with Red status on Sophos (XG) Firewall and green on the device and Sophos Central? 

    Thanks,

    Yashraj Singha

    Community Team Lead, Support & Services| Sophos Technical Support
    Support Videos | Product Documentation@SophosSupport | Sign up for SMS Alerts
    If a post solves your question use the 'Verify Answer' button.

  • Hi Chris,

    I’ve done some troubleshooting yesterday and today and on computers where the docking station firmware (ThinkPad Thunderbolt 3 Dock Gen 2)  has been updated to the latest version, the problem has cleared for now at least, but it will probably take a few days to see if this was the cause.

  • Yes, the device endpoint is reporting green and the device is green on Sophos Central. The XG firewall is reporting it "At Risk".

  • We are using "Lenovo ThinkPad USB-C Dock Gen2 (UK) 40AS0090UK", my colleague confirms it is up to date on version: April 13, 2021 (1.0.8.0.3231)

    As an aside - in case anyone needs this information - if we bypass the networking on the docking station - the issue seems to resolve itself. We will monitor over the coming days, but initial signs look good.

  • I would suggest you open a support case at this stage and get an SDU from the affected machine. On the endpoint, go to the Sophos panel and click on About > Open Endpoint Self Help Tool > Launch SDU

    With that, you can provide to support and they can look at the registry values and compare with your SFOS logs to see if this is a mismatch in reporting or if there is a value stuck in the registry.

    RichardP

    Snr. New Product Introduction Engineer | CISSP | Sophos Technical Support
    Support Videos | Product Documentation | @SophosSupport | Sign up for SMS Alerts
    If a post solves your question use the 'Verify Answer' link.

  • It does look like this was a docking station issue. After upgrading the ThinkPad Thunderbolt 3 Dock and also the older ThinkPad Pro Dock firmware, the stuck in red health and also a green/missing flapping issue has all cleared. The first docking stations were upgraded two weeks ago and has been stable since.

  • Hi

    Thanks letting us know. Please reach out if you face any further issues. 

    Thanks,

    Yashraj Singha

    Community Team Lead, Support & Services| Sophos Technical Support
    Support Videos | Product Documentation@SophosSupport | Sign up for SMS Alerts
    If a post solves your question use the 'Verify Answer' button.