Heartbeat is always a bit tricky here.
As we have several rules with block clients with no HB, the impact off technical heartbeat issues is always high.
Endpoints have the latest official Client versions from Central. Currently 2024.2.3.4.0
For years, the heartbeatd.log on the firewall is always full of SSL errors.
The most frequent error is
Incoming connection from 10.xxx.xxx.xxx failed. SSL error:
Incoming connection from 172.xxx.xxx.xxx failed. SSL error:
The IP ranges are directly connected and routed by the XGS. IP 10 is SSL VPN, 172 is LAN.
We have 5-10 per minute of them.
During SSL handshake the Firewall answers with a encrypted alert.
The Clients sends a RST packet during SSL handshake after Server Hello.
hb log:
[2024-09-24 08:11:28.752Z] WARN HBSession.cpp[26394]:344 bufferDisconnectEvent - Incoming connection from 10.yyy.yyy.131 failed. SSL error:
[2024-09-24 08:11:29.436Z] WARN HBSession.cpp[26394]:344 bufferDisconnectEvent - Incoming connection from 10.yyy.yyy.131 failed. SSL error:
[2024-09-24 08:11:33.624Z] WARN HBSession.cpp[26394]:344 bufferDisconnectEvent - Incoming connection from 10.yyy.yyy.131 failed. SSL error
tcpdump taken on FW:
filter for client IP and TLS
99100 2024-09-24 10:11:28,724384 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data 99109 2024-09-24 10:11:28,752450 10.yyy.yyy.131 52.5.76.173 TLSv1.2 768 Application Data, Application Data, Application Data 99111 2024-09-24 10:11:28,752605 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data 99144 2024-09-24 10:11:28,831272 10.yyy.yyy.131 52.5.76.173 TLSv1.2 207 Client Hello 99150 2024-09-24 10:11:28,831711 52.5.76.173 10.yyy.yyy.131 TLSv1.2 1422 Server Hello 99167 2024-09-24 10:11:28,850463 52.5.76.173 10.yyy.yyy.131 TLSv1.2 1171 Certificate, Server Key Exchange, Certificate Request, Server Hello Done 99195 2024-09-24 10:11:28,896512 10.yyy.yyy.131 52.5.76.173 TLSv1.2 1092 Certificate 99223 2024-09-24 10:11:28,967032 10.yyy.yyy.131 52.5.76.173 TLSv1.2 451 Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message 99225 2024-09-24 10:11:28,967313 52.5.76.173 10.yyy.yyy.131 TLSv1.2 107 Change Cipher Spec, Encrypted Handshake Message 99261 2024-09-24 10:11:29,015035 10.yyy.yyy.131 52.5.76.173 TLSv1.2 115 Application Data 99262 2024-09-24 10:11:29,015078 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data 99286 2024-09-24 10:11:29,043512 10.yyy.yyy.131 52.5.76.173 TLSv1.2 1052 Application Data, Application Data 99287 2024-09-24 10:11:29,043758 52.5.76.173 10.yyy.yyy.131 TLSv1.2 136 Application Data 99289 2024-09-24 10:11:29,063528 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data 99290 2024-09-24 10:11:29,063532 10.yyy.yyy.131 52.5.76.173 TLSv1.2 93 Application Data 99355 2024-09-24 10:11:29,137979 10.yyy.yyy.131 52.5.76.173 TLSv1.2 301 Application Data 99357 2024-09-24 10:11:29,138049 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data 99594 2024-09-24 10:11:29,416737 10.yyy.yyy.131 52.5.76.173 TLSv1.2 294 Application Data 99595 2024-09-24 10:11:29,416921 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data 99602 2024-09-24 10:11:29,427061 10.yyy.yyy.131 52.5.76.173 TLSv1.2 294 Application Data 99603 2024-09-24 10:11:29,427198 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data 99692 2024-09-24 10:11:29,536222 10.yyy.yyy.131 52.5.76.173 TLSv1.2 207 Client Hello 99694 2024-09-24 10:11:29,536401 52.5.76.173 10.yyy.yyy.131 TLSv1.2 1422 Server Hello 99715 2024-09-24 10:11:29,565157 52.5.76.173 10.yyy.yyy.131 TLSv1.2 1171 Certificate, Server Key Exchange, Certificate Request, Server Hello Done 99768 2024-09-24 10:11:29,648546 10.yyy.yyy.131 52.5.76.173 TLSv1.2 1092 Certificate 99839 2024-09-24 10:11:29,714316 10.yyy.yyy.131 52.5.76.173 TLSv1.2 451 Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message 99841 2024-09-24 10:11:29,714571 52.5.76.173 10.yyy.yyy.131 TLSv1.2 107 Change Cipher Spec, Encrypted Handshake Message 99893 2024-09-24 10:11:29,784144 10.yyy.yyy.131 52.5.76.173 TLSv1.2 115 Application Data 99894 2024-09-24 10:11:29,784206 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data 99902 2024-09-24 10:11:29,803184 10.yyy.yyy.131 52.5.76.173 TLSv1.2 1052 Application Data, Application Data 99903 2024-09-24 10:11:29,803383 52.5.76.173 10.yyy.yyy.131 TLSv1.2 136 Application Data 99906 2024-09-24 10:11:29,822811 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data 99907 2024-09-24 10:11:29,822834 10.yyy.yyy.131 52.5.76.173 TLSv1.2 93 Application Data 102253 2024-09-24 10:11:33,552360 10.yyy.yyy.131 52.5.76.173 TLSv1.2 294 Application Data 102255 2024-09-24 10:11:33,552468 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data 102283 2024-09-24 10:11:33,624670 52.5.76.173 10.yyy.yyy.131 TLSv1.2 87 Encrypted Alert 102301 2024-09-24 10:11:33,643726 10.yyy.yyy.131 52.5.76.173 TLSv1.2 207 Client Hello 102303 2024-09-24 10:11:33,643910 52.5.76.173 10.yyy.yyy.131 TLSv1.2 1422 Server Hello 102311 2024-09-24 10:11:33,672658 52.5.76.173 10.yyy.yyy.131 TLSv1.2 1171 Certificate, Server Key Exchange, Certificate Request, Server Hello Done 102335 2024-09-24 10:11:33,744687 10.yyy.yyy.131 52.5.76.173 TLSv1.2 1092 Certificate 102373 2024-09-24 10:11:33,806659 10.yyy.yyy.131 52.5.76.173 TLSv1.2 451 Client Key Exchange, Certificate Verify, Change Cipher Spec, Encrypted Handshake Message 102375 2024-09-24 10:11:33,806957 52.5.76.173 10.yyy.yyy.131 TLSv1.2 107 Change Cipher Spec, Encrypted Handshake Message 102437 2024-09-24 10:11:33,896198 10.yyy.yyy.131 52.5.76.173 TLSv1.2 115 Application Data 102438 2024-09-24 10:11:33,896244 52.5.76.173 10.yyy.yyy.131 TLSv1.2 93 Application Data ... 197843 2024-09-24 10:14:18,918870 10.yyy.yyy.131 52.5.76.173 TLSv1.2 97 Application Data 205073 2024-09-24 10:14:33,917076 10.yyy.yyy.131 52.5.76.173 TLSv1.2 97 Application Data
full dump available
one other is SSL error: SSL routines:ssl3_read_bytes sslv3 alert certificate expired
- as far as I know from the past this is due to the client caching old HB certificates and uses them, when connecting. When it recognizes, it has expired, it uses the next one in the HB cert store and has success.
SFOS 20.0.1
Edited TAGs
[edited by: Erick Jan at 9:35 AM (GMT -7) on 24 Sep 2024]