LAN Clients unable to receive External NTP Server Time

Hi Brad Peter 

I would recommend you to capture drop packets and also check traffic by packet capture.

You may create source IP based LAN to WAN firewall rule and position the rule on top and do not apply any web or app filter policy or scanning and verify.

For packet capture, please use below given articles.

https://community.sophos.com/kb/en-us/123189

https://community.sophos.com/kb/en-us/127647

I had to create a specific firewall rule for this-

Source - LAN

Device - ANY  (You can define this more)

Destination - WAN

Device - the location you are pulling NTP from ex ntp.pool.org enter as IP or FQDN (host name)

Service - NTP

Then place at the top, you do not need protection for this since it will only allow to the specific destination, on the specific port.

Hi,

are you using the XG supplied definition of NTP or did you create your own? NTP needs to be UDP.

Ian

I have created a custom service for NTP mirroring the XG rule, with UDP and TCP at 1:65535/123.

I have examined the packet captures and there are no dropped packets on 123 and it appears that all traffic is being passed appropriately (e.g.  I see the NTP packet outbound and the reply back to the WAN IP address).  It just never makes it back to the internal client system.  I have a number of different OSes and each one of them is unable to receive the NTP data  I've verified local OS firewalls are off, etc.

Did you create the associated network policy with it?  Or Step 2 in here: https://community.sophos.com/kb/en-us/123034

Hi,

Once a connection is established you do not see traffic in both directions unless you are using an analyser. Logviewer only shows the connection and because it is created from your devices there is never any returned traffic showing.

Why do you think that the devices are not receiving the NTP updates?

Ian

I run an ESX server at home along with a virtual Windows Domain Controller.  Shortly following my move to XG from UTM, my PC clients all started showing about 10 minutes fast.  This is what led me to begin investigating the NTP issue.

Within ESX i can watch ntpq and they sit at RefID .INIT. for a very long time.  At some point they will return an IP address, but it is many hours before it pulls a valid time from source and each successful poll is also hours apart from the previous poll.  At the time of this writing, here is the ntpq output (the "when" is showing 16 hours since last poll):

remote              refid              st t   when poll reach delay      offset    jitter
==============================================================================
ntp.wdc1.us.lea 130.133.1.10  2  u    16h  64        0 44.178   59.819  0.000

My Windows Domain controllers also will not sync with an external time server in a timely fashion as it had before.  Performing a w32tm /resync returns "The computer did not resync because no time data was available."  But, as with the ESX host, it does get a response periodically.

It's almost as if the return time data is passed along intermittently.  One thing I've thought of, and before I move to look into this, I'd like to pose this question:  I'm running ESX on a 4-core single-socket CPU.  Understanding that NTP is UDP,  How sensitive are XG virtual appliances to higher load averages with respect to UDP traffic?  If the LA is high (2-3 on single vCPU), will the firewall appliance drop packets in favor of TCP traffic?  Would I expect to see these drops anywhere in a log, or are they just silently dropped?

SFVH_VM01_SFOS 17.5.7 MR-7# uptime
08:20:54 up 3 days, 20:56, load average: 3.30, 2.90, 2.79

CPU usage is around 10-20% on average.

Probably a late reply for your post. Best practice for NTP configuration is

1. The DC (PDC )  will be the main NTP server on the network. The NTP source for this server should be an external souce. (ntp.org1,2... )

2. If you have multiple DC's - there is a chance that FSMO roles will be transferred. So a group policy should exist so ntp source for the network clients are from only the DC's

3. If you have vmware in your enviroment - the time source should be the DC and the service should start and stop with the host.

4. IF you are on hyper-v environment - Disable hyper v time sync service inside the DC 

5. Ensure the polling internval setup on the DC is 900 ms or less and not default 3600 in a virtual environment

6. Change the ntp source on all your network devices to your primary ntp server

7. XG  would not block port 123 on the LAN. Also on XG configure the ntp source as external (ntp.org1,2... )

First, thanks to those who took the time to reply.  I ultimately ended up taking a little hiatus on this issue.  I've just been keeping up with the time sort of manually.  Decided to pick this up again.

I dug around a bit today and it seems that when the NTP request is sent out by a LAN client, for the most part its source port is 123.  I don't think that the XG firewall is correctly assigning a random port for the NAT, therefore when the FW receives the return packet, it doesn't know where to send the reply (perhaps I misunderstand how this is supposed to work, and if I do, please advise).

For those requests sent with port 123 I don't seem to get a reply on the client device.  Occasionally, some of those packets are sent with a random high port and those requests I do get a reply on the client.

Understand this is all observational, but I wonder if this has some merit?  I'm not sure how to validate this.  Guidance would be suggested.

These hits were when I ran w32tm /resync on my Domain controller, which errored out with:

C:\Users\Administrator>w32tm /resync
Sending resync command to local computer
The computer did not resync because no time data was available.

I then run:

C:\Users\Administrator>w32tm /stripchart /dataonly /computer:0.us.pool.ntp.org
Tracking 0.us.pool.ntp.org [23.239.24.67:123].
The current time is 9/24/2019 11:48:58 PM.
23:48:58, -62.3465844s
23:49:00, -62.3321461s
23:49:02, -62.3591517s
23:49:04, -62.3593514s

And this is the result in the log:

Notice the source ports are now "random" high ports and I received time data back.

I stumbled accross this when setting up a SG firewall in a new customers network.

I had the same issue with NTP server access because the PDC could not access any external NTP server. Firewall rules were in place.

After a while I found out that someone misconfigured the internal server with PDC FSMO role with a bad w32tm command line setting some time ago.

The problem is the AnnounceFlag 0x8 behind the IP address of the external NTP server. The 0x8 setting sets Windows Time to use client mode.

w32tm /config /manualpeerlist:NTP_server_IP_Address,0x8

Solution:

do not use 0x1 or 0,8 or whatever, just use the IP addresses, no flags.

Simply enter the correct command on the PDC to overwrite the current setting

e.g.

w32tm /config /manualpeerlist:"0.de.pool.ntp.org 1.de.pool.ntp.org 2.de.pool.ntp.org" /syncfromflags:manual /reliable:yes /update

restart Windows Time service.

Update: 02 June 2020

At an other customer I had a similar problem. NTP was configured correctly in Windows. But between Sophos SG and WAN was a HP OfficeConnect 1920s Layer2 Switch. The Switch has a Anti-DoS feature enabled by default called Automatic Denial-of-Service (DoS) protection

A Sub-feature is:
Prevent UDP Blat Attack

This drops packets that have a UDP source port equal to the UDP destination port. That is what matches to UDP Communication. So there the switch broke NTP. After disabling UDP Blat Attack NTP was working.