Ubuntu 16.04 time stops syncing timedatectl fails to query server, but timesyncd is active

Original Question:

See updates below.

I’m hoping I can get some assistance working through an issue I’ve been having on my Ubuntu server. I’m running the desktop version of 16.04 as a server and occasionally I have noticed that the system time gets out of sync/stops updating. Typically this precedes total loss of remote access and requires a restart to fix. Here is the output of systemctl status systemd-timesyncd.service:

    systemd-timesyncd.service - Network Time Synchronization
    Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled; 
    vendor preset: enabled
    Drop-In: /lib/systemd/system/systemd-timesyncd.service.d
       └─disable-with-time-daemon.conf
    Active: active (running) since Fri 2018-10-19 00:17:02 PDT; 2 days ago
    Docs: man:systemd-timesyncd.service(8)
    Main PID: 642 (systemd-timesyn)
    Status: "Synchronized to time server 91.189.91.157:123 
    (ntp.ubuntu.com)."
    CGroup: /system.slice/systemd-timesyncd.service
       └─642 /lib/systemd/systemd-timesyncd
    Oct 19 00:17:02 eric-SFF systemd[1]: Starting Network Time 
    Synchronization...
    Oct 19 00:17:02 eric-SFF systemd[1]: Started Network Time 
    Synchronization.
    Oct 19 00:37:24 eric-SFF systemd-timesyncd[642]: Synchronized to time 
    server 91.189.94.4:123 (ntp
    Oct 19 10:19:00 eric-SFF systemd-timesyncd[642]: Timed out waiting for 
    reply from 91.189.94.4:123
    Oct 19 10:19:00 eric-SFF systemd-timesyncd[642]: Synchronized to time 
    server 91.189.91.157:123 (n

And here is the output of timedatectl:

~$ timedatectl
Failed to query server: Connection timed out

The system time at this point was off by about 12 hours (reporting time of yesterday ~ 7pm when it is actually ~ 11am the following day). Time zone is correct.

I have also noticed that my cron jobs stop running when this happens, as I have a cron job that checks internet connection on regular intervals and reports its output to a log file. This log file stops updating around the same time that the time gets out of sync. I also have a cron job that backs up certain directories, and these backups are not up-to-date.

Please let me know if any additional information would be helpful. I’m still a relative novice to Ubuntu.

Thanks!!

Update:

@Damocles Thanks for your suggestion. Yes, the computer has a working network connection. I did not have any firewall rules set to allow NTP (port 123) through UFW or my firewall on my Ubiquiti router. However, it would still work for a while (sometimes weeks at a time) before bugging out. Could this still be a firewall issue?

I went ahead and added the ufw rule and added a firewall exception and NAT translation in my router and rebooted the server. It syncs for now, but my firewall did not seem to be in the way because the traffic count for that rule did not increase.

Is there anything else that could be blocking this connection? Perhaps something that would allow it to work at first, but cause a dropped/blocked NTP access down the line?

I also found this command to check time sync status, but it returns and unknown operation error:

:~$ timedatectl timesync-status
Unknown operation timesync-status

New Update 11/1/18

I installed chrony as suggested, but still getting the same issue, within 24 hours. Something is definitely wrong. And ~16 hours after a fresh boot and chrony installed, here is the output of some chrony commands:

:~$ chronyc sources
210 Number of sources = 11
MS Name/IP address         Stratum Poll Reach LastRx Last sample
===========================================================================
^? up2.com                       0   6   377   10y     +0ns[   +0ns] +/-    
0ns
^? 216-228-47-167.midrivers.     0   6   377   10y     +0ns[   +0ns] +/-    
0ns
^? blue.1e400.net                0   6   377   10y     +0ns[   +0ns] +/-    
0ns
^? 12.167.151.1                  0   6   377   10y     +0ns[   +0ns] +/-    
0ns
^? lithium.constant.com          0   6   377   10y     +0ns[   +0ns] +/-    
0ns
^? clock.xmission.com            0   6   377   10y     +0ns[   +0ns] +/-    
0ns
^? ha81.smatwebdesign.com        0   6   377   10y     +0ns[   +0ns] +/-    
0ns
^? srcf-ntp.stanford.edu         0   6   377   10y     +0ns[   +0ns] +/-    
0ns
^? time1.plumdev.net             0   6   377   10y     +0ns[   +0ns] +/-            
0ns
^? SunSITE.icm.edu.pl            0   6   377   10y     +0ns[   +0ns] +/-    
0ns
^? time.no-such-agency.net       0   6   377   10y     +0ns[   +0ns] +/-    
0ns
:~$ chronyc sourcestats
210 Number of sources = 11
Name/IP Address            NP  NR  Span  Frequency  Freq Skew  Offset  Std 
Dev
============================================================================
up2.com                     0   0     0     +0.000   2000.000     +0ns  
4000ms
216-228-47-167.midrivers.   0   0     0     +0.000   2000.000     +0ns  
4000ms
blue.1e400.net              0   0     0     +0.000   2000.000     +0ns  
4000ms
12.167.151.1                0   0     0     +0.000   2000.000     +0ns  
4000ms
lithium.constant.com        0   0     0     +0.000   2000.000     +0ns  
4000ms
clock.xmission.com          0   0     0     +0.000   2000.000     +0ns  
4000ms
ha81.smatwebdesign.com      0   0     0     +0.000   2000.000     +0ns  
4000ms
srcf-ntp.stanford.edu       0   0     0     +0.000   2000.000     +0ns  
4000ms
time1.plumdev.net           0   0     0     +0.000   2000.000     +0ns  
4000ms
SunSITE.icm.edu.pl          0   0     0     +0.000   2000.000     +0ns  
4000ms
time.no-such-agency.net     0   0     0     +0.000   2000.000     +0ns  
4000ms

I believe the “Reach” being nonzero means it’s getting past the firewall, so I don’t think that’s the issue. Also, these commands returned normal output at first.