Re: [chrony-users] Synchronizing clock with GPS with PPS

[Ryan Govostes <rgovostes@xxxxxxxx>]

I can confirm that /dev/ttyAMA1 streams incoming NMEA messages. I can confirm /dev/pps0 is a working PPS device.

I can confirm that gpsd is configured to access /dev/ttyAMA1 and when I launch it with

	sudo gpsd -n -N -D3 -F /tmp/gpsd.sock /dev/ttyAMA1

I see it getting a satellite fix, suggesting it is receiving NMEA messages just fine, although it has the /dev/pps1 error messages as below.

If I use `strace` on gpsd, then I see that it connects to /var/run/chrony.ttyAMA1.sock (/var/run being a symlink to /run it should be OK):

	connect(8, {sa_family=AF_UNIX, sun_path="/var/run/chrony.ttyAMA1.sock"}, 30) = 0

I never see it reading or writing file descriptor 8, so it clearly is not sending anything to chronyd.


It appears that /dev/pps1 is created some time after gpsd starts. /sys/devices/virtual/pps/pps1 appears and the path file contains the text /dev/ttyAMA1. gpsd scans through all of these pps/*/path files to find the one matching the NMEA device it is configured for, so that’s why it ends on /dev/pps1.

I still don’t know who exactly is creating /sys/devices/virtual/pps/pps1 — strace doesn’t put the blame on either chronyd or gpsd, but otherwise this is a vanilla system I installed just a few days ago.

Ryan 



> On Sep 15, 2020, at 4:05 PM, Avamander <avamander@xxxxxxxxx> wrote:
> 
> Check what you have specified in the list of devices to be used. 
> 
> I personally couldn't convince gpsd to use pps1 instead of pps0, but you might be having the opposite issue :S 
> 
> What actually is pps1 on your system? It might play a role.
> 
> On Tue, Sep 15, 2020 at 11:02 PM Ryan Govostes <rgovostes@xxxxxxxx> wrote:
> I’m not sure. Its logs look OK, but it prints out:
> 
>         gpsd:INFO: KPPS:/dev/ttyAMA1 RFC2783 path:/dev/pps1, fd is 9
> 
> Note that /dev/pps1 is _not_ the PPS device I expect to use. This device only appears while gpsd is running so it appears to be created by it? ppstest only reports timeouts for this one, while pps0 continues to work as expected.
> 
> More context:
> 
>         gpsd:INFO: KPPS:/dev/ttyAMA1 RFC2783 path:/dev/pps1, fd is 9
>         gpsd:INFO: KPPS:/dev/ttyAMA1 pps_caps 0x1133
>         gpsd:INFO: KPPS:/dev/ttyAMA1 have PPS_CANWAIT
>         gpsd:INFO: KPPS:/dev/ttyAMA1 kernel PPS will be used 
>         …
>         gpsd:INFO: KPPS:/dev/ttyAMA1 kernel PPS timeout Interrupted system call
>         …
>         gpsd:INFO: KPPS:/dev/ttyAMA1 kernel PPS timeout Connection timed out
> 
> Ryan
> 
> > On Sep 15, 2020, at 3:57 PM, Avamander <avamander@xxxxxxxxx> wrote:
> > 
> > > However, `chronyc` does not report any updates being received from this source.
> > 
> > If you aren't seeing anything on SHM1 either, then gpsd still has issues with reading the PPS source. Check its logs.
> > 
> > On Tue, Sep 15, 2020 at 10:56 PM Ryan Govostes <rgovostes@xxxxxxxx> wrote:
> > Ah OK, I guess the part that was not clear to me was that chronyd _creates_ this socket when configured with refclock SOCK, rather than simply connecting to it.
> > 
> > When I configured this and restarted chronyd and then gpsd, it did create the socket file. However, `chronyc` does not report any updates being received from this source.
> > 
> > I did not find an AppArmor profile that is currently being enforced on gpsd.
> > 
> > I did notice that AppArmor was preventing chronyd from creating /run/chronyd.pps0.sock, so I corrected that and switched the configuration to:
> > 
> >         refclock SOCK /run/chrony.ttyAMA1.sock refid GPS precision 1e-1
> >         refclock SOCK /run/chrony.pps0.sock refid PPS precision 1e-7
> > 
> > However I still do not receive any updates.
> > 
> > Ryan
> > 
> > > On Sep 15, 2020, at 3:20 PM, Avamander <avamander@xxxxxxxxx> wrote:
> > > 
> > > > but also I don’t see that socket you’re referencing being created.
> > > > I don’t see any AppArmor logs that seem to indicate anyone was prevented from creating this file.
> > > 
> > > Have you actually told chrony to create it?
> > > 
> > > > for a while but the PPS never updated:
> > > 
> > > Yes, this is exactly why I suggested you check the AppArmor policy itself. When the pps device is inaccessible gpsd gives some vague error about the pps device and that's it. Open gpsd's log and verify it hasn't thrown an error about the pps device during startup.
> > > 
> > > > I couldn’t readily find much documentation on SHM 0 vs SHM 1.
> > > 
> > > Yes, this is awfully documented online.
> > > 
> > > 
> > > 
> > > On Tue, Sep 15, 2020 at 10:08 PM Ryan Govostes <rgovostes@xxxxxxxx> wrote:
> > > Below Bill argued against using /var/run/chrony.ttyAMA0.sock, but also I don’t see that socket you’re referencing being created. I don’t see any AppArmor logs that seem to indicate anyone was prevented from creating this file. Perhaps the version is too old?
> > > 
> > > I couldn’t readily find much documentation on SHM 0 vs SHM 1. I did find this, which suggests using SHM 1 instead of having chrony go directly to the PPS device, as in:
> > > 
> > >         refclock SHM 0 refid GPS precision 1e-1
> > >         refclock SHM 1 refid PPS precision 1e-7
> > > 
> > > https://gpsd.gitlab.io/gpsd/gpsd-time-service-howto.html#_feeding_chrony_from_gpsd
> > > 
> > > I restarted chronyd with this configuration and watched `chronyc sources` for a while but the PPS never updated:
> > > 
> > >         210 Number of sources = 2
> > >         MS Name/IP address         Stratum Poll Reach LastRx Last sample
> > >         ===============================================================================
> > >         #* GPS                           0   4   377    22  +1104us[+4464us] +/-  100ms
> > >         #? PPS                           0   4     0     -     +0ns[   +0ns] +/-    0ns
> > > 
> > > I disabled systemd-timesyncd as you suggested, thanks.
> > > 
> > > Ryan
> > > 
> > > > On Sep 15, 2020, at 2:42 PM, Avamander <avamander@xxxxxxxxx> wrote:
> > > > 
> > > > First, disable systemd-timesyncd if you're using chrony: sudo systemctl disable --now systemd-timesyncd
> > > > 
> > > > Second, enable chrony, pretty sure it isn't enabled by default after install: sudo systemctl enable chrony
> > > >  
> > > > Why do you want to SHM 0 (non-PPS-corrected) NMEA time, instead of SHM 1 or /var/run/chrony.ttyAMA0.sock?
> > > > 
> > > > On Tue, Sep 15, 2020 at 9:36 PM Ryan Govostes <rgovostes@xxxxxxxx> wrote:
> > > > Thanks, I removed the offset and delay so the reference clock configuration is now:
> > > > 
> > > >         refclock SHM 0 refid GPS precision 1e-1
> > > >         refclock PPS /dev/pps0 refid PPS
> > > > 
> > > > My intention is to have GPS set the system date and time and then have the PPS signal keep it from drifting.
> > > > 
> > > > After applying this, I ran again and am now getting:
> > > > 
> > > >         MS Name/IP address         Stratum Poll Reach LastRx Last sample
> > > >         ===============================================================================
> > > >         #x GPS                           0   4   377    16   +587us[ +587us] +/-  100ms
> > > >         #x PPS                           0   4   160    82   -128ms[ -128ms] +/-  759ns
> > > > 
> > > > The #x suggests that “time may be in error.” However I am still seeing gpsd work (monitored via cgps) and the PPS device still appears to be working (according to ppstest).
> > > > 
> > > > Furthermore timedatectl suggests that the system clock is not synchronized:
> > > > 
> > > >         $ timedatectl status
> > > >         Local time: Tue 2020-09-15 18:34:48 UTC
> > > >         Universal time: Tue 2020-09-15 18:34:48 UTC
> > > >         RTC time: n/a
> > > >         Time zone: Etc/UTC (UTC, +0000)
> > > >         System clock synchronized: no
> > > >         systemd-timesyncd.service active: yes
> > > >         RTC in local TZ: no
> > > > 
> > > > What appears to be the problem?
> > > > 
> > > > Ryan
> > > > 
> > > > > On Sep 15, 2020, at 12:47 PM, Bill Unruh <unruh@xxxxxxxxxxxxxx> wrote:
> > > > > 
> > > > > 
> > > > > 
> > > > > William G. Unruh __| Canadian Institute for|____ Tel: +1(604)822-3273
> > > > > Physics&Astronomy _|___ Advanced Research _|____ Fax: +1(604)822-5324
> > > > > UBC, Vancouver,BC _|_ Program in Cosmology |____ unruh@xxxxxxxxxxxxxx
> > > > > Canada V6T 1Z1 ____|____ and Gravity ______|_ https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.theory.physics.ubc.ca%2F&amp;data=02%7C01%7Crgovostes%40whoi.edu%7C054736ed4c39434c82db08d8599714d3%7Cd44c5cc6d18c46cc8abd4fdf5b6e5944%7C0%7C0%7C637357852718387997&amp;sdata=qHAUE4iPB4YLvLVCtFetR33HnDjEowrOa%2FpWLANdNpE%3D&amp;reserved=0
> > > > > 
> > > > > On Tue, 15 Sep 2020, Ryan Govostes wrote:
> > > > > 
> > > > >> Hi all,
> > > > >> 
> > > > >> I am setting up chronyd on an embedded Linux device to synchronize the system clock using a GPS module. The GPS device sends NMEA strings over the character device /dev/ttyAMA1 and I have also configured /dev/pps0, both of which appear to be working OK.
> > > > >> 
> > > > >> The system is running Ubuntu 18.04 and the latest package versions are chronyd 3.2 and gpsd 3.17.
> > > > >> 
> > > > >> I configured gpsd to listen to the serial device and then added these lines to my chrony.conf:
> > > > >> 
> > > > >>      refclock SHM 0 refid GPS precision 1e-1 offset 0.9999 delay 0.2
> > > > > 
> > > > > Why those figures for ooffset? That is 1 sec offset. NMEA is not that bad.
> > > > > 
> > > > > 
> > > > >>      refclock PPS /dev/pps0 refid PPS
> > > > >> 
> > > > >> When I run `chronyc sources` they both seem to be kind of working:
> > > > >> 
> > > > >>      210 Number of sources = 2
> > > > >>      MS Name/IP address         Stratum Poll Reach LastRx Last sample
> > > > >>      ===============================================================================
> > > > >>      #- GPS                           0   4   377    12   +128ms[ +128ms] +/-  200ms
> > > > >>      #* PPS                           0   4   377    12     +6ns[ +119ns] +/-  203ns
> > > > >> 
> > > > >> However it looks like the GPS source is “not combined”. Is this a degraded state, e.g., it is using one of these two sources?
> > > > > 
> > > > > Why would one want to combine GPS with PPS. PPS is good to the nanosecond
> > > > > level. GPS toabut 100 ms -- that is almost a  million times worse. It would be like combining your wristwatch with some clock which says "its
> > > > > spring so it must be April".
> > > > > 
> > > > >> 
> > > > >> Also, I am not sure why the LastRx from the PPS (or frankly either) ticks upwards so long—shouldn’t it constantly be receiving updates?
> > > > > 
> > > > > Yout tell it that POLL is 4 which means 16 seconds. So every 16 seconds that
> > > > > clock is read. The driver massages the input ( once a second) to get rid of
> > > > > obvious outliers but reports to chrony once every 16 seconds. Note it is a bad
> > > > > idea to reduce the poll even further. Then obvious ouliers are not thrown out,
> > > > > and the ability to determine the rate of the clock is degraded.
> > > > > 
> > > > >> 
> > > > >> I am just using the precision / offset / delay figures that several examples use. Is there documentation on calibrating these values?
> > > > > 
> > > > > Get rid of the offset and delay. The GPS is useless except for setting actual
> > > > > number of the seconds.
> > > > > 
> > > > >> 
> > > > >> Finally, I read that using Unix sockets rather that shared memory is preferable, but my chronyd does not seem to create those sockets.
> > > > > 
> > > > > Why is it better? Leave things as they are. With PPS your time will be
> > > > > accurate to microseconds just as things are now. Do you need any better time?
> > > > > If you do need time to nanoseconds, then you will really have to throw away
> > > > > your computer (its ability to read interrupts is only at the microsecond
> > > > > level) and begin compensating for propagation delays in your gps unit, and
> > > > > also the sawtooth offset on the ns level due to your gps receiver innards. But
> > > > > then, why would you want to know the time to 1 billionth of a second?
> > > > 
> > > 
> > 
>