Re: [chrony-users] Leveraging PTM

[Miroslav Lichvar <mlichvar@xxxxxxxxxx>]

On Thu, Aug 17, 2023 at 12:26:46PM +0700, James Clark wrote:
> My experimental setup combines this with a Raspberry Pi CM4 working as
> PTP grandmaster. In my case, I have the CM4 connected to the
> mini-server via a switch, because I am also experimenting with PTP,

Do you have any recommendations for cheap switches with good PTP
support? That seems to be the most expensive part.

> Although the CM4 works well for syncing the NIC with a GPS receiver,
> it has some limitations compared to the mini-server: it is painfully
> slow to get the time from the PHC on the CM4; there's only one NIC; it
> can only timestamp PTP packets. This setup works around these
> limitations.

I think there is a CM4 IO board with a PCIe slot which should work
with the I210. That might work better than the CM4 NIC and it could be
just one well-performing device.

As for the PTP-specific timestamping of the CM4 NIC, I'm interested in
tests using the NTP-over-PTP transport with the latest chrony version
(you might need to set hwtstimeout to 0.1 or even longer for the CM4).

> This isn't quite as good as the results from example config, which
> shows an RMS offset of 4ns. I don't understand how it manages to be
> this accurate. The docs for the example config say that using the same
> NIC clock for PPS input as for timestamping NTP packets "cancels out
> any asymmetry between the system clock and hardware clock in the
> server’s timestamps of NTP packets". There's something clever going on
> here, which I would like to understand better. I am thinking I ought
> to add hwtimestamp to my config, but in my case the NIC clock for
> timestamping NTP packets will be different from the NIC clock used to
> set the system clock.

If both NICs support cross-timestamping, that should help the clients
to get more accurate timestamps.

> I am planning to write this up, but, before I do, I would like to get
> some feedback about whether this overall approach makes sense and, if
> so, how I can refine the configuration, in particular:
> 
> - Is hwtimestamp going to help? Is it just a matter of enabling
> hardware timestamping on interfaces other than the one connected to
> the CM4?

hwtimestamp will help the clients of the NTP server. It won't make a
difference for the server's own synchronization.

> - Is it better to use phc2sys -E ntpshm instead of the PHC refclock?

Yes, or even better the SOCK refclock supported in latest linuxptp.
The advantage over the PHC refclock is that it automatically updates
the TAI-UTC offset and stops the synchronization when PTP stops
synchronizing the PHC.

> - How do I figure out optimum polling/update rates? Is it just a
> matter of trial and error? Any hints?

Yes, it's mostly trial and error. The number of points reported by
sourcestats is an indicator. Ideally, it should stay close to 64, but
not get stuck too much to 64.

> - How do I get an accurate error of measurement from "chronyc
> sources"? I think this depends on  the precision in the refclock, but
> I'm not sure how I make that  accurate.

With cross-timestamping the delay (which determines the maximum error)
is unknown. The kernel would need to provide the timestamps from the
PTM exchange and IIRC that would give us the total delay only if there
was no PCIe switch between the CPU and NIC. I suspect you would need
to hook it up to a scope with PCIe analyzer to determine an accurate
value for the refclock delay option.

It shouldn't matter much. I'd suggest to compare graphs of reported
offsets with and without the nocrossts option to verify it improves
the stability. Verifying accuracy is much more difficult.

> - Would it be better to use the PTP layer 2 ethernet transport for this?

I wouldn't expect it to make a difference, but if you try it, please
let us know the results. I think I saw only one NIC which performed
better with L2 transport and that was a Mellanox one, not Intel or
Broadcom.

-- 
Miroslav Lichvar


-- 
To unsubscribe email chrony-users-request@xxxxxxxxxxxxxxxxxxxx 
with "unsubscribe" in the subject.
For help email chrony-users-request@xxxxxxxxxxxxxxxxxxxx 
with "help" in the subject.
Trouble?  Email listmaster@xxxxxxxxxxxxxxxxxxxx.