| Re: [chrony-users] Leveraging PTM |
[ Thread Index |
Date Index
| More chrony.tuxfamily.org/chrony-users Archives
]
- To: chrony-users@xxxxxxxxxxxxxxxxxxxx
- Subject: Re: [chrony-users] Leveraging PTM
- From: James Clark <jjc@xxxxxxxxxx>
- Date: Mon, 4 Sep 2023 08:39:49 +0700
- Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jclark-com.20230601.gappssmtp.com; s=20230601; t=1693791599; x=1694396399; darn=chrony.tuxfamily.org; h=content-transfer-encoding:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=vjjXWz+gqX2ApyjwkGat39rcqWLP+fRfaOQs9P6MmWk=; b=w7WjfCaWfF68cSwKk0lo11EwwrhwaXv51p5zAWGLJ5FJV3H+q4J1+il/yI94u2r9VY elfq8jLuAIXqaXzCpMSb700BURcaiS2l5hDT/7LH3cxc5I55MScn82zh47fXohabocCt Svg6zskcMC/ortpMccgS6QjN8WCDCKh1QTgyM922zssr7TL7yADFCLi3v/L5RBvK/qhj pXOVr4ljsZYx7xWtWRgJQZIPIgrkzI4jlwFkjxOuKeZGo4lbH0OUv6EfIv/r2IpnioWx YFjY3342cYmynDJ9p+m6M94jUxMfhpMlQRrXfUnrm5O7zo7Vy/VoF0E67rlNlN0ryBro E44Q==
On Tue, Aug 22, 2023 at 5:58 PM Miroslav Lichvar <mlichvar@xxxxxxxxxx> wrote:
> I think that would rely on PTM and PCIe between the CPU and the other
> NIC having no asymmetry (including the asymmetry of the NIC reading
> its HW clock), or at least them canceling out. I have not seen a
> proper verification of PTM accuracy yet.
....
> With two identical NICs in the system you can measure the PCIe and
> clock reading asymmetry between them by connecting them directly with
> a cable and measuring the PTP or NTP offset.
This is a nice idea. I set up my system with 2 x i225-T1s (with a 12th
gen CPU). But it should also work on the 4-port box I mentioned
earlier.
My i225s are on enp4s0 and enp5s0, and I connected them with a short
patch cable. The steps I went through are:
1. Use phc_ctl to get the PHCs roughly right:
phc_ctl enp4s0 set
phc_ctl enp5s0 set
2. Start two instances of ptp4l to sync and measure the PHCs:
ptp4l -q -m -2 -i enp4s0
ptp4l -q -m -2 -i enp5s0
This will sync them and show the offsets, which become small (<10ns)
after it settles down. (I'm using -2 because they don't have IP
addresses.)
3. Stop chrony (forgot this the first time)
4. Get the system clock synced to the PHCs
phc2sys -s enp4s0 -q -m -O 0
Let this settle and then interrupt
5. Finally do
phc2sys -s CLOCK_REALTIME -c enp4s0 -c enp5s0 -O 0 -q -m --free_running=1
This prints out pairs of offsets for enp4s0 and enp5s0
phc2sys[333052.959]: enp5s0 sys offset -1472 s0 freq +0 delay 0
phc2sys[333052.959]: enp4s0 sys offset -1480 s0 freq +0 delay 0
phc2sys[333053.960]: enp5s0 sys offset -1511 s0 freq +0 delay 0
phc2sys[333053.960]: enp4s0 sys offset -1515 s0 freq +0 delay 0
phc2sys[333054.960]: enp5s0 sys offset -1563 s0 freq +0 delay 0
phc2sys[333054.960]: enp4s0 sys offset -1561 s0 freq +0 delay 0
phc2sys[333055.961]: enp5s0 sys offset -1602 s0 freq +0 delay 0
phc2sys[333055.961]: enp4s0 sys offset -1611 s0 freq +0 delay 0
phc2sys[333056.961]: enp5s0 sys offset -1658 s0 freq +0 delay 0
phc2sys[333056.961]: enp4s0 sys offset -1665 s0 freq +0 delay 0
which are within about 10ns of each other.
Doing
sudo lspci -vv | grep -i granularity
shows a PTM granularity of 4ns. The clock rate of 2.5GBASE-T is
200MHz, which I think limits PTP accuracy to 5ns.
So if the difference of the phc2sys offsets are within 10ns, I believe
PTM is working as one would hope.
James
--
To unsubscribe email chrony-users-request@xxxxxxxxxxxxxxxxxxxx
with "unsubscribe" in the subject.
For help email chrony-users-request@xxxxxxxxxxxxxxxxxxxx
with "help" in the subject.
Trouble? Email listmaster@xxxxxxxxxxxxxxxxxxxx.