| Re: [chrony-users] interpolation of offsets in chrony - a robust approach |
[ Thread Index |
Date Index
| More chrony.tuxfamily.org/chrony-users Archives
]
- To: chrony-users@xxxxxxxxxxxxxxxxxxxx
- Subject: Re: [chrony-users] interpolation of offsets in chrony - a robust approach
- From: Miroslav Lichvar <mlichvar@xxxxxxxxxx>
- Date: Thu, 18 Feb 2021 11:17:32 +0100
- Authentication-results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=mlichvar@xxxxxxxxxx
- Dkim-signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1613643458; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=zF9KNrMG3PD6lbk6Cwh/cHejy8gWJu882UhLV2dfRt8=; b=HteG38kfj2VhQsNZYhnzZG38YqtfhVuQf8qHBhxeiqctdDrfAlMyx6hYmyp7aPvnlZMYpA TFE8Gk7pEEV02XXo3W8gv0YppevuqhaUvZwcb1/MYA8eZz9NGDgv5NhXQ6s9ZzIC5VPtlN GmGokxJHD9XDELfmsPmmEq/1iDgZ8v4=
On Wed, Feb 17, 2021 at 12:02:12PM -0500, Charlie Laub wrote:
> A linear regression is not the best way to process this kind of data.
> Instead a robust analysis method is best. There is a simple and effective
> one for obtaining the "best fit" slope of a dataset called a Thiel-Sen
> estimator.
I think I've seen that one before. chrony uses a different robust
linear regression. Not sure what is the official name. It was
implemented from a book. See the regress.c file in the sources. It's
an iterative method and it is supposed to have a higher breakdown
point. But it is used only for RTC samples, manual input and reference
clocks, not for NTP samples.
For NTP, there is a weighted linear regression. The weights are based
on the measured NTP delay. An outlier, if it passes the delay filters,
should have a large delay, i.e. small weight. I tried using the robust
estimator for NTP before, but it didn't seem to work as well. YMMV.
Those weights help a lot. If the non-robust estimator was to be
replaced with a robust one, I think it would need to include the
delay. I spent some time on that, but failed.
If you have some ideas and can implement them, I'll be happy to test
it.
If you are considering to spend more time on chrony, there is a
related area where improvements might have a bigger impact and that is
the mechanism that controls the number of samples for the regression.
It is too sensitive to noise. I think it would help if chrony could
make long-term observations of the clock to better detect rapid
changes in frequency, i.e. when the number of samples needs to be
reduced. Maybe a neural network. They are used everywhere these days,
why not NTP? :)
--
Miroslav Lichvar
--
To unsubscribe email chrony-users-request@xxxxxxxxxxxxxxxxxxxx
with "unsubscribe" in the subject.
For help email chrony-users-request@xxxxxxxxxxxxxxxxxxxx
with "help" in the subject.
Trouble? Email listmaster@xxxxxxxxxxxxxxxxxxxx.