I am confused as to why my Debian Pi below chooses ntp4.ntp.net.nz over my local Stratum 1 servers which both show better delay, offset and jitter results.

Keen to learn your thoughts.
=============================
pi@raspberrypi:~ $ ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
-ns2.snap.net.nz 201.226.228.222 4 u 1287 1024 334 6.616 5.159 1.149
-163.47.220.60 ( 202.46.177.18 2 u 938 1024 377 18.330 4.893 0.407
-192.168.178.248 .GPS. 1 u 714 1024 377 0.734 1.339 0.126
+192.168.178.232 .GPS. 1 u 27 1024 377 0.821 1.218 0.144
192.168.178.1 192.168.178.123 3 u 316 1024 377 0.542 0.215 0.792
-ntp1.ntp.net.nz .GPS. 1 u 867 1024 377 70.523 5.686 0.936
-ntp2.ntp.net.nz .GPS. 1 u 67 1024 377 22.032 5.404 0.648
+ntp3.ntp.net.nz .GPS. 1 u 973 1024 377 15.764 2.315 1.673
*ntp4.ntp.net.nz .GPS. 1 u 879 1024 377 11.574 2.434 0.726
pi@raspberrypi:~ $

On 04/06/22 05:53, Derek B wrote:
> I am confused as to why my Debian Pi below chooses ntp4.ntp.net.nz over my local Stratum 1 servers which both show better delay, offset and jitter results.
>
> Keen to learn your thoughts.
> =============================
> pi@raspberrypi:~ $ ntpq -p
> remote refid st t when poll reach delay offset jitter
> ==============================================================================
> -ns2.snap.net.nz 201.226.228.222 4 u 1287 1024 334 6.616 5.159 1.149
> -163.47.220.60 ( 202.46.177.18 2 u 938 1024 377 18.330 4.893 0.407
> -192.168.178.248 .GPS. 1 u 714 1024 377 0.734 1.339 0.126
> +192.168.178.232 .GPS. 1 u 27 1024 377 0.821 1.218 0.144
> 192.168.178.1 192.168.178.123 3 u 316 1024 377 0.542 0.215 0.792
> -ntp1.ntp.net.nz .GPS. 1 u 867 1024 377 70.523 5.686 0.936
> -ntp2.ntp.net.nz .GPS. 1 u 67 1024 377 22.032 5.404 0.648
> +ntp3.ntp.net.nz .GPS. 1 u 973 1024 377 15.764 2.315 1.673
> *ntp4.ntp.net.nz .GPS. 1 u 879 1024 377 11.574 2.434 0.726
> pi@raspberrypi:~ $

The best way to ensure your local server takes priority is to setup to
use a pps signal. Here's the reuslt from the local setup here:

root@ntp-host:~ # ntpq -pn
remote refid st t when poll reach delay offset jitter
===================================================================
o127.127.22.0 .PPS. 0 l 2 8 377 0.000 0.001 0.001
*192.9.200.168 .GPS. 1 u 58 64 377 0.174 -0.004 0.005
+192.9.200.169 .GPS. 1 u 3 64 377 0.343 -0.007 0.029
-158.43.128.33 .GPS. 1 u 29 64 317 14.445 0.062 0.447

The first line is the pps signal, second two are local time server
hosts and the 4th is s single external host used as a sanity check.
Not sure how PPS i/p is done on an rpi, but the gps module should have
a pps logic output somewhere...

Chris

On 06/04/2022 12:01, chris wrote:
> Not sure how PPS i/p is done on an rpi, but the gps module should have
> a pps logic output somewhere...
>
> Chris

I have some (perhaps somewhat dated) notes here:

https://www.satsignal.eu/ntp/Raspberry-Pi-quickstart.html

Basically GPIO 18 can be connected to a 3.3V PPS line, and the kernel will
accept it as triggering interrupts. NTP and GPSd can handle that.

Cheers,
David
--
Cheers,
David
Web: http://www.satsignal.eu

Thank you but no; I wish to understand the internal decisions made by the algorithm that determines the preferred reference.
The fact that I mentioned my own GPS is irrelevant, surely the better the delay, offset, jitter, should win out?

Apparently not.

On 2022-04-07, Derek B <decibel47@gmail.com> wrote:

> Thank you but no;
"Thank you but no" what? You do not want your clock to driven by PPS?
You do not want anyone telling you anything? It really helps to keep
enough context to let others know what you are talking about.

>I wish to understand the internal decisions made by the algorithm that determines the preferred reference.

But your original post did not even tell us which source was which so it
is really hard to know what you are talking about.

> The fact that I mentioned my own GPS is irrelevant, surely the better the delay, offset, jitter, should win out?

Since your evidence is supposed to be your own system so it is surely
not irrelevant.

Another item is history. chrony does not go hopping around frm source to
source just because the stats become a bit better on one rather than the
other at this time.

And offset is a very bad thing to use. For example, if the system uses
itself to discipline itself, the offset will always be 0, even if the
clock is out by a year and the standard deviation will be small.

Another is voting-- even if one clock has better stats, if its time
disagrees with all of the other clocks (given its stats) then it will be
ignored.

You have the logs (eg the statistics log) so you can see what is happening over time.
>
> Apparently not.

You do have the source code. You can see what the selection algorithm is.

On 2022-04-07, William Unruh <unruh@invalid.ca> wrote:
>
> On 2022-04-07, Derek B <decibel47@gmail.com> wrote:
>
>> Thank you but no;
> "Thank you but no" what? You do not want your clock to driven by PPS?
> You do not want anyone telling you anything? It really helps to keep
> enough context to let others know what you are talking about.
>
>>I wish to understand the internal decisions made by the algorithm that determines the preferred reference.
>
> But your original post did not even tell us which source was which so it
> is really hard to know what you are talking about.
>
>
>> The fact that I mentioned my own GPS is irrelevant, surely the better the delay, offset, jitter, should win out?
>
> Since your evidence is supposed to be your own system so it is surely
> not irrelevant.
>
> Another item is history. chrony does not go hopping around frm source to
> source just because the stats become a bit better on one rather than the
> other at this time.
>
> And offset is a very bad thing to use. For example, if the system uses
> itself to discipline itself, the offset will always be 0, even if the
> clock is out by a year and the standard deviation will be small.
>
> Another is voting-- even if one clock has better stats, if its time
> disagrees with all of the other clocks (given its stats) then it will be
> ignored.
>
> You have the logs (eg the statistics log) so you can see what is happening over time.
>>
>> Apparently not.
>
> You do have the source code. You can see what the selection algorithm is.
PS: it is sources.c that contains the selection.

Derek B <decibel47@gmail.com> wrote:
> Thank you but no; I wish to understand the internal decisions made by the algorithm that determines the preferred reference.
> The fact that I mentioned my own GPS is irrelevant, surely the better the delay, offset, jitter, should win out?
>
> Apparently not.

The simplified answer is that the decision is based on long term and
complex statistics and not single snapshots in time.

If you were to plot all the logged data it may, or may not, become
obvious why a particular server was picked.

A complete explaination of how the algorithms work is far too complex
for a USENET post.

Look at:

https://www.eecis.udel.edu/~mills/ntp/html/warp.html

Likely the most relevant topics to your question are Clock Filter
Algorithm, Clock Select Algorithm and Clock Cluster Algorithm.

On 04/07/22 05:26, Derek B wrote:
> Thank you but no; I wish to understand the internal decisions made by the algorithm that determines the preferred reference.
> The fact that I mentioned my own GPS is irrelevant, surely the better the delay, offset, jitter, should win out?
>
> Apparently not.

If you have just a few sources, that may hold true, but with a whole
list of sources, it may take a long time to work that out via it's
internal algorithms. I would try again with 3 or 4 remote servers
and see if that converges in less time. Also, istr, there is a prefer
keyword that can be used in the ntp.conf file, but never used it here.
The convergence algorithm may also get confused with a long list of
servers, especially if the list of members changes over a short
period. ntp can take hours or even days to optimise everything.

Having said that, if i'm running pps, the list soon converges onto
the best server, so perhaps that is relevant...

Chris

William Unruh <unruh@invalid.ca> wrote:
>

<snip>

> You do have the source code. You can see what the selection algorithm is.

Highly unlikely as the vast majority of people would be installing using
a pre-compiled package that does NOT include source code.

Yes, one can download the source, but why do that and attempt to reverse
engineer the source code when documentation exists on www.ntp.org?

chris <chris-nospam@tridac.net> wrote:
> On 04/07/22 05:26, Derek B wrote:
>> Thank you but no; I wish to understand the internal decisions made by the algorithm that determines the preferred reference.
>> The fact that I mentioned my own GPS is irrelevant, surely the better the delay, offset, jitter, should win out?
>>
>> Apparently not.
>
> If you have just a few sources, that may hold true, but with a whole
> list of sources, it may take a long time to work that out via it's
> internal algorithms. I would try again with 3 or 4 remote servers
> and see if that converges in less time. Also, istr, there is a prefer
> keyword that can be used in the ntp.conf file, but never used it here.
> The convergence algorithm may also get confused with a long list of
> servers, especially if the list of members changes over a short
> period. ntp can take hours or even days to optimise everything.
>
> Having said that, if i'm running pps, the list soon converges onto
> the best server, so perhaps that is relevant...
>
> Chris
If you have pps, then do use the prefer and iburst keywords with this
server(s) on your client machines.

If you have pps, there is no need for more than 3 or 4 total servers
in any machine's configuration.

FYI I run two pps servers, one is a Pi with a GNSS receiver and the
other is a Ubuntu machine with an oven controlled precision crystal
oscillator reference GNSS receiver.

The pps servers configurations includes 3 network servers which serve no
purpose other than to keep the selection algorithms sane. Do not have
the pps servers reference each other.

All the client machines use the two pps servers and two network servers,
which also server no purpose other than to keep the selection algorithms
sane. The pps servers have the prefer and iburst keywords.

Because of the way the selection algorithms work, you need at least 3
servers in any given client's configuration, so I have 4 just in case
one of the remote servers is dead on a restart.

I log all statistics and do graphing of all the machines on a web server
so I have a good idea what everything is doing.

If I reboot a client, it typically takes a minute or so to select one
of the pps machines, about a half hour to get to the 10's of millisecond
accuracy, and a couple of hours to get to the 10's of microsecond
accuracy.

Sorry I did not read your post carefully enough and thought you were
using chrony, not ntpd. Thus my comment below is irrelevant to you. The
idea still applies however-- the source code for ntpd will answer your
question, but I cannot point you to the relevant code.

On 2022-04-07, William Unruh <unruh@invalid.ca> wrote:
> On 2022-04-07, William Unruh <unruh@invalid.ca> wrote:
>>
>> On 2022-04-07, Derek B <decibel47@gmail.com> wrote:
>>
>>> Thank you but no;
>> "Thank you but no" what? You do not want your clock to driven by PPS?
>> You do not want anyone telling you anything? It really helps to keep
>> enough context to let others know what you are talking about.
>>
>>>I wish to understand the internal decisions made by the algorithm that determines the preferred reference.
>>
>> But your original post did not even tell us which source was which so it
>> is really hard to know what you are talking about.
>>
>>
>>> The fact that I mentioned my own GPS is irrelevant, surely the better the delay, offset, jitter, should win out?
>>
>> Since your evidence is supposed to be your own system so it is surely
>> not irrelevant.
>>
>> Another item is history. chrony does not go hopping around frm source to
>> source just because the stats become a bit better on one rather than the
>> other at this time.
>>
>> And offset is a very bad thing to use. For example, if the system uses
>> itself to discipline itself, the offset will always be 0, even if the
>> clock is out by a year and the standard deviation will be small.
>>
>> Another is voting-- even if one clock has better stats, if its time
>> disagrees with all of the other clocks (given its stats) then it will be
>> ignored.
>>
>> You have the logs (eg the statistics log) so you can see what is happening over time.
>>>
>>> Apparently not.
>>
>> You do have the source code. You can see what the selection algorithm is.
> PS: it is sources.c that contains the selection.
>
>

On 06/04/2022 05:53, Derek B wrote:
> I am confused as to why my Debian Pi below chooses ntp4.ntp.net.nz over my local Stratum 1 servers which both show better delay, offset and jitter results.
>
> Keen to learn your thoughts.
> =============================
> pi@raspberrypi:~ $ ntpq -p
> remote refid st t when poll reach delay offset jitter
> ==============================================================================
> -ns2.snap.net.nz 201.226.228.222 4 u 1287 1024 334 6.616 5.159 1.149
> -163.47.220.60 ( 202.46.177.18 2 u 938 1024 377 18.330 4.893 0.407
> -192.168.178.248 .GPS. 1 u 714 1024 377 0.734 1.339 0.126
> +192.168.178.232 .GPS. 1 u 27 1024 377 0.821 1.218 0.144
> 192.168.178.1 192.168.178.123 3 u 316 1024 377 0.542 0.215 0.792
> -ntp1.ntp.net.nz .GPS. 1 u 867 1024 377 70.523 5.686 0.936
> -ntp2.ntp.net.nz .GPS. 1 u 67 1024 377 22.032 5.404 0.648
> +ntp3.ntp.net.nz .GPS. 1 u 973 1024 377 15.764 2.315 1.673
> *ntp4.ntp.net.nz .GPS. 1 u 879 1024 377 11.574 2.434 0.726
> pi@raspberrypi:~ $
>

ntpq -p doesn't display everything about the peers. In particular, it
doesn't display the upstream statistics, such as root delay and root
dispersion, which are used to select the system peer; it only reports
the last hop figures.

offset is never used as a quality figure, except if it is far enough out
to declare a falseticker (using it would introduce confirmation bias).

Also, the system peer is not the only source of time; it is basically
used as the source for the downstream values of the missing parameters.
However the same root distance, etc., information is used to weight
the, in your case, three survivors.

On 04/07/22 19:12, Jim Pennino wrote:
> chris<chris-nospam@tridac.net> wrote:
>> On 04/07/22 05:26, Derek B wrote:
>>> Thank you but no; I wish to understand the internal decisions made by the algorithm that determines the preferred reference.
>>> The fact that I mentioned my own GPS is irrelevant, surely the better the delay, offset, jitter, should win out?
>>>
>>> Apparently not.
>>
>> If you have just a few sources, that may hold true, but with a whole
>> list of sources, it may take a long time to work that out via it's
>> internal algorithms. I would try again with 3 or 4 remote servers
>> and see if that converges in less time. Also, istr, there is a prefer
>> keyword that can be used in the ntp.conf file, but never used it here.
>> The convergence algorithm may also get confused with a long list of
>> servers, especially if the list of members changes over a short
>> period. ntp can take hours or even days to optimise everything.
>>
>> Having said that, if i'm running pps, the list soon converges onto
>> the best server, so perhaps that is relevant...
>>
>> Chris
>
> If you have pps, then do use the prefer and iburst keywords with this
> server(s) on your client machines.
>
> If you have pps, there is no need for more than 3 or 4 total servers
> in any machine's configuration.
>
> FYI I run two pps servers, one is a Pi with a GNSS receiver and the
> other is a Ubuntu machine with an oven controlled precision crystal
> oscillator reference GNSS receiver.
>
> The pps servers configurations includes 3 network servers which serve no
> purpose other than to keep the selection algorithms sane. Do not have
> the pps servers reference each other.
>
> All the client machines use the two pps servers and two network servers,
> which also server no purpose other than to keep the selection algorithms
> sane. The pps servers have the prefer and iburst keywords.
>
> Because of the way the selection algorithms work, you need at least 3
> servers in any given client's configuration, so I have 4 just in case
> one of the remote servers is dead on a restart.
>
> I log all statistics and do graphing of all the machines on a web server
> so I have a good idea what everything is doing.
>
> If I reboot a client, it typically takes a minute or so to select one
> of the pps machines, about a half hour to get to the 10's of millisecond
> accuracy, and a couple of hours to get to the 10's of microsecond
> accuracy.
>

Here, i'm using a mini itx box, FreeBSD 12, two network interfaces.
The private subnet talks to a couple of refurbished commercial time
servers in 1u boxes. PPS comes either from one of the two time servers,
or from the lab 10MHz GPS frequency standard, which also has a PPS
output. The PPS goes to the serial port on the mini itx host, which
also has a firewall rule set on the internet facing side. The whole
thing is on a ups, as are the isp routers. System currently showing
an uptime of around 10 months. May be overkill, but it's an
experimental system and was fun to build and setup...

Chris

OK, so which command gives a better indication of how peers are evaluated?
One of these?
ntpq> help
ntpq commands:
:config drefid mreadlist readvar
addvars exit mreadvar reslist
apeers help mrl rl
associations host mrulist rmvars
authenticate hostnames mrv rv
authinfo ifstats ntpversion saveconfig
cl iostats opeers showvars
clearvars kerninfo passociations sysinfo
clocklist keyid passwd sysstats
clockvar keytype peers timeout
config-from-file lassociations poll timerstats
cooked lopeers pstats version
cv lpassociations quit writelist
debug lpeers raw writevar
delay monstats readlist

On 10/04/2022 11:42, Derek B wrote:
> OK, so which command gives a better indication of how peers are evaluated?

I assume you posted this to the wrong sub-thread. The standard Debian
install doesn't include the reference implementation of NTP, and I'm not
sure I can package install it without changing the daemon used, so I
can't dry run the commands, but I would expect mreadlist or multiple
invocations of readlist, preceded by associations (to get the
association IDs), would give most of the information used.

> One of these?
> ntpq> help
> ntpq commands:
> :config drefid mreadlist readvar
> addvars exit mreadvar reslist
> apeers help mrl rl
> associations host mrulist rmvars
> authenticate hostnames mrv rv
> authinfo ifstats ntpversion saveconfig
> cl iostats opeers showvars
> clearvars kerninfo passociations sysinfo
> clocklist keyid passwd sysstats
> clockvar keytype peers timeout
> config-from-file lassociations poll timerstats
> cooked lopeers pstats version
> cv lpassociations quit writelist
> debug lpeers raw writevar
> delay monstats readlist
>

On 2022-04-10, David Woolley <david@ex.djwhome.demon.invalid> wrote:
> On 10/04/2022 11:42, Derek B wrote:
>> OK, so which command gives a better indication of how peers are evaluated?
>
> I assume you posted this to the wrong sub-thread. The standard Debian
> install doesn't include the reference implementation of NTP, and I'm not

ntpd is the reference implementation of NTP.

> sure I can package install it without changing the daemon used, so I
> can't dry run the commands, but I would expect mreadlist or multiple
> invocations of readlist, preceded by associations (to get the
> association IDs), would give most of the information used.
>
>
>> One of these?
>> ntpq> help
>> ntpq commands:
>> :config drefid mreadlist readvar
>> addvars exit mreadvar reslist
>> apeers help mrl rl
>> associations host mrulist rmvars
>> authenticate hostnames mrv rv
>> authinfo ifstats ntpversion saveconfig
>> cl iostats opeers showvars
>> clearvars kerninfo passociations sysinfo
>> clocklist keyid passwd sysstats
>> clockvar keytype peers timeout
>> config-from-file lassociations poll timerstats
>> cooked lopeers pstats version
>> cv lpassociations quit writelist
>> debug lpeers raw writevar
>> delay monstats readlist
>>
>

On 10/04/2022 16:27, William Unruh wrote:
> ntpd is the reference implementation of NTP.

Standard Debian comes with systemd-timesyncd, and SNTP client, so it
does not include the reference implementation.

The nearest they offer to the reference implementation is secntpd. I
get the impression that this is a fork, but I haven't looked in detail.
That is the package that includes ntpq. I can't therefore easily
install ntpq without risking replacing the current time synch daemon,
which is good enough for my current purposes, so I can't easily dry run
ntpq, with my current system, even though I've used it on work systems,
in the past.

On 10/04/2022 18:23, David Woolley wrote:
> On 10/04/2022 16:27, William Unruh wrote:
>> ntpd is the reference implementation of NTP.
> Standard Debian comes with systemd-timesyncd, and SNTP client, so it
> does not include the reference implementation.
>
> The nearest they offer to the reference implementation is secntpd. I
> get the impression that this is a fork, but I haven't looked in detail.
> That is the package that includes ntpq. I can't therefore easily
> install ntpq without risking replacing the current time synch daemon,
> which is good enough for my current purposes, so I can't easily dry run
> ntpq, with my current system, even though I've used it on work systems,
> in the past.

Same on the Raspberry Pi - no reference NTP. I just:

sudo apt-get install ntp

and all works as expected.
--
Cheers,
David
Web: http://www.satsignal.eu

On 2022-04-10, David Woolley <david@ex.djwhome.demon.invalid> wrote:
> On 10/04/2022 16:27, William Unruh wrote:
>> ntpd is the reference implementation of NTP.
>
> Standard Debian comes with systemd-timesyncd, and SNTP client, so it
> does not include the reference implementation.

Thanks. Did not realise that. However I think what the OP meant by
reference is the clock which ntp program marks with a *, the time standard it
selects as the clock is will strive to duplicate.
>
> The nearest they offer to the reference implementation is secntpd. I
> get the impression that this is a fork, but I haven't looked in detail.
> That is the package that includes ntpq. I can't therefore easily
> install ntpq without risking replacing the current time synch daemon,
> which is good enough for my current purposes, so I can't easily dry run
> ntpq, with my current system, even though I've used it on work systems,
> in the past.

On Monday, April 11, 2022 at 6:10:45 AM UTC+12, David Taylor wrote:
> On 10/04/2022 18:23, David Woolley wrote:
> > On 10/04/2022 16:27, William Unruh wrote:
> >> ntpd is the reference implementation of NTP.
> > Standard Debian comes with systemd-timesyncd, and SNTP client, so it
> > does not include the reference implementation.
> >
> > The nearest they offer to the reference implementation is secntpd. I
> > get the impression that this is a fork, but I haven't looked in detail.
> > That is the package that includes ntpq. I can't therefore easily
> > install ntpq without risking replacing the current time synch daemon,
> > which is good enough for my current purposes, so I can't easily dry run
> > ntpq, with my current system, even though I've used it on work systems,
> > in the past.
> Same on the Raspberry Pi - no reference NTP. I just:
>
> sudo apt-get install ntp
>
> and all works as expected.
> --
> Cheers,
> David
> Web: http://www.satsignal.eu
============================================
OK, I have now done that; where to now?

pi@raspberrypi:~ $ sudo apt-get install ntp
Reading package lists... Done
Building dependency tree
Reading state information... Done
ntp is already the newest version (1:4.2.8p10+dfsg-3+deb9u2).
0 upgraded, 0 newly installed, 0 to remove and 131 not upgraded.
pi@raspberrypi:~ $

On 11/04/2022 02:04, Derek B wrote:

> 0 upgraded, 0 newly installed, 0 to remove and 131 not upgraded.

It was already installed. What I was saying about my system is that I
had to rely on the documentation of ntpq to know what options to use, as
I'd otherwise break the "if it isn't broken, don't fix it rule".

What you do now is try ntpq with the same options as you could have
used before your null install.

On 2022-04-08, David Woolley <david@ex.djwhome.demon.invalid> wrote:
> ntpq -p doesn't display everything about the peers. In particular, it
> doesn't display the upstream statistics, such as root delay and root
> dispersion, which are used to select the system peer; it only reports
> the last hop figures.

Yes, it would help to see the "ntpq -c rv" output for all the servers.

> offset is never used as a quality figure, except if it is far enough out
> to declare a falseticker (using it would introduce confirmation bias).

The offset is also used in the clustering algorithm:
https://www.eecis.udel.edu/~mills/ntp/html/cluster.html

--
Miroslav Lichvar