Empathy List Archives

TP

Trent Piepho

Mon, Mar 20, 2017 9:00 PM

Hello time nuts,

I'm working on a custom embedded Linux device, with a custom inertial reference unit, which contains a GPS module. The module is a Telit JN3, which is based on the SiRFSTAR IV I believe. I'd like to use the GPS to sync the Linux system clock. Eventually I'd like to use the PPS signal, which is routed to a FPGA that's part of the CPU, to implement a custom PPS hardware module that I can write a kernel driver for and use the Linux hardpps system. And maybe make that feedback to the CPU's main clock source, since the FPGA also controls that and could create a PLL between the TCXO that serves as the master clock signal and the CPU's source clock.

But first things first. I'm just grabbing the time from NMEA sentences. And there's quite a bit of jitter there! Clearly using the first sentence output by the GPS is critical. I've tried to account for any time delays in the software. I think it's the GPS module that is creating the largest source of jitter. It appears to go in four hour cycles, peaking at 0:00Z, 4:00Z, 8:00Z, etc.

Does this sound like something that one would expect with the NMEA output of a non-timing GPS? Is it related to satellite orbits? Or perhaps is has something to do with the design of the SiRFStar IV?

I'll attach a graph of what I'm seeing. If the attachment doesn't come though it's viewable at https://goo.gl/photos/JtYfJCpRSZb3hCnM8.

Methodology for the graph:
System clock is left free running and not disciplined, after an initial one time set based on the GPS time.
On each NMEA GGA sentence, sent at 1 Hz, the system clock's offset from the NMEA timestamp is measured.
Each minute, the mean, std.dev, min and max are found for the last 60 offset samples. This is plotted on the graph.
Any outlier samples, defined as more than 3 sigma from the previous mean, are also plotted.
Concurrently, the chrony NTP daemon is running and monitoring the IT dept's NTP server, but NOT being used to set the local system clock.
Once a minute, the system clock's offset to chrony's idea of the NTP server's clock is also measured. Chrony is using an algorithm based on median filtering to get its idea of the NTP server's clock.
The NTP server is just a windows domain controller synced to the internet NTP pool and far from a precision source.

Hello time nuts, I'm working on a custom embedded Linux device, with a custom inertial reference unit, which contains a GPS module. The module is a Telit JN3, which is based on the SiRFSTAR IV I believe. I'd like to use the GPS to sync the Linux system clock. Eventually I'd like to use the PPS signal, which is routed to a FPGA that's part of the CPU, to implement a custom PPS hardware module that I can write a kernel driver for and use the Linux hardpps system. And maybe make that feedback to the CPU's main clock source, since the FPGA also controls that and could create a PLL between the TCXO that serves as the master clock signal and the CPU's source clock. But first things first. I'm just grabbing the time from NMEA sentences. And there's quite a bit of jitter there! Clearly using the first sentence output by the GPS is critical. I've tried to account for any time delays in the software. I think it's the GPS module that is creating the largest source of jitter. It appears to go in four hour cycles, peaking at 0:00Z, 4:00Z, 8:00Z, etc. Does this sound like something that one would expect with the NMEA output of a non-timing GPS? Is it related to satellite orbits? Or perhaps is has something to do with the design of the SiRFStar IV? I'll attach a graph of what I'm seeing. If the attachment doesn't come though it's viewable at https://goo.gl/photos/JtYfJCpRSZb3hCnM8. Methodology for the graph: System clock is left free running and not disciplined, after an initial one time set based on the GPS time. On each NMEA GGA sentence, sent at 1 Hz, the system clock's offset from the NMEA timestamp is measured. Each minute, the mean, std.dev, min and max are found for the last 60 offset samples. This is plotted on the graph. Any outlier samples, defined as more than 3 sigma from the previous mean, are also plotted. Concurrently, the chrony NTP daemon is running and monitoring the IT dept's NTP server, but NOT being used to set the local system clock. Once a minute, the system clock's offset to chrony's idea of the NTP server's clock is also measured. Chrony is using an algorithm based on median filtering to get its idea of the NTP server's clock. The NTP server is just a windows domain controller synced to the internet NTP pool and far from a precision source.

BC

Bob Camp

Mon, Mar 20, 2017 10:14 PM

Hi

NMEA sentences are not the best thing to use for timing. If you do decide to use them, configure the
receiver so that one and only one sentence comes out. Any time you have more than one, you run the risk
of collision in the serial buffer on the part. Next thing to do is to pick the shortest useful sentence you
can find. In some cases you only have one option.

Next up is to be sure that your PC is set up so that there is minimal lag in serial processing. That may not
be as simple as you might think. All modern OS’s head off to do “interesting things” from time to time. The
usual approach is to kill just about anything that might create an issue.

Once you have all that, you start pruning the outliers and fitting to the center of the data. Since you don’t have
a super duper clock on your motherboard, you are fitting both the randomness of the clock and the GPS. The
answer is to go slow and look at data over a lot of samples.

The other answer to all this is to use a GPS with a PPS out. Feed that into some sort of capture process and
go from there. The result is likely to improve things by at least a couple orders of magnitude.

Bob

On Mar 20, 2017, at 5:00 PM, Trent Piepho tpiepho@kymetacorp.com wrote:

Hello time nuts,

I'm working on a custom embedded Linux device, with a custom inertial reference unit, which contains a GPS module. The module is a Telit JN3, which is based on the SiRFSTAR IV I believe. I'd like to use the GPS to sync the Linux system clock. Eventually I'd like to use the PPS signal, which is routed to a FPGA that's part of the CPU, to implement a custom PPS hardware module that I can write a kernel driver for and use the Linux hardpps system. And maybe make that feedback to the CPU's main clock source, since the FPGA also controls that and could create a PLL between the TCXO that serves as the master clock signal and the CPU's source clock.

But first things first. I'm just grabbing the time from NMEA sentences. And there's quite a bit of jitter there! Clearly using the first sentence output by the GPS is critical. I've tried to account for any time delays in the software. I think it's the GPS module that is creating the largest source of jitter. It appears to go in four hour cycles, peaking at 0:00Z, 4:00Z, 8:00Z, etc.

Does this sound like something that one would expect with the NMEA output of a non-timing GPS? Is it related to satellite orbits? Or perhaps is has something to do with the design of the SiRFStar IV?

I'll attach a graph of what I'm seeing. If the attachment doesn't come though it's viewable at https://goo.gl/photos/JtYfJCpRSZb3hCnM8.

Methodology for the graph:
System clock is left free running and not disciplined, after an initial one time set based on the GPS time.
On each NMEA GGA sentence, sent at 1 Hz, the system clock's offset from the NMEA timestamp is measured.
Each minute, the mean, std.dev, min and max are found for the last 60 offset samples. This is plotted on the graph.
Any outlier samples, defined as more than 3 sigma from the previous mean, are also plotted.
Concurrently, the chrony NTP daemon is running and monitoring the IT dept's NTP server, but NOT being used to set the local system clock.
Once a minute, the system clock's offset to chrony's idea of the NTP server's clock is also measured. Chrony is using an algorithm based on median filtering to get its idea of the NTP server's clock.
The NTP server is just a windows domain controller synced to the internet NTP pool and far from a precision source.
<plot1.png>_______________________________________________
time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

Hi NMEA sentences are not the best thing to use for timing. If you *do* decide to use them, configure the receiver so that one and only one sentence comes out. Any time you have more than one, you run the risk of collision in the serial buffer on the part. Next thing to do is to pick the shortest useful sentence you can find. In some cases you only have one option. Next up is to be sure that your PC is set up so that there is minimal lag in serial processing. That may not be as simple as you might think. All modern OS’s head off to do “interesting things” from time to time. The usual approach is to kill just about anything that *might* create an issue. Once you have all that, you start pruning the outliers and fitting to the center of the data. Since you don’t have a super duper clock on your motherboard, you are fitting both the randomness of the clock and the GPS. The answer is to go slow and look at data over a lot of samples. The other answer to all this is to use a GPS with a PPS out. Feed that into some sort of capture process and go from there. The result is likely to improve things by at least a couple orders of magnitude. Bob > On Mar 20, 2017, at 5:00 PM, Trent Piepho <tpiepho@kymetacorp.com> wrote: > > Hello time nuts, > > I'm working on a custom embedded Linux device, with a custom inertial reference unit, which contains a GPS module. The module is a Telit JN3, which is based on the SiRFSTAR IV I believe. I'd like to use the GPS to sync the Linux system clock. Eventually I'd like to use the PPS signal, which is routed to a FPGA that's part of the CPU, to implement a custom PPS hardware module that I can write a kernel driver for and use the Linux hardpps system. And maybe make that feedback to the CPU's main clock source, since the FPGA also controls that and could create a PLL between the TCXO that serves as the master clock signal and the CPU's source clock. > > But first things first. I'm just grabbing the time from NMEA sentences. And there's quite a bit of jitter there! Clearly using the first sentence output by the GPS is critical. I've tried to account for any time delays in the software. I think it's the GPS module that is creating the largest source of jitter. It appears to go in four hour cycles, peaking at 0:00Z, 4:00Z, 8:00Z, etc. > > Does this sound like something that one would expect with the NMEA output of a non-timing GPS? Is it related to satellite orbits? Or perhaps is has something to do with the design of the SiRFStar IV? > > I'll attach a graph of what I'm seeing. If the attachment doesn't come though it's viewable at https://goo.gl/photos/JtYfJCpRSZb3hCnM8. > > Methodology for the graph: > System clock is left free running and not disciplined, after an initial one time set based on the GPS time. > On each NMEA GGA sentence, sent at 1 Hz, the system clock's offset from the NMEA timestamp is measured. > Each minute, the mean, std.dev, min and max are found for the last 60 offset samples. This is plotted on the graph. > Any outlier samples, defined as more than 3 sigma from the previous mean, are also plotted. > Concurrently, the chrony NTP daemon is running and monitoring the IT dept's NTP server, but NOT being used to set the local system clock. > Once a minute, the system clock's offset to chrony's idea of the NTP server's clock is also measured. Chrony is using an algorithm based on median filtering to get its idea of the NTP server's clock. > The NTP server is just a windows domain controller synced to the internet NTP pool and far from a precision source. > <plot1.png>_______________________________________________ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts > and follow the instructions there.

CA

Chris Albertson

Mon, Mar 20, 2017 10:19 PM

Does this sound like something that one would expect with the NMEA output of a non-timing GPS? Is it related to satellite orbits? Or perhaps is has something to do with the design of the SiRFStar IV?

Remember the phone based time service? "At the tome time time will be
.... BEEP" With a GPS the NEMA sentence take the place of the spoken
words on the phone. The NEMA specification allows the sentences to
up to one second "off". That said most GPS receivers do MUCH better
than the NEMA spec but you should never count on non-specified
performance in a professional design. The PPS is of course doing the
job of the BEEP on the old phone system. Set you watch based on the
beep, not NEMA.

If you have an FPGA available then you could significantly improve
system time keeping. Currently the PPS interrupts the CPU to
snapshot internal counter. Unpredictable interrupt latency lifts NTP
timekeeping to about 1 or 2 microseconds but is the counter snap
shooting could be moved out to FPGA hardware there would be no unknown
latency and you could get NTP to break a "magic" 1uS barrier. I've
only hear of 1 uS being broken with hardware. You would actually
not ned to write much software to make this happen, just move the
counter outside the CPU to the FPGA and you about have it.

Chris Albertson
Redondo Beach, California

> Does this sound like something that one would expect with the NMEA output of a non-timing GPS? Is it related to satellite orbits? Or perhaps is has something to do with the design of the SiRFStar IV? > Remember the phone based time service? "At the tome time time will be .... BEEP" With a GPS the NEMA sentence take the place of the spoken words on the phone. The NEMA specification allows the sentences to up to one second "off". That said most GPS receivers do MUCH better than the NEMA spec but you should never count on non-specified performance in a professional design. The PPS is of course doing the job of the BEEP on the old phone system. Set you watch based on the beep, not NEMA. If you have an FPGA available then you could significantly improve system time keeping. Currently the PPS interrupts the CPU to snapshot internal counter. Unpredictable interrupt latency lifts NTP timekeeping to about 1 or 2 microseconds but is the counter snap shooting could be moved out to FPGA hardware there would be no unknown latency and you could get NTP to break a "magic" 1uS barrier. I've only hear of 1 uS being broken with hardware. You would actually not ned to write much software to make this happen, just move the counter outside the CPU to the FPGA and you about have it. Chris Albertson Redondo Beach, California

KG

Kiwi Geoff

Mon, Mar 20, 2017 11:43 PM

Hi Trent,

But first things first. I'm just grabbing the time from NMEA sentences.
And there's quite a bit of jitter there! Clearly using the first sentence
output by the GPS is critical. I've tried to account for any time delays in
the software. I think it's the GPS module that is creating the largest
source of jitter. It appears to go in four hour cycles, peaking at 0:00Z,
4:00Z, 8:00Z, etc.

From my observations on various GPS receivers, the 'time" that the

NMEA data is transmitted can be highly variable.

For example, here is a (24 hour) graph from my Garmin 18x (firmware
v3.6) where a plot (thanks to Hal) shows the start time of the NMEA
sentence from the time of the GPS 1PPS edge.

http://www.satsignal.eu/ntp/Garmin-18x-3.7.png

David Taylor explains more about this Garmin firmware variation here:

http://www.satsignal.eu/ntp/Garmin-GSP18x-LVC-firmware-issue.htm

Where an earlier version of the 18x, the latency could be so looooong
that it actually gave the timestamp for the wrong second !

So yes, beware of using the "transmit time" of NMEA sentences, many
GPS receivers appear to have a cuppa tea, a scone and solve a Sudoko
before telling us the time !

Regards, Kiwi Geoff (Christchurch , New Zealand).

Hi Trent, > But first things first. I'm just grabbing the time from NMEA sentences. > And there's quite a bit of jitter there! Clearly using the first sentence > output by the GPS is critical. I've tried to account for any time delays in > the software. I think it's the GPS module that is creating the largest > source of jitter. It appears to go in four hour cycles, peaking at 0:00Z, > 4:00Z, 8:00Z, etc. >From my observations on various GPS receivers, the 'time" that the NMEA data is transmitted can be highly variable. For example, here is a (24 hour) graph from my Garmin 18x (firmware v3.6) where a plot (thanks to Hal) shows the start time of the NMEA sentence from the time of the GPS 1PPS edge. http://www.satsignal.eu/ntp/Garmin-18x-3.7.png David Taylor explains more about this Garmin firmware variation here: http://www.satsignal.eu/ntp/Garmin-GSP18x-LVC-firmware-issue.htm Where an earlier version of the 18x, the latency could be so looooong that it actually gave the timestamp for the wrong second ! So yes, beware of using the "transmit time" of NMEA sentences, many GPS receivers appear to have a cuppa tea, a scone and solve a Sudoko before telling us the time ! Regards, Kiwi Geoff (Christchurch , New Zealand).

GE

Gary E. Miller

Tue, Mar 21, 2017 12:06 AM

Yo Chris!

On Mon, 20 Mar 2017 15:19:25 -0700
Chris Albertson albertson.chris@gmail.com wrote:

I've only hear of 1 uS being broken with hardware.

A Raspberry Pi can get down to a Standard Deviation of about 350 nano seconds
using NTPsec..

https://blog.ntpsec.org/2017/02/01/heat-it-up.html

RGDS
GARY

Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
gem@rellim.com Tel:+1 541 382 8588

    Veritas liberabit vos. -- Quid est veritas?
"If you can’t measure it, you can’t improve it." - Lord Kelvin

Yo Chris! On Mon, 20 Mar 2017 15:19:25 -0700 Chris Albertson <albertson.chris@gmail.com> wrote: > I've only hear of 1 uS being broken with hardware. A Raspberry Pi can get down to a Standard Deviation of about 350 nano seconds using NTPsec.. https://blog.ntpsec.org/2017/02/01/heat-it-up.html RGDS GARY --------------------------------------------------------------------------- Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 gem@rellim.com Tel:+1 541 382 8588 Veritas liberabit vos. -- Quid est veritas? "If you can’t measure it, you can’t improve it." - Lord Kelvin

CA

Chris Albertson

Tue, Mar 21, 2017 5:22 AM

Wow. 350 nS. I had not been following this for the last few years. It
seems the ARM is a simply CPU with much more predictable interrupts timing.

One might ask way you'd need such good internal timing. What got me into
this years ago was scientific data acquisition. I wanted to time stamp the
data. So I had a Linux based PC connect periodically to the Internet
using a dial-up phone modem and run NTP. It worked well enough. A GPS
receiver at $10,000 each was out of the question.

On Mon, Mar 20, 2017 at 5:06 PM, Gary E. Miller gem@rellim.com wrote:

Yo Chris!

On Mon, 20 Mar 2017 15:19:25 -0700
Chris Albertson albertson.chris@gmail.com wrote:

I've only hear of 1 uS being broken with hardware.

A Raspberry Pi can get down to a Standard Deviation of about 350 nano
seconds
using NTPsec..

https://blog.ntpsec.org/2017/02/01/heat-it-up.html

RGDS
GARY

Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703
gem@rellim.com Tel:+1 541 382 8588

         Veritas liberabit vos. -- Quid est veritas?
 "If you can’t measure it, you can’t improve it." - Lord Kelvin

time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/
mailman/listinfo/time-nuts
and follow the instructions there.

--

Chris Albertson
Redondo Beach, California

Wow. 350 nS. I had not been following this for the last few years. It seems the ARM is a simply CPU with much more predictable interrupts timing. One might ask way you'd need such good internal timing. What got me into this years ago was scientific data acquisition. I wanted to time stamp the data. So I had a Linux based PC connect periodically to the Internet using a dial-up phone modem and run NTP. It worked well enough. A GPS receiver at $10,000 each was out of the question. On Mon, Mar 20, 2017 at 5:06 PM, Gary E. Miller <gem@rellim.com> wrote: > Yo Chris! > > On Mon, 20 Mar 2017 15:19:25 -0700 > Chris Albertson <albertson.chris@gmail.com> wrote: > > > I've only hear of 1 uS being broken with hardware. > > A Raspberry Pi can get down to a Standard Deviation of about 350 nano > seconds > using NTPsec.. > > https://blog.ntpsec.org/2017/02/01/heat-it-up.html > > > RGDS > GARY > ------------------------------------------------------------ > --------------- > Gary E. Miller Rellim 109 NW Wilmington Ave., Suite E, Bend, OR 97703 > gem@rellim.com Tel:+1 541 382 8588 > > Veritas liberabit vos. -- Quid est veritas? > "If you can’t measure it, you can’t improve it." - Lord Kelvin > _______________________________________________ > time-nuts mailing list -- time-nuts@febo.com > To unsubscribe, go to https://www.febo.com/cgi-bin/ > mailman/listinfo/time-nuts > and follow the instructions there. > -- Chris Albertson Redondo Beach, California

time-nuts@lists.febo.com

Four hour cycle in GPS NMEA jitter

RGDS
GARY

RGDS
GARY