Hi Deirdre,

I'd like to repeat your measurement at a different location (eastern
england).

What did you use to capture the data and write it as a vcd file ?

-adrian

On Tue, Feb 6, 2018 at 6:40 PM, Deirdre O'Byrne deirdre.dub@gmail.com
wrote:

In message CAGJ4F+6Nvic=RoUH7EozhKLCDziBTorMGu0iKj-sK6iXjw+zsQ@mail.gmail.com
, "Deirdre O'Byrne" writes:

You can: I baptised it "the blame algoritm".

The trick is not to try to accept pulses as valid but to try to
throw out pulses which are impossible.

Imagine you have a 120 second long shift-register, and you feed
your received pulses into it.

Then try brute force, for every one of the newest 60 positions if
that can be the start of a minute or not, by testing all the
constraints you can think of, and there are surprisingly many.

Some are obvious, the bits encoding the hour cannot contain "39",
but that is a remarkable weak filter that seldom kicks in.

A much stronger filter is that the bits encoding the hour must be
the same as in the previous minute unless minutes were 59 in the
previous minute and zero in this minute.

If you count it up, that is a strong and very peculiar relationship
on all the hour-bits and all the minutes-bits and if even one single
of them are wrong, you can definitively discard that theory for the
start of the minute.

A similar thing holds for the date bits, the time in the
previous minute must be 23:59:59 and in this 00:00:00 for
there to be any difference between the dates, and even
then, only a small number of possible changes in the date
bits are valid.

If you look in http://phk.freebsd.dk/phkrel/NTPns.20080902.tgz
you will find a file called dcf77_blame.c with my code,
here is a couple of the simpler tests:

    /* LSB of minutes must be different from previous minute */
    j = ip->shiftprev[(offset + 21) % 60];
    if (j * ip->shiftreg[(offset + 21) % 60] > 0)
            FAIL((why, " 0"));

    /*
     * If the LSB of minutes was '1' in previous minute
     * the next higher bit must have changed, if it was
     * a '0' it must not.
     */
    if (j *
        ip->shiftreg[(offset + 22) % 60] *
        ip->shiftprev[(offset + 22) % 60] > 0)
            FAIL((why, " 1"));

When using this algorithm, missing pulses is almost a
non-issue up to around 40% of them missing, and even
in an enviroment like that, it is not uncommon to see
the algorithm lock on to the minute in about 34 seconds
and know the full time in less than 3 minutes.

If you make your pulse width discriminator really selective,
which you might as well, you can "blacklist" disproved
minute positions for the next many minutes as the
risk of a '1' and '0' being confused is close to zero.

That will get you minute lock, even with 70%-90% missing
pulses, in a matter of minutes if you use a longer
shift register.

I did a parallel prototype for MSF, but I didn't need it
so I never completed it, not sure I have it around any
more.

I should really write an article about that code...

Poul-Henning

--
Poul-Henning Kamp      | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG        | TCP/IP since RFC 956
FreeBSD committer      | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

On Tue, Feb 6, 2018 at 2:37 PM, Poul-Henning Kamp phk@phk.freebsd.dk wrote:

How difficult would it be to complete these modules and integrate them
with the rest of NTP, as NTP decoder modules?  So instead of an AM HF
receiver, the setup for these signals would be:

LF receiver set to CW demodulation set to appropriate parameters ->
NTP timekeeping system sound card

One of my organization's projects consists of robust public time
transfer via NTP over the Internet, based on a combination of various
on-site standards (rackmount OCXO, rubidium, and/or cesium standards)
and external signals (incl. WWV and CHU, using preamplifier ->
preselector -> analog parametric demodulator -> sound card, controlled
by ionospheric prediction daemon software on GNU/Linux via GPIB), the
nodes being geographically dispersed throughout the US and Canada.
It's probable that I will end up relocating to Western Europe
(coincidentally, the Republic of Ireland!) in the moderate future, and
therefore it would be nice if these LF or HF signals were to be
supported, for use as fallbacks to the standard GNSS sources (each
site typically will have one military and one industrial civilian
rackmount GPS receiver).

-Ruslan

--
Ruslan Nabioullin
Wittgenstein Laboratories
rnabioullin@gmail.com
(508) 523-8535
50 Louise Dr.
Hollis, NH 03049

I used this cheap (but good for the price!) signal analyser -

http://www.ebay.ie/itm/Hobby-Components-UK-USB-24M-8CH-24MHz-Logic-Analyser/161309221423?ssPageName=STRK%3AMEBIDX%3AIT&_trksid=p2057872.m2749.l2649

I used sigrok to do the recording -

https://www.sigrok.org/

Let me know if you collect some data - I'd be interested in seeing your
results!

On 6 February 2018 at 19:36, Adrian Godwin artgodwin@gmail.com wrote:

In message CAHDZN8n=pKMnve=L-i539tiHo214Ev80Oe0gD_eXdhgmk+jy+g@mail.gmail.com
, Ruslan Nabioullin writes:

Well, NTPns is a NTP server program, but a very special one.

If you really want to receive VLF for NTP purposes, you should hook
a 1M sample/sec ADC to your antenna and implement all the
demodulation in software.

(The BeagleBoneBlack with its two PRU units would be close to
a perfect platform for this)

The more stable your sample frequency your ADC has, the easier
the software will be to write.

You can carrier-track DCF77, 162kHz France, MSF and Loran-C in one
go, by simply averaging the raw antennasignal in a long enough
circular buffer, then multiply by a suitable SIN/COS complex signal.

For signals on kHz spacing (50, 162, 198 any other you care for)
you can make a single one millisecond (ie: 1000 samples) circular
buffer, and extract all the phases from that buffer, by multiplying
by suitable SIN/COS and averaging the result to DC.

For 77.5 kHz you will need a 2 millisecond buffer.

If you want to recover the phase modulation on 162 kHz you can
use the phase from the first buffer as your reference.

(See: http://phk.freebsd.dk/loran-c/SW)

If you want to recover the PRNG phase on DCF77 you will need to
do the full early/prompt/late correlation, but that is also
pretty cheap as you can pre-generate the sequence to compare with
at compile time.

Alternatively make a full-second buffer and average into that with
polarity suitably swapped based on AM pulse width, and do the
early/prompt/later about once a minute or so in high-level software.

Loran-C is the same basic story, I did that on a 32bit ARM
some years ago.

(See: http://phk.freebsd.dk/AducLoran/)

So much code to write and so little time...

--
Poul-Henning Kamp      | UNIX since Zilog Zeus 3.20
phk@FreeBSD.ORG        | TCP/IP since RFC 956
FreeBSD committer      | BSD since 4.3-tahoe
Never attribute to malice what can adequately be explained by incompetence.

That's the next level of error detection. Unfortunately I notice with MSF
and these cheap receivers it can be difficult to determine what the binary
values being transmitted during the second were, which was my motivation
for the analysis I did.

I notice that there is one significant difference between DCF and MSF - the
former does not entirely turn off the carrier during the second, whereas
the latter does. I suspect that is the reason why with MSF receivers it can
so often be difficult to determine what the binary values transmitted were,
as the receiver AGC circuitry possibly has a difficult time keeping up with
drastically changing received power levels, which subsequently give rise to
noise in the received signal.

Splitting the MSF received signal into 100ms chunks, all of the seconds
apart from the start-of-minute marker are of the form 0AB11111. Using "x"
to represent a 100ms chunk whose value could not be determined, I notice
that many of the received seconds were of the form "0AB1x111" or "0AB11x11"
etc - i.e. there was only one 100ms chunk within the second whose value
could not be reliably determined and whose value was non-critical. With a
blame algorithm in place it should be possible to recover these signals.

On 6 February 2018 at 19:37, Poul-Henning Kamp phk@phk.freebsd.dk wrote:

Deirdre,

That's a nice piece of work. Thanks for posting it. And I hope we continue to see results as you refine the decoding process.

Some comments:

1. Knowing the UTC tick (via GPS) is, of course, a benefit. If it were an isolated MSF receiver you wouldn't have this information, but for your purposes of exploring algorithms it is useful.

2. Not all decoding errors are equal. Since this is a time code instead of arbitrary binary data you can use the internal structure of the data to your benefit. For example, you know all the bits for the year and month and day of week shouldn't change over an entire day and so this increases your decoding confidence. Day of week is correlated to date. Hours and minutes will increment and you can anticipate this. A large number of month, day, hour, and minute values will never be transmitted. For any BCD value, 6 of 16 states are invalid. There are parity bits. So all of these constraints can guide your optimal weighted decoding process. You essentially decode against a known template and this is more robust than having to decode blind.

3. It's clever to use a 20 kHz synchronous logic analyzer for this analysis. As you observe, the 50 us granularity is sufficient for your needs. A side-effect of your data set is that you can track performance of the oscillator inside the logic analyzer: convert the 700k GPS timestamps into interval, find and replace the 4 glitch lines with 2 lines of 1.000150, and then use Stable32 or TimeLab to plot. I used a 10 minute running average to reduce the 50 us quantization noise. Note the mean frequency of your timebase is 152 ppm low. Over 8 days this results in a cumulative sampling error of 105 seconds. If your decoding algorithms are relative instead of absolute this won't be a problem. OTOH, you may be able to use your decoding process to detect this drift and then compensate for it in software. You have the beginnings of a MSF-Disciplined-Oscillator project.

Attached are some frequency and ADEV plots for your oscillator. Note the initial frequency drift during the first hour of the run; probably a thermal effect. Ok, these plots have nothing to do with decoding MSF, but it does show how one can use a logic analyzer as a cheap "counter" for long-term frequency accuracy and stability measurements. It's essentially what the picPET, or any other time-stamping counter, does.

/tvb

----- Original Message -----
From: "Deirdre O'Byrne" deirdre.dub@gmail.com
To: "Discussion of precise time and frequency measurement" time-nuts@febo.com
Sent: Tuesday, February 06, 2018 10:40 AM
Subject: [time-nuts] Receiving the MSF time signal on cheap radio modules

Hi

If you want to get even more “nutty", look at the “seed” that you likely already have
for the computation. In this day and age, you probably know what day / month / year it is.
Since you might not (say) know the hour, you have a +/- 1 day sort of tolerance on that. It rolls
into month and year in some cases. The seed adds complexity, but probably makes
things more robust.

If the purpose is to “always be right” then retaining a seed probably improves things.

The flip side is (of course) “what if I’ve been lied to?”. That applies with or without a seed.
Heading off into a situation where you never (re)lock could be one result. How long do you go before
you decide to try something else?

One cute thing is that this stuff is (in general) not very compute intensive. If data past the
minute tick is being looked at, you probably can afford to run multiple parallel solutions (even
on a < $5 MCU).

Lots of zigs and zags ….

Bob

Tom,

Thanks for the feedback!

On 6 February 2018 at 20:29, Tom Van Baak tvb@leapsecond.com wrote:

As I said to Poul-Henning, that is the next level of error detection, which
also has application in error correcting some of the "almost-right" signals.

I made it out to be 152.2ppm, which is kinda disappointing. But the signal
analyser cost very little, and you get what you pay for.

I have not yet wrapped my head around how to create ADEV plots, so thanks
for your work on that - it's interesting to see that (presumed) initial
thermal effect.

MSF disciplined oscillator?! I don't trust these receivers to any better
than about the 20ms mark, so such a disciplined oscillator would have quite
a long integration time!

Thanks again.