Time math libraries
(Apologies if this question has been addressed before. Archive search is rather cumbersome on a month-by-month basis.)
Few people will be surprised to learn that MS Excel does not account for leap seconds when doing time math. See below for an example. This is just an example of many instances of programming failures.
Are there good time math software libraries (e.g., Java, C++/C#, etc) that will do time math correctly for the chosen time scale?
Thanks.
— Eric
2016 Dec 31 Sat 23:59:57
2016 Dec 31 Sat 23:59:58
2016 Dec 31 Sat 23:59:59
2017 Jan 01 Sun 00:00:00
2017 Jan 01 Sun 00:00:01
2017 Jan 01 Sun 00:00:02
My machine is on EST right now. Is it a time zone question?
2016 Dec 31 Sat 18:59:57
2016 Dec 31 Sat 18:59:58
2016 Dec 31 Sat 18:59:59
2016 Dec 31 Sat 19:00:00
2016 Dec 31 Sat 19:00:01
2016 Dec 31 Sat 19:00:02
Nope!
What about past leap seconds?
2015 Jun 30 Tue 23:59:57
2015 Jun 30 Tue 23:59:58
2015 Jun 30 Tue 23:59:59
2015 Jul 01 Wed 00:00:00
2015 Jul 01 Wed 00:00:01
2015 Jul 01 Wed 00:00:02
Also fail!
I have had some success with Perl DateTime CPAN module's support for leap
seconds - doing time delta math without using Unix Epoch Seconds properly
handles leap seconds.
Converting back and forth to Unix Epoch time works as well as it can (given
non uniqueness).
It also supports the concept of a "floating timezone" which never takes
into account leap seconds.
Tim N3QE
On Mon, Dec 12, 2016 at 1:45 PM, Eric Scace eric@scace.org wrote:
(Apologies if this question has been addressed before. Archive search
is rather cumbersome on a month-by-month basis.)
Few people will be surprised to learn that MS Excel does not account
for leap seconds when doing time math. See below for an example. This is
just an example of many instances of programming failures.
Are there good time math software libraries (e.g., Java, C++/C#, etc)
that will do time math correctly for the chosen time scale?
Thanks.
— Eric
2016 Dec 31 Sat 23:59:57
2016 Dec 31 Sat 23:59:58
2016 Dec 31 Sat 23:59:59
2017 Jan 01 Sun 00:00:00
2017 Jan 01 Sun 00:00:01
2017 Jan 01 Sun 00:00:02
My machine is on EST right now. Is it a time zone question?
2016 Dec 31 Sat 18:59:57
2016 Dec 31 Sat 18:59:58
2016 Dec 31 Sat 18:59:59
2016 Dec 31 Sat 19:00:00
2016 Dec 31 Sat 19:00:01
2016 Dec 31 Sat 19:00:02
Nope!
What about past leap seconds?
2015 Jun 30 Tue 23:59:57
2015 Jun 30 Tue 23:59:58
2015 Jun 30 Tue 23:59:59
2015 Jul 01 Wed 00:00:00
2015 Jul 01 Wed 00:00:01
2015 Jul 01 Wed 00:00:02
Also fail!
time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/
mailman/listinfo/time-nuts
and follow the instructions there.
Eric,
I speak as someone who has implemented many calendrical packages. I have
always tried to achieve the maximum degree of generality, and have considered
the issues with leap seconds on more than one occassion.
The first problem with leap seconds is that you need a list of them, and that
list needs constant updating. If date/times are maintained in a standard form
such as seconds since the beginning of an epoch (e.g. 0 AD) then even the
conversion into a conventional date/time is dependent upon the country you are
in. That is just too much overkill for the usual purpose.
In practice, a date/time is loaded from a file or obtained from an operating
system (which MAY take leap seconds into account). At that point it is just a
date/time to be manipulated. If you should be unlucky enough to get a
date/time exactly in a leap second - it really doesn't matter because
date/times are just not synchronized well enough for it to matter.
Where it does matter, there is some local clock which is used and the
date/times are relative to it. This is almost always for timestamps, and the
time since the system boot will generally do well unless you need it across a
network. Then even cable delays become a problem.
So taking leap seconds into account requires a substantial amount of
additional coding, constant updates of possibly distributed code to keep the
list of leap seconds maintained and adversely affects both storage and
performance. For practical matters, even if it could be done at zero cost,
there is no real benefit. Just allowing a day to have 86,401 seconds causes
all sorts of problems with data storage, consistency verification, editing and
ensuring that data entry is correct.
It is the rare application that needs leap seconds - even though it may be
"correct", and certainly an option I have considered many times - but it has
never been justified for anything that I have ever written (which actually
includes operating systems). Consider that the system clock for any processor
will "slip" on a regular basis and need to be reset. The accuracy that most
operating systems can maintain in practice is, at best, only on the order of
a second or two - and that is with regular updates from a timing source.
Now, if you have a processor with a built-in GPSDO, perhaps that could be used
to regulate the system clock (which is just a cheap quartz crystal) and keep
time synchronized to perhaps a millisecond or so. Maybe. But, probably not.
All that is left would be specialized applications that actually require leap
seconds. I don't know of any, but there may be some available. Certainly an
operating system could maintain the necessary lists (assuming that it is
regularly updated). It could use that data for conversions. I don't know of an
operating system that does that though because of the data format problems.
They would report a time such as 24:00:00 which would otherwise be invalid.
That specific time is usually taken as 00:00:00 of the next day. Very few, if
any applications would understand 24:00:00 as a valid time. So, operating
systems can't really afford to take leap seconds into account either.
Michael
(Apologies if this question has been addressed before. Archive search is
rather cumbersome on a month-by-month basis.)
Few people will be surprised to learn that MS Excel does not account for
leap seconds when doing time math. See below for an example. This is just an
example of many instances of programming failures.
Are there good time math software libraries (e.g., Java, C++/C#, etc)
that will do time math correctly for the chosen time scale?
Thanks.
— Eric
2016 Dec 31 Sat 23:59:57
2016 Dec 31 Sat 23:59:58
2016 Dec 31 Sat 23:59:59
2017 Jan 01 Sun 00:00:00
2017 Jan 01 Sun 00:00:01
2017 Jan 01 Sun 00:00:02
My machine is on EST right now. Is it a time zone question?
2016 Dec 31 Sat 18:59:57
2016 Dec 31 Sat 18:59:58
2016 Dec 31 Sat 18:59:59
2016 Dec 31 Sat 19:00:00
2016 Dec 31 Sat 19:00:01
2016 Dec 31 Sat 19:00:02
Nope!
What about past leap seconds?
2015 Jun 30 Tue 23:59:57
2015 Jun 30 Tue 23:59:58
2015 Jun 30 Tue 23:59:59
2015 Jul 01 Wed 00:00:00
2015 Jul 01 Wed 00:00:01
2015 Jul 01 Wed 00:00:02
Also fail!
time-nuts mailing list -- time-nuts@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.
--
Best regards,
Timenut mailto:timenut@metachaos.net
I should clarify my intended application for “time math”.
My company is deep in development of a system that, in part, records events on a blockchain. For various reasons a precise and accurate representation of time of the event can be important. How precise and accurate depends on the application. The first applications need millisecond precision and tens of millisecond accuracy — not a huge demand by TimeNut standards, but not to be neglected. Leap seconds (and the differing contortions being used by various organizations to evade implementation of leap seconds) should not be ignored.
The blockchain runs as a generic event-recording ledger utility supporting many applications. Over time, we will need to get more precise and accurate on time.
To be general, we take the following approach:
Timestamp (and, often, geolocation) are acquired at the point of measurement for the data set. This is treated as the opinion of the collecting device. We record meta-data about the collecting device. For some applications the meta-data includes information as to how the device formed its opinion of time.
The application submits the event to the ledger service’s API for recording.
The API server(s) timestamp the client’s request to record to the ledger. This timestamp is relatively important because it is the declaration of the ledger, an immutable reference of the sequence of events, upon which other systems will rely in future.
The present scheme is to use TAI as the timescale for all events recorded to the ledger. Some of the implications:
Collecting device timestamps may need to be convertible to TAI.
This task can be a mess, because the sources of time opinions may be vast; e.g., cell phones, AWS, Google, MS Azure, etc. We work around this by capturing meta-data about the device’s timescale and source. This allows retrospective consideration of the devices’ time opinion relative to TAI when it is important for the downstream data consumer.
Independent systems participating in the blockchain (e.g., as API server, or as consensus-forming systems) should have consistent opinions of TAI.
Ledger timestamps (in TAI) need to be converted to the data consumer’s desired time scale.
Here’s another instance of the need for a set of time math utilities.
I realize this description also opens a Pandora’s box that involves distribution of time issues. But, for the purposes of this thread, I wanted to focus on the TAI-to-other-timescales conversion utility question.
If good timescale conversion utilities exist, we would use them.
If good timescale conversion utilities do not exist, then maybe we’ll contribute one to the open software space.
— Eric
Eric, and time nuts -- A very nice set of questions and an interesting application. In fact it's so time nutty that I'll send it over to the LEAPSECS mailing list, which is the niche of time-nuts where we deal with issues of UTC, TAI, and leap seconds. You can pick up your replies there.
LEAPSECS -- please help Eric out. I know there are answers from bloated universal date/time packages to one page of elegant code.
Thanks,
/tvb
LEAPSECS mailing list
LEAPSECS@leapsecond.com
https://pairlist6.pair.net/mailman/listinfo/leapsecs
----- Original Message -----
From: "Eric Scace" eric@scace.org
To: "Discussion of precise time and frequency measurement" time-nuts@febo.com
Sent: Tuesday, December 13, 2016 5:50 AM
Subject: Re: [time-nuts] Time math libraries
I should clarify my intended application for “time math”.
My company is deep in development of a system that, in part, records events on a blockchain. For various reasons a precise and accurate representation of time of the event can be important. How precise and accurate depends on the application. The first applications need millisecond precision and tens of millisecond accuracy — not a huge demand by TimeNut standards, but not to be neglected. Leap seconds (and the differing contortions being used by various organizations to evade implementation of leap seconds) should not be ignored.
The blockchain runs as a generic event-recording ledger utility supporting many applications. Over time, we will need to get more precise and accurate on time.
To be general, we take the following approach:
Timestamp (and, often, geolocation) are acquired at the point of measurement for the data set. This is treated as the opinion of the collecting device. We record meta-data about the collecting device. For some applications the meta-data includes information as to how the device formed its opinion of time.
The application submits the event to the ledger service’s API for recording.
The API server(s) timestamp the client’s request to record to the ledger. This timestamp is relatively important because it is the declaration of the ledger, an immutable reference of the sequence of events, upon which other systems will rely in future.
The present scheme is to use TAI as the timescale for all events recorded to the ledger. Some of the implications:
Collecting device timestamps may need to be convertible to TAI.
This task can be a mess, because the sources of time opinions may be vast; e.g., cell phones, AWS, Google, MS Azure, etc. We work around this by capturing meta-data about the device’s timescale and source. This allows retrospective consideration of the devices’ time opinion relative to TAI when it is important for the downstream data consumer.
Independent systems participating in the blockchain (e.g., as API server, or as consensus-forming systems) should have consistent opinions of TAI.
Ledger timestamps (in TAI) need to be converted to the data consumer’s desired time scale.
Here’s another instance of the need for a set of time math utilities.
I realize this description also opens a Pandora’s box that involves distribution of time issues. But, for the purposes of this thread, I wanted to focus on the TAI-to-other-timescales conversion utility question.
If good timescale conversion utilities exist, we would use them.
If good timescale conversion utilities do not exist, then maybe we’ll contribute one to the open software space.
— Eric