I'm working with a variety of things which work in UTC or GPS week/millisecond, so we're doing a lot of conversion back and forth. (the spacecraft puts out time in week:millisecond, all the ground systems processing is in UTC) The question comes up when converting back and forth, and whether various libraries handle leap seconds correctly. For now, I can use a hack of not computing back to 6 Jan 1980, but use an epoch like 15 Dec 2018 (week 2031: 518,400.000 seconds) and hope there's no leap second in the offing. For instance, in Python, if I do a datetime(2016,12,31,0,0,0) + timedelta(hours=30) does it come out as 1/1/2017 6:00:00 or 5:59:59 (it comes out 0600) Similarly, does Excel's time formatting allow for some minutes having an extra second, or does it just assume all minutes are 60 seconds. I'll probably test it for the cases I'm interested in (Ruby, Python, Excel, Matlab, Octave), but if someone else has already done it, then I've got something to cross check against. (python does NOT know about leap seconds) import datetime d = datetime.datetime(2016,12,31) dt = datetime.timedelta(hours=30) d Out[4]: datetime.datetime(2016, 12, 31, 0, 0) dt Out[5]: datetime.timedelta(1, 21600) d+dt Out[6]: datetime.datetime(2017, 1, 1, 6, 0)

AFIK no programming systems professing POSIX compliance implements Leap Seconds. There is however a timezone UTCLeapSeconds will allow this apparently. >> t1.TimeZone = ‘UTCLeapSeconds' Obviously youR TZ data base needs to have it and your Python needs to be at the required level; My test didn’t work, maybe I am not at the right levels. >>> t1 = datetime >>> t1.timezone = 'UTCLeapSeconds' >>> t1 = datetime.datetime(2016,12,31,23,59,58) >>> dt = datetime.timedelta(seconds=1) >>> print t1 2016-12-31 23:59:58 >>> print t1+dt 2016-12-31 23:59:59 >>> print t1+dt+dt 2017-01-01 00:00:00 >>> print t1+dt+dt+dt 2017-01-01 00:00:01 I guess the same facility exists for other programming systems MATLAB has it I believe. Google is your friend. Mike > Le 15 janv. 2019 à 15:54, jimlux <jimlux@earthlink.net> a écrit : > > I'm working with a variety of things which work in UTC or GPS week/millisecond, so we're doing a lot of conversion back and forth. > (the spacecraft puts out time in week:millisecond, all the ground systems processing is in UTC) > > The question comes up when converting back and forth, and whether various libraries handle leap seconds correctly. > For now, I can use a hack of not computing back to 6 Jan 1980, but use an epoch like 15 Dec 2018 (week 2031: 518,400.000 seconds) and hope there's no leap second in the offing. > > > For instance, in Python, if I do a datetime(2016,12,31,0,0,0) + timedelta(hours=30) does it come out as 1/1/2017 6:00:00 or 5:59:59 (it comes out 0600) > > Similarly, does Excel's time formatting allow for some minutes having an extra second, or does it just assume all minutes are 60 seconds. > > I'll probably test it for the cases I'm interested in (Ruby, Python, Excel, Matlab, Octave), but if someone else has already done it, then I've got something to cross check against. > > > (python does NOT know about leap seconds) > > import datetime > > d = datetime.datetime(2016,12,31) > > dt = datetime.timedelta(hours=30) > > d > Out[4]: datetime.datetime(2016, 12, 31, 0, 0) > > dt > Out[5]: datetime.timedelta(1, 21600) > > d+dt > Out[6]: datetime.datetime(2017, 1, 1, 6, 0) > > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > and follow the instructions there. I am not a a vegetarian because I love animals. I’m a vegetarian because I hate plants.

I double check the Python code, and I can confirm it does not take LEAP seconds into account. I highly doubt you will find standard time libraries for the most common languages which will deal with LEAP seconds. They would rather just ignore it and have one less of a headache to worry about (remember all the bugs that pop up even when we switch in and out of DST, like applications crashing because NTP applies the 1 hour change in a discontinous manner, as well as iphone alarms not working when the DST date is modified?). The C++ boost time library docs seem to imply it's possible to use them and have them account for LEAP seconds: https://www.boost.org/doc/libs/1_58_0/doc/html/date_time.html What is worse is tha the POSIX standard requires a day to be 86400 seconds, and pretty much all the code which relies on POSIX makes such assumption, in fact in my experience the day length is usually hardcoded to 86400 seconds: https://www.ucolick.org/~sla/leapsecs/right+gps.html ================================= fcattane@linux-mint-64:~$ python Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import time >>> import datetime >>> >>> # last leap second got added on 1 Jan 2017 ... # ... >>> unix_time_2017_01_01 = 1483228800 >>> >>> >>> >>> unix_time_2017_01_01 1483228800 >>> >>> time.gmtime(unix_time_2017_01_01 - 2) time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=23, tm_min=59, tm_sec=58, tm_wday=5, tm_yday=366, tm_isdst=0) >>> time.gmtime(unix_time_2017_01_01 - 1) time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=23, tm_min=59, tm_sec=59, tm_wday=5, tm_yday=366, tm_isdst=0) >>> time.gmtime(unix_time_2017_01_01) time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=6, tm_yday=1, tm_isdst=0) >>> time.gmtime(unix_time_2017_01_01 + 1) time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=1, tm_wday=6, tm_yday=1, tm_isdst=0) >>> time.gmtime(unix_time_2017_01_01 + 2) time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=2, tm_wday=6, tm_yday=1, tm_isdst=0) -- Fio Cattaneo Universal AC, can Entropy be reversed? -- "THERE IS AS YET INSUFFICIENT DATA FOR A MEANINGFUL ANSWER." On Tue, Jan 15, 2019 at 10:01 AM jimlux <jimlux@earthlink.net> wrote: > > I'm working with a variety of things which work in UTC or GPS > week/millisecond, so we're doing a lot of conversion back and forth. > (the spacecraft puts out time in week:millisecond, all the ground > systems processing is in UTC) > > The question comes up when converting back and forth, and whether > various libraries handle leap seconds correctly. > For now, I can use a hack of not computing back to 6 Jan 1980, but use > an epoch like 15 Dec 2018 (week 2031: 518,400.000 seconds) and hope > there's no leap second in the offing. > > > For instance, in Python, if I do a datetime(2016,12,31,0,0,0) + > timedelta(hours=30) does it come out as 1/1/2017 6:00:00 or 5:59:59 (it > comes out 0600) > > Similarly, does Excel's time formatting allow for some minutes having an > extra second, or does it just assume all minutes are 60 seconds. > > I'll probably test it for the cases I'm interested in (Ruby, Python, > Excel, Matlab, Octave), but if someone else has already done it, then > I've got something to cross check against. > > > (python does NOT know about leap seconds) > > import datetime > > d = datetime.datetime(2016,12,31) > > dt = datetime.timedelta(hours=30) > > d > Out[4]: datetime.datetime(2016, 12, 31, 0, 0) > > dt > Out[5]: datetime.timedelta(1, 21600) > > d+dt > Out[6]: datetime.datetime(2017, 1, 1, 6, 0) > > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > and follow the instructions there.

There are few libraries which offer time unadjusted for leap seconds, like TAI time: https://en.cppreference.com/w/cpp/chrono/tai_clock Some other libraries (typically in written in languages which very few people use like haskell ) offer leapsecond support, but the fundamental problem (as mentioned in the writeup) POSIX requires a day length to be 86400 *AND* requires to be UTC. These two requirements are fundamentally at odds with each other, which is why it's so complicated to deal with this in general purpose code. -- Fio Cattaneo Universal AC, can Entropy be reversed? -- "THERE IS AS YET INSUFFICIENT DATA FOR A MEANINGFUL ANSWER." On Tue, Jan 15, 2019 at 12:25 PM Fiorenzo Cattaneo <fio@cattaneo.us> wrote: > > I double check the Python code, and I can confirm it does not take > LEAP seconds into account. I highly doubt you will find standard time > libraries for the most common languages which will deal with LEAP > seconds. They would rather just ignore it and have one less of a > headache to worry about (remember all the bugs that pop up even when > we switch in and out of DST, like applications crashing because NTP > applies the 1 hour change in a discontinous manner, as well as iphone > alarms not working when the DST date is modified?). > > The C++ boost time library docs seem to imply it's possible to use > them and have them account for LEAP seconds: > https://www.boost.org/doc/libs/1_58_0/doc/html/date_time.html > > What is worse is tha the POSIX standard requires a day to be 86400 > seconds, and pretty much all the code which relies on POSIX makes such > assumption, in fact in my experience the day length is usually > hardcoded to 86400 seconds: > https://www.ucolick.org/~sla/leapsecs/right+gps.html > > ================================= > > > fcattane@linux-mint-64:~$ python > Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34) > [GCC 7.3.0] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import time > >>> import datetime > >>> > >>> # last leap second got added on 1 Jan 2017 > ... # > ... > >>> unix_time_2017_01_01 = 1483228800 > >>> > >>> > >>> > >>> unix_time_2017_01_01 > 1483228800 > >>> > > > >>> time.gmtime(unix_time_2017_01_01 - 2) > time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=23, > tm_min=59, tm_sec=58, tm_wday=5, tm_yday=366, tm_isdst=0) > > >>> time.gmtime(unix_time_2017_01_01 - 1) > time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=23, > tm_min=59, tm_sec=59, tm_wday=5, tm_yday=366, tm_isdst=0) > > >>> time.gmtime(unix_time_2017_01_01) > time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=0, > tm_min=0, tm_sec=0, tm_wday=6, tm_yday=1, tm_isdst=0) > > >>> time.gmtime(unix_time_2017_01_01 + 1) > time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=0, > tm_min=0, tm_sec=1, tm_wday=6, tm_yday=1, tm_isdst=0) > > >>> time.gmtime(unix_time_2017_01_01 + 2) > time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=0, > tm_min=0, tm_sec=2, tm_wday=6, tm_yday=1, tm_isdst=0) > > > > > > -- Fio Cattaneo > > Universal AC, can Entropy be reversed? -- "THERE IS AS YET > INSUFFICIENT DATA FOR A MEANINGFUL ANSWER." > > On Tue, Jan 15, 2019 at 10:01 AM jimlux <jimlux@earthlink.net> wrote: > > > > I'm working with a variety of things which work in UTC or GPS > > week/millisecond, so we're doing a lot of conversion back and forth. > > (the spacecraft puts out time in week:millisecond, all the ground > > systems processing is in UTC) > > > > The question comes up when converting back and forth, and whether > > various libraries handle leap seconds correctly. > > For now, I can use a hack of not computing back to 6 Jan 1980, but use > > an epoch like 15 Dec 2018 (week 2031: 518,400.000 seconds) and hope > > there's no leap second in the offing. > > > > > > For instance, in Python, if I do a datetime(2016,12,31,0,0,0) + > > timedelta(hours=30) does it come out as 1/1/2017 6:00:00 or 5:59:59 (it > > comes out 0600) > > > > Similarly, does Excel's time formatting allow for some minutes having an > > extra second, or does it just assume all minutes are 60 seconds. > > > > I'll probably test it for the cases I'm interested in (Ruby, Python, > > Excel, Matlab, Octave), but if someone else has already done it, then > > I've got something to cross check against. > > > > > > (python does NOT know about leap seconds) > > > > import datetime > > > > d = datetime.datetime(2016,12,31) > > > > dt = datetime.timedelta(hours=30) > > > > d > > Out[4]: datetime.datetime(2016, 12, 31, 0, 0) > > > > dt > > Out[5]: datetime.timedelta(1, 21600) > > > > d+dt > > Out[6]: datetime.datetime(2017, 1, 1, 6, 0) > > > > _______________________________________________ > > time-nuts mailing list -- time-nuts@lists.febo.com > > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > > and follow the instructions there.

Hi Welcome to yet another chapter in the very long book “why leap seconds are bad”. There are a lot of posts going into may of the ways they messes up code and updates. Keeping a library up to date with a leap second that might be a few months out …. yikes ….. Recompiling all your code (if it’s all compiled) in time to incorporate the update …. even more “interesting”. Trusting your code to wander out on the internet to grab live information …. I can’t think of *any* way that could go wrong …. Bob > On Jan 15, 2019, at 3:33 PM, Fiorenzo Cattaneo <fio@cattaneo.us> wrote: > > There are few libraries which offer time unadjusted for leap seconds, > like TAI time: > https://en.cppreference.com/w/cpp/chrono/tai_clock > > Some other libraries (typically in written in languages which very few > people use like haskell ) offer leapsecond support, but the > fundamental problem (as mentioned in the writeup) POSIX requires a day > length to be 86400 *AND* requires to be UTC. These two requirements > are fundamentally at odds with each other, which is why it's so > complicated to deal with this in general purpose code. > > > > > -- Fio Cattaneo > > Universal AC, can Entropy be reversed? -- "THERE IS AS YET > INSUFFICIENT DATA FOR A MEANINGFUL ANSWER." > > On Tue, Jan 15, 2019 at 12:25 PM Fiorenzo Cattaneo <fio@cattaneo.us> wrote: >> >> I double check the Python code, and I can confirm it does not take >> LEAP seconds into account. I highly doubt you will find standard time >> libraries for the most common languages which will deal with LEAP >> seconds. They would rather just ignore it and have one less of a >> headache to worry about (remember all the bugs that pop up even when >> we switch in and out of DST, like applications crashing because NTP >> applies the 1 hour change in a discontinous manner, as well as iphone >> alarms not working when the DST date is modified?). >> >> The C++ boost time library docs seem to imply it's possible to use >> them and have them account for LEAP seconds: >> https://www.boost.org/doc/libs/1_58_0/doc/html/date_time.html >> >> What is worse is tha the POSIX standard requires a day to be 86400 >> seconds, and pretty much all the code which relies on POSIX makes such >> assumption, in fact in my experience the day length is usually >> hardcoded to 86400 seconds: >> https://www.ucolick.org/~sla/leapsecs/right+gps.html >> >> ================================= >> >> >> fcattane@linux-mint-64:~$ python >> Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34) >> [GCC 7.3.0] on linux2 >> Type "help", "copyright", "credits" or "license" for more information. >>>>> import time >>>>> import datetime >>>>> >>>>> # last leap second got added on 1 Jan 2017 >> ... # >> ... >>>>> unix_time_2017_01_01 = 1483228800 >>>>> >>>>> >>>>> >>>>> unix_time_2017_01_01 >> 1483228800 >>>>> >> >> >>>>> time.gmtime(unix_time_2017_01_01 - 2) >> time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=23, >> tm_min=59, tm_sec=58, tm_wday=5, tm_yday=366, tm_isdst=0) >> >>>>> time.gmtime(unix_time_2017_01_01 - 1) >> time.struct_time(tm_year=2016, tm_mon=12, tm_mday=31, tm_hour=23, >> tm_min=59, tm_sec=59, tm_wday=5, tm_yday=366, tm_isdst=0) >> >>>>> time.gmtime(unix_time_2017_01_01) >> time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=0, >> tm_min=0, tm_sec=0, tm_wday=6, tm_yday=1, tm_isdst=0) >> >>>>> time.gmtime(unix_time_2017_01_01 + 1) >> time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=0, >> tm_min=0, tm_sec=1, tm_wday=6, tm_yday=1, tm_isdst=0) >> >>>>> time.gmtime(unix_time_2017_01_01 + 2) >> time.struct_time(tm_year=2017, tm_mon=1, tm_mday=1, tm_hour=0, >> tm_min=0, tm_sec=2, tm_wday=6, tm_yday=1, tm_isdst=0) >> >> >> >> >> >> -- Fio Cattaneo >> >> Universal AC, can Entropy be reversed? -- "THERE IS AS YET >> INSUFFICIENT DATA FOR A MEANINGFUL ANSWER." >> >> On Tue, Jan 15, 2019 at 10:01 AM jimlux <jimlux@earthlink.net> wrote: >>> >>> I'm working with a variety of things which work in UTC or GPS >>> week/millisecond, so we're doing a lot of conversion back and forth. >>> (the spacecraft puts out time in week:millisecond, all the ground >>> systems processing is in UTC) >>> >>> The question comes up when converting back and forth, and whether >>> various libraries handle leap seconds correctly. >>> For now, I can use a hack of not computing back to 6 Jan 1980, but use >>> an epoch like 15 Dec 2018 (week 2031: 518,400.000 seconds) and hope >>> there's no leap second in the offing. >>> >>> >>> For instance, in Python, if I do a datetime(2016,12,31,0,0,0) + >>> timedelta(hours=30) does it come out as 1/1/2017 6:00:00 or 5:59:59 (it >>> comes out 0600) >>> >>> Similarly, does Excel's time formatting allow for some minutes having an >>> extra second, or does it just assume all minutes are 60 seconds. >>> >>> I'll probably test it for the cases I'm interested in (Ruby, Python, >>> Excel, Matlab, Octave), but if someone else has already done it, then >>> I've got something to cross check against. >>> >>> >>> (python does NOT know about leap seconds) >>> >>> import datetime >>> >>> d = datetime.datetime(2016,12,31) >>> >>> dt = datetime.timedelta(hours=30) >>> >>> d >>> Out[4]: datetime.datetime(2016, 12, 31, 0, 0) >>> >>> dt >>> Out[5]: datetime.timedelta(1, 21600) >>> >>> d+dt >>> Out[6]: datetime.datetime(2017, 1, 1, 6, 0) >>> >>> _______________________________________________ >>> time-nuts mailing list -- time-nuts@lists.febo.com >>> To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com >>> and follow the instructions there. > > _______________________________________________ > time-nuts mailing list -- time-nuts@lists.febo.com > To unsubscribe, go to http://lists.febo.com/mailman/listinfo/time-nuts_lists.febo.com > and follow the instructions there.

On Wed, Jan 16, 2019 at 2:01 AM Fiorenzo Cattaneo <fio@cattaneo.us> wrote: > headache to worry about (remember all the bugs that pop up even when > we switch in and out of DST, like applications crashing because NTP > applies the 1 hour change in a discontinous manner NTP does no such thing. NTP's timescale is in 136-year eras that begin on 1900-01-01 (meaning that it rolls in 2036, which will make an interesting dry run for the UNIX epoch rollover in 2038). As the author says, ``the NTP timescale [...] knows nothing about days, years or centuries, only the seconds since the beginning of the current era which began on 1 January 1900. '' (http://doc.ntp.org/4.1.2/leap.htm). Any DST adjustment is done by the OS where NTP is running. NTP does know leap seconds even if the OS where the daemon is running gets confused. Always fun to watch the clock strike 23:59:60. -- Peter Laws | N5UWY | plaws plaws net | Travel by Train!

You are right sorry I said "DST change" while I really meant "leap seconds" adjustments - apologies for the confusion. There is nothing wrong per with NTP in applying leap seconds adjustments in a single 1 step of course, but there were several outages due to that during the last leap second adjustment, due to applications not being able to deal with the leap second change. Which is why google decided to apply the 1 second adjustment smeared over 24 hours : https://developers.google.com/time/smear As I mentioned to the other commenter, this goes to show how horrible is POSIX code at dealing with leap seconds (in fact, by definition it cannot. POSIX defines a day to always be 86400 exactly, so POSIX code never really deals with leap seconds). Which implies that most programmers of time libraries are not really concerned about it, and I'd do a lot of extra testing before using any library which claims to be leap-second compliant. -- Fio Cattaneo Universal AC, can Entropy be reversed? -- "THERE IS AS YET INSUFFICIENT DATA FOR A MEANINGFUL ANSWER." On Wed, Jan 16, 2019 at 7:00 AM Peter Laws <plaws0@gmail.com> wrote: > > On Wed, Jan 16, 2019 at 2:01 AM Fiorenzo Cattaneo <fio@cattaneo.us> wrote: > > > headache to worry about (remember all the bugs that pop up even when > > we switch in and out of DST, like applications crashing because NTP > > applies the 1 hour change in a discontinous manner > > > NTP does no such thing. NTP's timescale is in 136-year eras that > begin on 1900-01-01 (meaning that it rolls in 2036, which will make an > interesting dry run for the UNIX epoch rollover in 2038). As the > author says, ``the NTP timescale [...] knows nothing about days, years > or centuries, only the seconds since the beginning of the current era > which began on 1 January 1900. '' (http://doc.ntp.org/4.1.2/leap.htm). > > Any DST adjustment is done by the OS where NTP is running. NTP does > know leap seconds even if the OS where the daemon is running gets > confused. Always fun to watch the clock strike 23:59:60. > > > -- > Peter Laws | N5UWY | plaws plaws net | Travel by Train!