Bug 2917 – std.date fails for all years before 1970

For years prior to 1970, conversion from string to d_time returns d_time_nan. Strings are parsed correctly, but conversion to d_time fails the test at line 440 of date.d in the makeDay() function. Commenting out lines 440 through 445 allows a d_time to be returned, but the value is skewed. Line 447 works for years >= 1970: return day(t) + date - 1; but for years < 1970, this adjustment: return day(t) + date; returns the expected value. There is also another problem: attempting to convert a negative d_time back to a string requires adding ticksPerSecond to d_time to return the original value from secFromTime(), which, of course, trashes the ms value. Adding the value of ticksPerSecond appears to work for all negative d_time values, except for -1000. NOTE: I was only working with a resolution of seconds, so the effect on milliseconds/ticks was not taken into account. I'm new to D, so please don't be too critical of my code. I'm including it so you can see the kludges I added to work around these issues. BTW, an addMonths() function (more elegant than mine) would be a welcome addition to std.date. My attempt at writing one along with a unittest section was what revealed this bug. Unittest is a brilliant addition to the language. import std.stdio, std.date, std.conv, std.random; int main(char[][] args) { bool pass; int adj; string tscmp; d_time tt, rtt; string[] testdates = [ "1969-12-31 23:59:58", "1969-12-31 23:59:59", "1970-01-01 00:00:00", "1970-01-01 00:00:01", "1970-01-01 00:00:02" ]; writefln("Crossing the 1970 epoch...\n"); foreach(int i, string s; testdates) { tt = dbgParse(s ~ " GMT"); // KLUDGE adj = (i < 2) ? ticksPerSecond - 5: 0; rtt = tt + adj; tscmp = toDbDateTime(rtt); pass = (tscmp == s); writefln("%s -> %5d", s, tt); writefln("%s <- %5d %s\n", tscmp, rtt, pass); } // Random tests auto gen = Mt19937(unpredictableSeed); int yr, mh, dy, hr, me, sd, days; int yr2, mh2, dy2, hr2, me2, sd2; int tested, passed, before1970, show = 7; string ts; bool first; for (int i; i < 10000; i++) { tested++; yr = uniform(1601, 2400, gen); if (yr < 1970) before1970++; mh = uniform(1, 12, gen); days = daysInMonth(yr, mh); dy = uniform(1, days, gen); hr = uniform(0, 23, gen); me = uniform(0, 59, gen); sd = uniform(0, 59, gen); ts = std.string.format("%d-%02d-%02d %02d:%02d:%02d", yr, mh, dy, hr, me, sd); tt = dbgParse(ts ~ " GMT"); // KLUDGE adj = (yr < 1970) ? ticksPerSecond - 5: 0; rtt = tt + adj; tscmp = toDbDateTime(rtt); pass = (ts == tscmp); if (! pass || show > 0) { if (! first) { first = true; writefln("Showing first %d random samples...\n", show); } show--; writefln("%s -> %16d", ts, tt); writefln("%s <- %16d %s\n", tscmp, rtt, pass); } if (pass) passed++; } writefln(""); writefln("Tested: %7d", tested); writefln("< 1970: %7d", before1970); writefln("Passed: %7d", passed); return 0; } string toDbDateTime(d_time t) { auto yr = yearFromTime(t); auto mh = monthFromTime(t) + 1; auto dy = dateFromTime(t); auto hr = hourFromTime(t); auto me = minFromTime(t); auto sd = secFromTime(t); return std.string.format("%d-%02d-%02d %02d:%02d:%02d", yr, mh, dy, hr, me, sd); } d_time dbgParse(string s) { try { Date dp; dp.parse(s); auto time = makeTime(dp.hour, dp.minute, dp.second, dp.ms); if (dp.tzcorrection == int.min) time -= localTZA; else { time += cast(d_time)(dp.tzcorrection / 100) * msPerHour + cast(d_time)(dp.tzcorrection % 100) * msPerMinute; } auto day = dbgMakeDay(dp.year, dp.month - 1, dp.day); d_time result = makeDate(day,time); return timeClip(result); } catch { return d_time_nan; // erroneous date string } } d_time dbgMakeDay(d_time year, d_time month, d_time date) { int[12] mdays = [ 0,31,59,90,120,151,181,212,243,273,304,334 ]; const y = cast(int)(year + floor(month, 12)); const m = dmod(month, 12); const leap = leapYear(y); auto t = timeFromYear(y) + cast(d_time) mdays[m] * msPerDay; if (leap && month >= 2) t += msPerDay; /* DISABLED: this test failes for y < 1970 if (yearFromTime(t) != y || monthFromTime(t) != m || dateFromTime(t) != 1) { writefln("Bad match"); writefln("%d : %d, %d : %d, %d : %d", y, yearFromTime(t), m, monthFromTime(t), 1, dateFromTime(t)); return d_time_nan; } */ // KLUDGE int adj = (y < 1970) ? 0 : 1; return day(t) + date - adj; } d_time addMonths(d_time dt, int months) { int days = 0; if (months <> 0) { int mon = monthFromTime(dt); // zero-based month int yr = yearFromTime(dt); while (months > 0) { days += daysInMonth(yr, mon + 1); // one-based month mon++; if (mon > 11) { mon = 0; yr++; } months--; } while (months < 0) { mon--; if (mon < 0) { mon = 11; yr--; } days -= daysInMonth(yr, mon + 1); // one-based month months++; } } return dt += days * cast(d_time) ticksPerDay; } unittest { d_time dt1, dt2, dt3, dt4; dt1 = dbgParse("1979-10-31 13:14:15 GMT"); dt2 = dbgParse("1981-05-31 13:14:15 GMT"); dt3 = addMonths(dt1, 19); assert(dt3 == dt2, text(dt3, " != ", dt2)); dt4 = addMonths(dt3, -19); assert(dt4 == dt1, text(dt4, " != ", dt1)); dt1 = dbgParse("1969-10-31 13:14:15 GMT"); dt2 = dbgParse("1971-05-31 13:14:15 GMT"); dt3 = addMonths(dt1, 19); assert(dt3 == dt2, text(dt3, " != ", dt2)); dt4 = addMonths(dt3, -19); assert(dt4 == dt1, text(dt4, " != ", dt1)); dt1 = dbgParse("1599-10-31 13:14:15 GMT"); dt2 = dbgParse("1601-05-31 13:14:15 GMT"); dt3 = addMonths(dt1, 19); assert(dt3 == dt2, text(dt3, " != ", dt2)); dt4 = addMonths(dt3, -19); assert(dt4 == dt1, text(dt4, " != ", dt1)); dt1 = dbgParse("2399-10-31 13:14:15 GMT"); dt2 = dbgParse("2401-05-31 13:14:15 GMT"); dt3 = addMonths(dt1, 19); assert(dt3 == dt2, text(dt3, " != ", dt2)); dt4 = addMonths(dt3, -19); assert(dt4 == dt1, text(dt4, " != ", dt1)); }

Created attachment 348 patch for std.date.d v. 2.029 fixes issues with negative time values Rather than whine about the date issues, I decided to lend a hand and address the issues myself. If you find the modifications useful you are welcome to use the code in any way you see fit. I only added my name as a modifier so that Walter won't get blamed for any mistakes I made. You may remove my name or leave it, as you see fit. The changes I made aren't anything special; just some grunt work that needed doing. I tinkered with some of the more vital functions in this module. In particular, I modified floor(), dmod(), day(), and dayFromYear(). The latter got a little messy with year value 0 and lower. Of course, pushing the Gregorian calendar back that far back is purely hypothetical. I added unittest sections to day() and dayFromYear, as the proper working of these functions is vital to the entire module. I threw in a bit of trivial tidying up as well. The calls to toInteger() and timeClip() were removed, since they don't do anything. I left the functions in place, in case some existing code calls them. I changed the name of the 'day' variable that called the day() function, as it seems like a bad idea to duplicate the name in the same scope. I added an overload for dateFromTime(d_time t, int months). In most cases, where it's used in the module, the value for month is already known, so there's no need for dateFromTime() to recalculate it. My addition of toDbDateTime() is trivial, but I found it useful in testing, and it has practical value. It's a fairly common format, and it doesn't add much weight to the module. I also added two other functions: addMonths() and addYears(). These are both common date/time-manipulation tasks, which aren't as straightforward as adding fixed values like days or weeks. addMonths() is rather clunky, but it gets the job done. It never cranks through more than 11 months on any call. Beyond 11, it calls addYears() so that adding 60 months is almost as quick as adding 5 years. I've rebuilt the 2.029 library with this patch to std.date on linux. All appears to be working well. Someone needs to test the DosDate functions to make sure my changes didn't break them. There were no unit tests for them. I don't have D installed on a Windows box. Thanks in advance for considering these changes.

Created attachment 366 patch for std.date.d v. 2.030 fixes issues with negative time values

This bit of code shows the bug. There are larger effects as well. ------------------------------------------- string[] testdt = [ "1969-12-31 23:59:58 UTC", "1969-12-31 23:59:59 UTC", "1970-01-01 00:00:00 UTC", // epoch "1970-01-01 00:00:01 UTC", "1970-01-01 00:00:02 UTC"]; foreach(int i, string s; testdt) writefln("%s = %5d", s, parse(s)); d_time[] times = [-2000,-1000,0,1000,2000]; foreach(tm; times) writefln("%5d = %s", tm, toUTCString(tm)); ------------------------------------------- Unpatched results: 1969-12-31 23:59:58 UTC = -9223372036854775808 (d_time_nan) 1969-12-31 23:59:59 UTC = -9223372036854775808 (d_time_nan) 1970-01-01 00:00:00 UTC = 0 1970-01-01 00:00:01 UTC = 1000 1970-01-01 00:00:02 UTC = 2000 -2000 = Wed, 31 Dec 1969 23:59:57 UTC (off by 1 sec) -1000 = Wed, 31 Dec 1969 23:59:58 UTC (off by 1 sec) 0 = Thu, 01 Jan 1970 00:00:00 UTC 1000 = Thu, 01 Jan 1970 00:00:01 UTC 2000 = Thu, 01 Jan 1970 00:00:02 UTC Patched results: 1969-12-31 23:59:58 UTC = -2000 1969-12-31 23:59:59 UTC = -1000 1970-01-01 00:00:00 UTC = 0 1970-01-01 00:00:01 UTC = 1000 1970-01-01 00:00:02 UTC = 2000 -2000 = Wed, 31 Dec 1969 23:59:58 UTC -1000 = Wed, 31 Dec 1969 23:59:59 UTC 0 = Thu, 01 Jan 1970 00:00:00 UTC 1000 = Thu, 01 Jan 1970 00:00:01 UTC 2000 = Thu, 01 Jan 1970 00:00:02 UTC

Created attachment 555 patch for std.date.d v.2.039 fixes issues with negative time values

Created attachment 556 date_assert.d contains assertions for date.d v2.039 patch date_assert.d contains test cases, both to demonstrate what's wrong with std.date and to validate modifications by the supplied patch. Please note that this file and the provided patch were created on Linux and contain UNIX-style line endings. In order to compile date_assert.d with v2.039 phobos before the patch is applied, you'll need to comment out lines 68 through 84. These lines contain tests that call addMonths(), which is not available before the patch is applied. You can search for "fails w/o patch" to identify the assert() tests that fail with out the patch.

This bug report is symptomatic for what's wrong with the D development process, in my opinion. There's a bug, there's a fix, there's a patch, there's a distinct lack of feedback from Walter/Andrei. No wonder so few people work on D; a situation like this would drive any contributor away. My apologies to Mr. Haecker.

std.date has now been superseded by std.datetime.