Being in the zone may not be the best way to get the time right

by Michael S. Kaplan, published on 2010/02/04 07:11 -05:00, original URI: http://blogs.msdn.com/b/michkap/archive/2010/02/04/9958063.aspx


So the other day a developer at Microsoft who was doing a ton of grabbing of UTC SYSTEMTIME values and logging them in the local time of the user1 noticed some significant performance regressions between Windows Server 2003 and Windows Server 2008 R2.

After doing some work to isolate where the extra time was being spent, he found that s ton of the time was spent around the following key:

HKLM\Software\Microsoft\Windows NT\CurrentVersion\Time Zones\Pacific Standard Time\Dynamic

Luckily, the original "Father Time" Geoff Pease2 was around to describe what was going on here in some detail....

You are hitting this because of Dynamic Time Zone support that was added.
 
Everything you need is NOT in a TIME_ZONE_INFORMATION structure. The structure only has room for the “current” time zone and not historical ones.
 
In 2007, the US changed the start and end dates for Daylight Savings time. Any date BEFORE the transition needs to use a different set of rules than those AFTER the transition.
 
Because of historical applications that have no clue about this change and to keep them displaying the historically accurate time stamps (meaning then do not change whichever side of the transition you are on), the system hast to play some games.
 
To get around this, the system has to play some games in matching a “current time zone” to lookup as needed historical time zone data about the year. 
 
If you want the entire server to disable historical lookups, turn off Daylight Savings Time (which you’ve already tried and hated) OR do what you did – call GetTimeZoneInformation() to get the TZI and pass it in… but then add zero’ing out the Standard and Daylight names. This will cause a time zone comparison to fail where the legacy API is trying to determine if the “current time zone” is the same time zone you just past it (and if true, the do the historical lookups which hits the registry where these tables are stored). By failing this comparison, the historical comparison’s do not kick in and the registry accesses go away.
 
This technique comes with a down side – since you are in essence “making your own TZI,” you’ll need to refresh the TZI structure once a year or once a day or whatever you like (less often, the better but not enough to make you uncomfortable J). You also need to make sure that the timestamps you are passing in are match the TZI that you pass in. Converting times from different years using this technique will generate “bad times” during the area where DST change from year to year. If you are using this for “current time logging,” this should work pretty well for you.

Now as it turns out, the developer stumbled across another workaround as well (one that also avoids the dynamic support that was added) -- basically changing code depending on SystemTimeToTzSpecificLocalTime like this:

FILETIME ftUtc;
::GetSystemTimeAsFileTime( &ftUtc );
//
// code to perform elapsed time calcs would go here...
//
SYSTEMTIME stUtc, stLocal;
::FileTimeToSystemTime( &ftUtc, &stUtc );
::SystemTimeToTzSpecificLocalTime( NULL, &stUtc, &stLocal );
// log away using the nicely structured stLocal...

and instead do code more like this:

FILETIME ftUtc, ftLocal;
::GetSystemTimeAsFileTime( &ftUtc );
//
// code to perform elapsed time calcs would go here...
//
::FileTimeToLocalFileTime( &ftUtc, &ftLocal );
SYSTEMTIME stLocal;
::FileTimeToSystemTime( &ftlocal, &stLocal );
// log away using the nicely structured stLocal...

Since FileTimeToLocalFileTime never does any of the dynamic time zone lookup work (it just uses the current time) you will get performance behavior analogous to the Server 2003 code before dynamic time zones existed.

Now both still do the static lookup and whether the extra several conversions is faster or slower than the one missed registry key hit for the non-existent time zone is probably too small to be measured in most cases....

The scenario and the solution raise two problems for me:

  1. The dynamic time zone stuff was added for a reason, so there is potentially a chance to be off by an hour if one is dealing with historical dates rather than the current time3.
  2. Taking the time to convert and log the times causes all of the logs to be in the local time, which makes both looking at the logs by people in other time zones and comparing values across the possible range of values of successive years harder to do with complete accuracy since the original source is now lost without going back with dynamic time zone information later; leaving all times in UTC and only converting for display purposes in an event log if and when it is ever needed4 is really always going to be faster.

Similar to that time that Opera was calling SetLocaleInfo tons of times before they fixed the problem (discussed here), any time one is doing something repeatedly, asking honest questions about how much work one has to with each iteration is not an unreasonable approach to optimization....

 

1 - Actually, what he was doing/why he was doing it was a bit more complicated than this but the details are not entirely relevant to this blog; the description above is close enough!

2 - Title explained previously in You want to change the time zone? Eto Akta Gamat!

3 - This may not be relevant if only the current time at the time the value is converted is being logged, i.e. the data is never historical when it is being converted.

4 - And no offense, but to be perfectly honest I doubt this is ever needed; the number of people who read event log details that are uncomfortable with UTC can all fit into one minivan and still have room for the cooler full of beer they would be taking along with them!


Jim DeLaHunt on 4 Feb 2010 9:38 AM:

Interesting. I wrote a bit about the philosophy behind this problem on my blog, in a post "Times Change, or, We Live In Complex Times" at http://blog.jdlh.com/en/2008/03/24/times-change/.  It's tempting to have simple abstractions for things like time zones, but that conflicts with the messy reality of how humans measure and name times.

While it's great that Windows now can handle the complexity of time zone rules that change over time, there are execution-time costs, and not every user is willing to pay those.  

parkrrrr on 4 Feb 2010 10:33 AM:

As I read this and Jim DeLaHunt's post on time zones, I suddenly came to the realization that my computer is probably set to the wrong time zone, so I went and checked. And I think I may have found something that slipped through the cracks.

You see, I live in Indiana, where we (stupidly, in my opinion) started observing DST in 2006. Before then, we did not observe DST. So, when I set up my new Windows 7 machine, I set it to the Eastern (UTC-05:00) time zone. Reading this post made me question whether I should actually be in the "Indiana (East)" time zone, since setting myself to the Eastern time zone loses that historical information and causes times before 2006 to potentially be wrong by an hour.

However, it appears that the Indiana (East) time zone, as included with Windows 7, still doesn't observe DST! So, as near as I can tell, there simply isn't a "right" time zone for most of this state included with Windows.

parkrrrr on 4 Feb 2010 10:37 AM:

Also, more in line with your usual post, what's that PUA character in Geoff Pease's quoted text supposed to be?

John Cowan on 4 Feb 2010 11:01 AM:

It's certainly better to keep logs and other records in UTC for consistency, but it's also often necessary to discover what local time corresponded to a given past UTC time at a specified location, because of the possible legal consequences  -- was the stock market open or closed, was an event after the close of business, and so on.  You can't in principle map future UTC times to local times, because you do not know how the politicians will change the time-zone mappings in the future, so you should store those as local times with an indication of the time zone (as is done in things like option exercise, where you are told that the option must be exercised by "5 PM New York time" or the like).

Sensible folks, therefore (including the maintainers of every OS other than Windows, and of Java and PHP even on Windows) use the zoneinfo time package, which contains very full and regularly maintained historical time-zone information, reflecting not only changes in DST but also secular changes in time zone.  For example, different parts of Alaska used to observer UTC-8, UTC-9, UTC-10, and UTC-11, but as of 1983 it all became UTC-9 except for some of the Aleutians, which went to UTC-10.  (This has some odd consequences: on the shortest day of the year in Nome, the sun actually rises in the afternoon!)  Similarly, Hawaii switched from UTC-10:30 to UTC-10 in 1947.

You can get the current C source for the library and the historical TZ data at ftp://elsie.nci.nih.gov/pub .

parkrrrr on 4 Feb 2010 11:08 AM:

(Never mind; figured it out for myself. It's a Wingdings smiley face. Smiley faces, of course, have their own codepoint but this one is encoded as PUA because Wingdings is a symbol font.)

Michael S. Kaplan on 4 Feb 2010 2:59 PM:

Okay, I just converted it to the other (non-PUA) Wingdings smiley face!


go to newer or older post, or back to index or month or day