Reporting one casualty in the operation; luckily it was the stupidest member of the unit

by Michael S. Kaplan, published on 2011/03/16 07:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2011/03/16/10141835.aspx


Breaking changes suck.

Even when the thing broken is of limited use and serves limited purpose.

The suckage still exists.

Anyway, the report was a clear enough indictment:

In .NET 2 it used to be that for neutral cultures CultureInfo.KeyboardLayoutId would return some specific culture.
For example, new GetCultureInfo(“en”). KeyboardLayoutId would be 1033, while LCID is 9. Same for other neutral cultures.
Not anymore in .NET 4: KeyboardLayoutId is 9 in the case above.

I cannot find any information regarding the reasons for this change. The article http://msdn.microsoft.com/en-us/netframework/dd890508.aspx claims the opposite of what I observed, saying that now we’ll get specific values instead of neutral. In reality, we used to get that with .NET 2, now we get neutral values for everything.

Being more specific!

Previous releases of the .NET Framework throw an exception if applications try to access some of the neutral culture properties such as the CultureInfo.DateTimeFormat.FirstDayOfWeek property. In .NET Framework 4, all neutral culture properties will return values which will come from the specific culture which is most dominant for that neutral culture. For example, French neutral locale will retrieve the values of most of its properties from French (France). The CultureInfo.DateTimeFormat.FirstDayOfWeek property would return Monday for French which maps to the value in the French (France) culture.
Some properties will be an exception to this rule where they will have different values from the dominant culture properties such as the language name. For example, the language name of the Norwegian neutral culture is Norwegian while the language name of the specific culture of Norwegian, Bokmål (Norway) is Norwegian (Bokmål).
Some properties and methods of neutral cultures will return specific cultures instead of the neutral cultures such as KeyboardLayoutId property and GetConsoleFallbackUICulture method in CultureInfo class.

Culture Name v3.5 KeyboardLayoutId v4.0 KeyboardLayoutId
ar 1 1025
es 10 1034
fr 12 1036
zh-CHS 4 2052

Every word of this description is true.

And yes this technically constitutes a breaking change.

Long time readers or web searchers of the KeyboardLayoutId may have run across probably the only useful historical discussion of this property, my blog What the #$!*& is a KeyboardLayoutId, anyway?.

The whole point of this property is to provide a "better than complete and abject failure" case for the InputLanguage.FromCulture method.

So by that metric, the .Net 4.0 regression is clearly not so very good.

From .Net Reflector, looking at the 4.0 framework:

public static InputLanguage FromCulture(CultureInfo culture)
{
    int keyboardLayoutId = culture.KeyboardLayoutId;
    foreach (InputLanguage language in InstalledInputLanguages)
    {
        if ((((int) ((long) language.handle)) & 0xffff) == keyboardLayoutId)
        {
            return language;
        }
    }
    return null;
}

Okay, that code will surely fail on all neutrals that the code used to succeed on.

Regression.

Now as to why, remember in my older blogs about neutrals from years ago, like What is a neutral culture? What is a neutral locale?, where I've talked about so many of the problems of neutrals and how they were implemented.

In Windows 7, the NLS team worked to fix many of the issues here, such as:

and more, and they did a bunch of work.

As mentioned in Mapping Locale Data:

List Neutral Locales

To enumerate neutral locales for Windows 7 and later, your application can call EnumSystemLocalesEx with dwFlags set to LOCALE_NEUTRALDATA. It can also use GetLocaleInfoEx with LCType set to LOCALE_INEUTRAL.

So they added a way to query for neutrals and get the actual neutrals rather than the defaults -- it is an opt-in flag.

And they added a bunch of script specific neutrals so that each multi-script language would have such a neutral to fall back on.

And then, as a part of syncing back up the managed cultures and the native locales, they gutted the old neutrals and made the new .Net 4.0 neutral cultures depend on this new native locales support.

In the process of this, they broke the KeyboardLayoutId for neutrals.

Thus the blog's title, because this blog is indeed reporting one casualty in Operation Rehabilitate Neutrals; luckily, it was the stupidest member of the unit....


John Cowan on 16 Mar 2011 7:46 AM:

Yes, but what, actually, like, *is* a neutral culture/locale?  You give examples, but don't explain.

Michael S. Kaplan on 16 Mar 2011 8:23 AM:

Explained long ago. :-)

The intent is to be like a "LanguageInfo", but the emphasis is always on how to extend it to a CultureInfo, anyway. From an OO standpoint, it violates some principles, I believe....


Please consider a donation to keep this archive running, maintained and free of advertising.
Donate €20 or more to receive an offline copy of the whole archive including all images.

referenced by

2011/05/27 If it's overtly over-applied and overarching, it may be an overreach (e.g. in nOrway!)

go to newer or older post, or back to index or month or day