Reporting one casualty in the operation; luckily it was the stupidest member of the unit

by Michael S. Kaplan, published on 2011/03/16 07:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2011/03/16/10141835.aspx

Breaking changes suck.

Even when the thing broken is of limited use and serves limited purpose.

The suckage still exists.

Anyway, the report was a clear enough indictment:

In .NET 2 it used to be that for neutral cultures CultureInfo.KeyboardLayoutId would return some specific culture.
For example, new GetCultureInfo(“en”). KeyboardLayoutId would be 1033, while LCID is 9. Same for other neutral cultures.
Not anymore in .NET 4: KeyboardLayoutId is 9 in the case above.

I cannot find any information regarding the reasons for this change. The article http://msdn.microsoft.com/en-us/netframework/dd890508.aspx claims the opposite of what I observed, saying that now we’ll get specific values instead of neutral. In reality, we used to get that with .NET 2, now we get neutral values for everything.

Being more specific!

Previous releases of the .NET Framework throw an exception if applications try to access some of the neutral culture properties such as the CultureInfo.DateTimeFormat.FirstDayOfWeek property. In .NET Framework 4, all neutral culture properties will return values which will come from the specific culture which is most dominant for that neutral culture. For example, French neutral locale will retrieve the values of most of its properties from French (France). The CultureInfo.DateTimeFormat.FirstDayOfWeek property would return Monday for French which maps to the value in the French (France) culture.
Some properties will be an exception to this rule where they will have different values from the dominant culture properties such as the language name. For example, the language name of the Norwegian neutral culture is Norwegian while the language name of the specific culture of Norwegian, Bokmål (Norway) is Norwegian (Bokmål).
Some properties and methods of neutral cultures will return specific cultures instead of the neutral cultures such as KeyboardLayoutId property and GetConsoleFallbackUICulture method in CultureInfo class.

Every word of this description is true.

And yes this technically constitutes a breaking change.

Long time readers or web searchers of the KeyboardLayoutId may have run across probably the only useful historical discussion of this property, my blog What the #$!*& is a KeyboardLayoutId, anyway?.

The whole point of this property is to provide a "better than complete and abject failure" case for the InputLanguage.FromCulture method.

So by that metric, the .Net 4.0 regression is clearly not so very good.

From .Net Reflector, looking at the 4.0 framework:

public static InputLanguage FromCulture(CultureInfo culture)
{
    int keyboardLayoutId = culture.KeyboardLayoutId;
    foreach (InputLanguage language in InstalledInputLanguages)
    {
        if ((((int) ((long) language.handle)) & 0xffff) == keyboardLayoutId)
        {
            return language;
        }
    }
    return null;
}

Okay, that code will surely fail on all neutrals that the code used to succeed on.

Regression.

Now as to why, remember in my older blogs about neutrals from years ago, like What is a neutral culture? What is a neutral locale?, where I've talked about so many of the problems of neutrals and how they were implemented.

In Windows 7, the NLS team worked to fix many of the issues here, such as:

• the inconsistent script problems in many built-in neutrals I talked about in CultureInfo subsetting attempts that suck
• the lack of neutrals in custom cultures I talked about in CultureInfo subsetting attempts that suck
• the way that neutrals in NLS are treated as the default locale of the neutral that I talked about in Neutral? I do not think that word means what you think it means!
• the way using neutrals didn't always lead to expected behavior I talked about in Using full locales rather than the neutral ones?

and more, and they did a bunch of work.

As mentioned in Mapping Locale Data:

List Neutral Locales

To enumerate neutral locales for Windows 7 and later, your application can call EnumSystemLocalesEx with dwFlags set to LOCALE_NEUTRALDATA. It can also use GetLocaleInfoEx with LCType set to LOCALE_INEUTRAL.

So they added a way to query for neutrals and get the actual neutrals rather than the defaults -- it is an opt-in flag.

And they added a bunch of script specific neutrals so that each multi-script language would have such a neutral to fall back on.

And then, as a part of syncing back up the managed cultures and the native locales, they gutted the old neutrals and made the new .Net 4.0 neutral cultures depend on this new native locales support.

In the process of this, they broke the KeyboardLayoutId for neutrals.

Thus the blog's title, because this blog is indeed reporting one casualty in Operation Rehabilitate Neutrals; luckily, it was the stupidest member of the unit....

John Cowan on 16 Mar 2011 7:46 AM:

Michael S. Kaplan on 16 Mar 2011 8:23 AM:

referenced by

go to newer or older post, or back to index or month or day