'Managing' [List] Separator Anxiety

Now I have praised the folks in GIFT Ireland in the past, like in this post and also this one, for example.

But when you are not talking to someone every day, you sometimes forget they are there. Luckily, sometimes they find a way to remind you that makes you really happy about their presence. This post is an example of that....

The other day, John Caffrey (an SDET across the puddle) sent the following in mail:

Hi guys,

Quick question if that’s ok… so during the testing of Locale Builder, we’ve noticed that the CARIB ignores the list separator data when loading it from an LDML file (for a replacement locale).

Is this a known issue with the CARIB? If so, can you maybe send me the relevant bug # so I can link our bug?

Thanks!

J. 

There was a brief pause because I had not seen the mail at first and both Shawn and Tarek were out that day, but by the next day I had looked at the code and was able to comment:

This is an unintentional bundling of a user overridable setting (the list separator) with a non overridable setting (the code pages) -- no one noticed this to copy the setting over.

Worth putting a bug in and easy enough to fix, though I'm not sure where it would be triaged...

In my mind I was thinking that the code in the CultureAndRegionInfoBuilder dates back to the original stuff I had written, which meant that (a) this bug was probably my fault though (b) this at least partially mitigated by the fact that many people had modified and tested the code since and no one had noticed. :-)

Since it was not intentional, I assumed that problem would be limited to trying to set CultureAndRegionInfoBuilder.TextInfo (which fails in replacement cultures) but that setting the CultureAndRegionInfoBuilder.TextInfo's actual TextInfo.ListSeparator property should work just fine. I had not actually tried it yet or anything like that before Erich Barnstedt (a dev from the Ireland team) responded to my reply to John's mail:

Yeah, that was my hunch since it sits in TextInfo. We’ll work around it in Locale Builder by reading the value directly from LDML and re-applying it to the CARIB after loading from LDML. Thanks for the quick response,

                Erich

Things I think about in my head but don't share with others do not count as ideas, and I certainly didn't put any thought into what to do with this information, so Erich's public statement of the idea really allows him to be credited with inspiring me to write the following code that tests the theory (while Erich put the actual fix into the MS Locale Builder code):

using System;
using System.Globalization;

namespace Testing {
  class oooo {
    [STAThread]
    static void Main(string[] args) {
      CultureInfo ci;
      string stCulture;

      // First figure out the name
      if(args.Length > 0) {
        stCulture = args[0];
      } else {
        stCulture = CultureInfo.CurrentCulture.Name;
      }

      // Create the culture and say what it is
      ci = new CultureInfo(stCulture, false);
      Console.WriteLine("\r\nUsing the following culture: '{0}' ({1})\r\n", ci.DisplayName, ci.Name);

      // Create the replacement and fill it
      CultureAndRegionInfoBuilder carib = new CultureAndRegionInfoBuilder(stCulture, CultureAndRegionModifiers.Replacement);
      carib.LoadDataFromCultureInfo(ci);
      carib.LoadDataFromRegionInfo(new RegionInfo(stCulture));

      // Change the list separator as appropriate
      if(args.Length > 1) {
        carib.TextInfo.ListSeparator = args[1];
      } else {
        carib.TextInfo.ListSeparator = ";";
      }

      // Return some status
      Console.WriteLine("List separator value");
      Console.WriteLine("  ...in the old ci: '{0}'", ci.TextInfo.ListSeparator);
      Console.WriteLine("  ...in the carib:  '{0}'", carib.TextInfo.ListSeparator);

      // Register the replacement
      carib.Register();

      // More status
      ci = new CultureInfo(stCulture, false);
      Console.WriteLine("  ...in the new ci: '{0}'", ci.TextInfo.ListSeparator);
      ci = null;

      // Unregister -- cleanup is important in samples
      CultureAndRegionInfoBuilder.Unregister(stCulture);

      // Even more status
      ci = new CultureInfo(stCulture, false);
      Console.WriteLine("  ...after removal: '{0}'", ci.TextInfo.ListSeparator);
    }
  }
}

So if you have .NET 2.0 on your machine you can stick this code into a file named carib.cs and compiled it from the .NET command line as follows:

 csc carib.cs /r:sysglobl.dll

The command line syntax is easy enough:

 carib.exe - uses the current culture and changes the separator to a semicolon

 carib.exe <culturename> - uses the specified culture and changes the separator to a semicolon

 carib.exe <culturename> <separator> - uses the specified culture and changes the separator to what is specified 

On my machine (which is set to a default user locale of en-AU to look at issues that this guy has been talking about!):

e:\test>csc carib.cs /r:sysglobl.dll
Microsoft (R) Visual C# 2005 Compiler version 8.00.50727.42
for Microsoft (R) Windows (R) 2005 Framework version 2.0.50727
Copyright (C) Microsoft Corporation 2001-2005. All rights reserved.

e:\test>carib.exe

Using the following culture: 'English (Australia)' (en-AU)

List separator value
  ...in the old ci: ','
  ...in the carib:  ';'
  ...in the new ci: ';'
  ...after removal: ','

e:\test>

So there you have it.... an easy workaround for the oversight about the list separator (which at this point pretty much everyone wishes were just a part of CultureInfo rather than TextInfo anyway, since the latter is supposed to be about the writing system and the list separator is defnitely more of a cultural preference!). And in the meantime it will be addressed in the final release of the Locale Builder when it goes out.

Special thanks to both John and Erich for not only finding my bug but also thinking ahead to the fix and making the Locale Builder behave more intuitively! :-)

This post brought to you by  ,  (U+002c, a.k.a. COMMA)