by Michael S. Kaplan, published on 2008/11/26 03:01 -05:00, original URI: http://blogs.msdn.com/b/michkap/archive/2008/11/26/9143136.aspx
I was asked the other day what I thought about string comparisons. What with all the different recommendations floating around, all of the different possibilities, and all the unclear guidelines, the question was whether there was a succinct way to describe the best way to know what to do?
Obviously this is an area that does not inspire intuitive thought. That train has already left the station, but not before the conductor sneered at us for presuming we would have seats on it.
But is there a way to build in something to pinch hit for intuitive behavior?
I think there is. :-)
Let me try it on for size here....
There are four points to keep in mind, which I will describe now.
The second point is another test -- one has to decide how one would feel if characters that look kind of the same even though they aren't were to be sorted together -- either as being identical or being right next to each other. If the answer is Yes, Michael, I think it would be great if stuff that looks th same got treated the same then this is a linguistic comparison, rather than a binary/ordinal/lexicographic type one.
The third point is yet another test -- one has to decide how one feels about characters that have no weight in sorting, either because they are unassigned anywhere, not yet assigned in the tables, or intentionally are given no weight. If the answer is Yes, Michael, if no one thought the character should be given weight then I don't see why I would want to then this is once again a linguistic comparison, rather than a binary/ordinal/lexicographic type one.
The fourth point is something to keep in mind -- and that is the fact that one should know exactly what the comparison is for, and especially that the same data might be sorted different ways in different situations. Only by knowing the scenario can one make a correct decision.
Perhaps examples would help here. :-)
Think of the first test as the Turkic I test -- whether one is okay with the way one looks at
I İ ı i
and the fact that sometimes the first and fourth items there are the casing pair, and other times it is the first/third and second/fourth that are. You can imagine plugging those strings right into your code and deciding how you feel about the results.
Think of the second test as the Small Capital Y test -- whether one is okay with the following three characters
Y y ʏ
being right next to each other and sometimes even considered equal to each other. You can once again imagine pluging those strings right into your code and deciding how you feel about the results.
And you can think of the third test as the IE Grave test (also known as the "Cyrillic E" test by the stubborn) -- whether one is okay with letters like the following:
Ѐ ѐ
being treated as if they weren't there at all, because some versions of Microsoft products will do just that. You can once again imagine plugging those strings right into your code and deciding how you feel about the results.
That fourth point is the easiest of all. Just imagine components like NTFS (the file system) or the registry. At the lowest levels they happen to use an ordinal/binary kind of collation, while at the most user visible levels (e.g. Windows Explorer, RegEdit) they use a linguistic one. The rule is that circumstances alter cases, so that even the same list can sort differently in different situations and that is okay.
So, three simple mental tests one can do and immediately come up with an answer on what comparison method to use....
Easy!
This blog brought to you, as you likely could have guessed, by the above nine characters
# Chip H. on 1 Dec 2008 1:09 PM:
I've been reading your blog for years Michael, and something that has struck me (and you, it sounds like, even if you've never directly expressed it) is that Microsoft's own products behave differently, with regards to Unicode.
I understand that Unicode is a moving target, but even products released close together in time do things quite differently.
What this tells me is that each product team is reinventing the wheel. Or at least embedding the code needed to deal with I18-related issues into their codebase and then not sharing it with other teams at Microsoft. I'm sure there's some emails being sent back & forth, but an oral history isn't the best way to do this.
Is there an effort underway to unify these disparate codebases and place it in a library that can be used by both the operating system folks, the application developers, AND 3rd party developers? Chock-full of canonical examples of the right way to do things?
# Michael S. Kaplan on 2 Dec 2008 2:54 AM:
Wow, that is a question worthy of (at a minimum) its own blog, if not its own Blog! :-)