If we sorted Bopomofo like we do Pinyin, would it still be considered "Traditional" Chinese?

by Michael S. Kaplan, published on 2008/03/17 10:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2008/03/17/8273915.aspx

Pronunciation based sorting for Traditional Chinese is commonly requested from people, and the requests fall within two broad categories:

Cantonese pronunciations -- The requests have been for Pinyin style orderings, based on the Cantonese pronunciations for the ideographs. As I mentioned in The Cantonese IME, however, there is no accepted single transliteration standard to map ideographs to Cantonese pronunciations, and accepting schemes like Jyutping would leave the system only of use to some people, and it is difficult to assess whether the benefits would outweigh the costs.

Mandarin pronunciations -- Again the requests have been for a Pinyin style ordering, but of pronunciations of Traditional Chinese ideographs. The data for this also does not exist (we only currently have Bopomofo pronunciation data for ~48566 ideographs, based on a Taiwanese standard that could be missing ideographs used in Hong Kong and Macao. And as I mentioned in Is it Macau or is it Macao?, it is unclear how close to expected results the pronunciations (provided by China are for the 62,289 ideographs they covered) in these other markets. Mandarin is Mandarin, obviously -- but it is unlikely that there is enough in the way of standards here to guide what the expected pronunciations to be.

Given the problems that seem to exist for stroke-based sorts (ref: How bad does it need to be in order to be not good enough, anyway?), perhaps I can be forgiven my skepticism for the ability of the PRC-provided Mandarin pronunciations to match what they would expect in Taiwan or Macao....

There are interestingly even some people in Taiwan who have expressed interest in a Mandarin-style ordering. This is theoretically easier to do by mapping Bopomofo to Pinyin if one can come up with agreed upon ways to do that mapping, e.g. taking the first twelve entries in the Bopomofo table which all have a Bopomofo pronunciation of ㄅㄚ (BOPOMOFO LETTER B + BOPOMOFO LETTER A), with the PRC-provided Pinyin provided for reference and to prove that using the data as i may not match expectations:

Now obviously one could take all 48,566 ideographs and use this information to produce Pinyin-esque letters, and if one goes further down the table other Bopomofo is there, including many that include tone marks:

In the table of data, the first tone is never included, so mapping as follows is easy enough:

So at least in Taiwan the resultant Pinyin-esque sort might be just what people are looking for, and whether that matches what other people expect would have to be determined to see if it would be a useful ordering in Macao or Hong Kong or parts of China where Traditional Chinese is preferred.

The overall problem of applicability in Hong Kong and Macao is really just another piece of the same puzzle that came up before -- without information it is hard to fathom how relevant the data would be.

Plus the political issues inherent with providing a Pinyin-esque sort for Taiwan because even with some people thinking it a good idea there may be just as many who could fear the long term consequences of such a thing, not to mention that it would bring differences between the PRC data and the Taiwan data into much sharper focus!

I think it might be interesting to have some people in Macao and Hong Kong look at all of the differences between the PRC Pinyin data and the transformed Bopomofo (such as the four examples I gave) and determine which one they would expect to be the most common pronunciation. If their expectations veered toward the Taiwan choice then a sensible Traditional Chinese Pinyin pronunciation could emerge for any/all countries using Traditional Chinese, though the characters needed for Mandarin (if any) in those other countries would probably also have to be added, too....

At this point I remind myself that I don't work on collation in Windows anymore, which does make pie-in-the-sky speculations thinking about future version support a less than useful endeavor....

It is fun to think about potential solutions to the problems from time to time, though.

This blog brought to you by ㄅㄆㄇㄈ (U+3105 U+3106 U+3107 U+3108, aka BOPOMOFO LETTERS B P M F)

I'd be more inclined to trust Bopomofo pronunciation data than any other kind, for two reasons: (a) Bopomofo is widely used in education, where people care about orthoepy, and (b) it has never been politicized -- nobody has ever suggested replacing hanzi with Bopomofo, and all the Chinese-speaking governments agree on what the letters mean and how they are to be used.

In addition, Bopomofo is a complete and unambiguous representation of the phonemic distinctions of Mandarin, so you can mechanically transliterate it to correct hanyu pinyin, tongyong pinyin, Wade-Giles, Gwoyeu Romatzyh, Yale Mandarin, or whatever.

The part that is still politicizable (or at least variable between markets and regions) is when you look at Han with multiple possible pronunciations, choosing the most common one.

Well, that and the fact that if one moves to something Pinyin-esque, one has to choose which of those systems to use! :-)

Even though "everyone knows Bopomofo", generally speasking no one is trying to use it outside of Taiwan for a collation at this point....

"I'd be more inclined to trust Bopomofo pronunciation data than any other kind..."

My experience is that most Taiwan data tends to be dismally unreliable, and that appears to be born out by the short sample that Michael provides, where the ㄅㄚ "ba" readings for U+738C, U+3EA9 and U+3EAB look very suspect to me. If you add one more column to the table, showing Mandarin readings from Unihan (kMandarin field), you can see that the "ba" readings for these characters are not supported by the standard dictionaries:

U+516B 八ㄅㄚ ba1 BA1

U+4EC8 仈ㄅㄚ ba1 BA1

U+5DF4 巴ㄅㄚ ba1 BA1

U+6252 扒ㄅㄚ ba1 BA1 PA1 PA2

U+53ED 叭ㄅㄚ ba1 BA1 BA5

U+738C 玌ㄅㄚ qiu2 QIU2

U+6733 朳ㄅㄚ ba1 BA1

U+3EA9 㺩ㄅㄚ jiu4 JIU4 SE4

U+3EAB 㺫ㄅㄚ qiu2 QIU2

U+5427 吧ㄅㄚ ba5 BA5 BA1

U+5C9C 岜ㄅㄚ ba1 BA1

U+23CA9 𣲩ㄅㄚ ba1 --

There may be a few characters that have different Mandarin pronunciations in Taiwan compared with PRC (the pronunciation of 和 in the sense "and" as "he" in PRC but as "han" in Taiwan springs to mind), but I strongly doubt that U+738C, U+3EA9 and U+3EAB do not have a special Taiwan Mandarin pronunciation of ba.

But given that very many characters have more than one pronunciation, I wonder how useful a phonetic sort is. Take U+6A02 樂 (乐) for example, it has two equally common meanings with very different readings (along with a few more less common readings), either "happy" pronounced "le4" or "music" pronounced "yue4". Whichever pronunciation you sort it under it's bound to be wrong for about half of the sort items.

Hi Andrew --

Yes, this is the limitation of table based sort without an underlying dictionary to differentiate between multiple pronunciations. It is therefore not suspect so much as potentially misleading to people looking for the wrong one.

Though the sort is though of well by the people I know of from Taiwan, FWIW. I just don't know how much that would apply to people in Hong Kong and Macao....

Tone	Codepoint	Character Name	Character
first	U+02c9	MODIFIER LETTER MACRON	ˉ
second	U+02ca	MODIFIER LETTER ACUTE ACCENT	ˊ
third	U+02c7	CARON	ˇ
fourth	U+02bcb	MODIFIER LETTER GRAVE ACCENT	ˋ
light	U+02d9	DOT ABOVE	˙

Unicode Codepoint	Ideograph	Bopomofo (Taiwan)	Pinyin (China)
U+516B	八	ㄅㄚ	ba1
U+4EC8	仈	ㄅㄚ	ba1
U+5DF4	巴	ㄅㄚ	ba1
U+6252	扒	ㄅㄚ	ba1
U+53ED	叭	ㄅㄚ	ba1
U+738C	玌	ㄅㄚ	qiu2
U+6733	朳	ㄅㄚ	ba1
U+3EA9	㺩	ㄅㄚ	jiu4
U+3EAB	㺫	ㄅㄚ	qiu2
U+5427	吧	ㄅㄚ	ba5
U+5C9C	岜	ㄅㄚ	ba1
U+23CA9	𣲩	ㄅㄚ	ba1