The report of the need for a Uyghur hotfix may be an overstatement

by Michael S. Kaplan, published on 2010/05/28 07:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2010/05/28/10016086.aspx


Okay, it all started with The inappropriate nature of getting the Feh out of Uighur, where I pointed out, due to rather frantic and passionate last-minute feedback about a reported problem with the Uyghur keyboard, a change was made, pretty much at the last minute.

And how, as it ended up, the report was in error: how the original keyboard was correct and the changed keyboard was in fact a break.

Then, not quite two years later, in The inappropriate nature of getting the Feh out of Uighur, Windows 7 edition, I explained how the bug was fixed by renaming that essentially incorrect keyboard to add a "(legacy)" to the name, and adding a new keyboard with the correct letter.

Not perfect to my way of thinking, mind you, since they chose not to add the collation equivalencies I suggested. But they may have actually considered that and gotten feedback to not do it. So that may have been the correct behavior; I don't know.

But all's well as ends well, right?

Well, not quite.

At the beginning of this week, someone with the handle One Uyghur posted a note in the Suggestion Box:

 Dear Michael Kaplan,

I hope, at least, this is the one way to contact you.

I am a Uyghur (Uighur) currently living in the US. We (a group of Uyghur Computing fans living around the world) have found some problems in Uyghur Keyboard Layout in Microsoft Windows 7.

We hope to discuss it with you about it, and if possible, we hope Microsoft can release a hotfix to correct the current problems.

Your notice to this problem and timely response are greatly appreciated.

Thank you.

One Uyghur

 

Hmmm.

Anonymous people. Always makes me worry a bit, since no information was included to help know what we're talking about.

So anyway, I hadn't quite gotten around to formulating the reply (which to be honest would have been little more than a request for the information that the specific 'group of Uyghur computing fans living around the world" could share to describe the problem that they wanted to report.

I took the precaution of digging up the work I did to make sure the whole alphabet was supported, which I didn't use last time. But I'll use it now, just cause. I mean I was hearing a vague report about keyboard problems. And we had seen those previously:

The Uyghur Alphabet according to this article
IPA Letter UCP Name
/ɑ/ ئا 0626 0627 ARABIC LETTER YEH WITH HAMZA ABOVE; ARABIC LETTER ALEF
/æ/ ئە 0626 06d5 ARABIC LETTER YEH WITH HAMZA ABOVE; ARABIC LETTER AE
/b/ ب 0628 ARABIC LETTER BEH
/p/ پ 067e ARABIC LETTER PEH
/t/ ت 062a ARABIC LETTER TEH
/d͡ʒ/ ج 062c ARABIC LETTER JEEM
/t͡ʃ/ چ 0686 ARABIC LETTER TCHEH
/x/ خ 062e ARABIC LETTER KHAH
/d/ د 062f ARABIC LETTER DAL
/r/ ر 0631 ARABIC LETTER REH
/z/ ز 0632 ARABIC LETTER ZAIN
/ʒ/ ژ 0698 ARABIC LETTER JEH
/s/ س 0633 ARABIC LETTER SEEN
/ʃ/ ش 0634 ARABIC LETTER SHEEN
/ʁ/ غ 063a ARABIC LETTER GHAIN
/f/ ف 0641 ARABIC LETTER FEH
/q/ ق 0642 ARABIC LETTER QAF
/k/ ك 0643 ARABIC LETTER KAF
/ɡ/ گ 06af ARABIC LETTER GAF
/ŋ/ ڭ 06ad ARABIC LETTER NG
/l/ ل 0644 ARABIC LETTER LAM
/m/ م 0645 ARABIC LETTER MEEM
/n/ ن 0646 ARABIC LETTER NOON
/h/ ھ 06be ARABIC LETTER HEH DOACHASHMEE
/o/ ئو 0626 0648 ARABIC LETTER YEH WITH HAMZA ABOVE; ARABIC LETTER WAW
/u/ ئۇ 0626 06c7 ARABIC LETTER YEH WITH HAMZA ABOVE; ARABIC LETTER U
/ø/ ئۆ 0626 06c6 ARABIC LETTER YEH WITH HAMZA ABOVE; ARABIC LETTER OE
/y/ ئۈ 0626 06c8 ARABIC LETTER YEH WITH HAMZA ABOVE; ARABIC LETTER YU
/w/ ۋ 06cb ARABIC LETTER VE
/e/ ئې 0626 06d0 ARABIC LETTER YEH WITH HAMZA ABOVE; ARABIC LETTER E
/ɪ/ ئى 0626 0649 ARABIC LETTER YEH WITH HAMZA ABOVE; ARABIC LETTER ALEF MAKSURA
/j/ ي 064a ARABIC LETTER YEH

I verified the new keyboard had every letter (the ARABIC LETTER YEH WITH HAMZA ABOVE was put on a separate key, but everything here was typeable).

Other reports I had seen of the need for and in Uyghur (that's ARABIC LETTER YEH WITH HAMZA ABOVE ISOLATED FORM and ARABIC LETTER YEH WITH HAMZA ABOVE MEDIAL FORM, compatibility characters in Unicode) would also not be needed -- they were handled already by having ARABIC LETTER YEH WITH HAMZA ABOVE there. This Hamza on a tooth (as I heard it called a few times), was covered.

The work I had done, I puit aside. I figured I'd write up my question about what exactly is missing at some point.

But the next day, One Uyghur apparently found the report of the fixed Windows 7 keyboard I mentioned in The inappropriate nature of getting the Feh out of Uighur, Windows 7 edition and left a comment to it:

Yes, the bug is fixed. But there are still some problems with this Uyghur keyboard layout.

I would like to know how Microsoft makes the local keyboard layout for some of the language. I mean, while making the Uyghur keyboard layout, which authority or standard does Microsoft follow?

I am a Uyghur and among the 12 Uyghur punctuation marks, 3 are missing on Uyghur keyboard. And moreover, we also need to type TATWEEL (or KASHIDA, U+0640, en.wikipedia.org/.../Kashida), but we can not with the current keyboard layout on Windows 7.

Is there anyway that Microsoft will issue a hotfix to fix the problems with the current Uyghur keyboard layout?

Hmmm.

Well first there is Does MS pull new keyboard layouts out of their @!#$%? that answers that first question.

And the comment does count as more information, too.

Kind of.

First the Kashida. The Tatweel.

I talked about that a bit back in You've got to be kashidding me, but really this is not a letter we tend to add to any Arabic script keyboards so far as I know, any more than we add the SOFT HYPHENto keyboards. The "character" is there for compatibility with old standards and fine Arabic typography really calls more for using Arabic justification as supported by libraries like Uniscribe that will fill in its own Kashidas (possibly with hints on where you want them) rather than injecting these "non-letters" into the text stream.

People may wish to use the TATWEEL and if you do use it, then it will work, but really the most common way to see them is in legacy applications that insert them in order to do their own justification system. Not by adding to the keyboard layout....

I do not expect such a fix to really be making it into the keyboard any time soon, really.

So let's focus on the rest of the report -- the "12 Uyghur punctuation marks, three of which are missing", I mean.

I had seen various reports on the web of a few "special Arabic punctuation marks", namely "؛ ؟ ،" (U+061b U+061f U+060c aka ARABIC SEMICOLON, ARABIC QUESTION MARK, and ARABIC COMMA).

But all of those are on the keyboard. So it isn't those three.

Even if it had been, this is not the thing of hotfixes....

There are other punctuation marks on the keyboard, but I don't know which ones the One Uyghur group are referring to as missing....

Though remembering one of the lessons of the original bug in The inappropriate nature of getting the Feh out of Uighur, where we had an actual letter wrong, and did not fix it until the new version of Windows, it seems very unlikely that any form of punctuation would be the subject of any kind of hotfix or really even service pack fix.

Also remembering the most important lesson: don't fix your product without enough substantive information to determine what fix, if any is needed.

I am not on the team that owns keyboard layouts anymore, let alone the person checking in the changs, but to be honest I doubt any change like that would ever be made without a pointer to a national standard that described a widely used layout or documentation on an actual provided hardware layout that was in wide use. Even in a new version.

Just as the burned child fears the fire. And in that sense, the folks who push for a bug to be added ruined the more easygoing verification tendencies in the face of passionate interest.

So without even getting the full information, my answer to the question: "Is there any way that Microsoft will issue a hotfix to fix the problems with the current Uyghur keyboard layout?" is probably not.

Working that way is not in the best interests of Uyghurs.

Or of Microsoft.

Or really of anyone -- the careful and deliberate actions based on substantive problem reports with supporting evidence is the way a product like Windows, the way a company like Microsoft, has to behave....


One Uyghur on 31 May 2010 5:34 PM:

Dear Michael Kaplan,

Thank you very much for your reply and starting this topic.

Here is the set of Uyghur punctuation marks along with their Unicode code point:

  1. [ 1] 002E      .  FULL STOP
  2. [ 2] 061F      ؟  ARABIC QUESTION MARK
  3. [ 3] 0021      !  EXCLAMATION MARK
  4. [ 4] 060C      ،  ARABIC COMMA
  5. [ 5] 061B      ؛  ARABIC SEMICOLON
  6. [ 6] 003A      :  COLON
  7. [ 7] 00AB 00BB «» LEFT-POINTING DOUBLE ANGLE QUOTATION MARK  (RIGHT-...)
  8. [ 8] 2039 203A ‹› SINGLE LEFT-POINTING ANGLE QUOTATION MARK  (... RIGHT-...)
  9. [ 9] 0028 0029 () LEFT PARENTHESIS  (RIGHT ...)
  10. [10] 002D      -  HYPHEN-MINUS
  11. [11] 2014      —  EM DASH
  12. [12] 2026      …  HORIZONTAL ELLIPSIS
  13. [13] 0640      ـ  ARABIC TATWEEL

Except the ARABIC TATWEEL, the other 12 marks (1 to 12) are standard Uyghur punctuation marks, defined in various Uyghur dictionaries, language references, and taught in middle schools through colleges and universities.

Except #11, the EM DASH, all other marks' code points are also defined in Chinese National Standard - "National Standard of Peoples Republic of China - GB 21669 - 2008 - Information Technology - Uyghur, Kazak and Kirghiz Coded Character Set". (Due to the negligence of some folks, #11, the EM DASH was not included in the standard at that time. But we belive that it will surely be included in the next version of the same standard.)

(Unfortunately, I do not know how to post a picture on this reply post. I have all of the necessary documents, materials, textbooks ... in my hands about those marks. I will soon try to find a way to post some crucial portion of those materials. I will really appreciate if you could let me know or show me the instruction on how to post a picture here. A single picture is far more worth of thousands of words.)

Ok, let's continue the very topic itself.

On the current Windows 7 Uyghur keyboard, the following punctuation marks are missing:

and

Without those 3 missing punctuation marks, it is not easy to type even a 2-3 page documents in an orthographically correct way, needless to say anything more than that.

For the ARABIC TATWEEL, the mainstream word processing applications, including the Microsoft Word, can handle the full justification of Uyghur text very well. But there are still many image processing and desktop publishing applications in use today among Uyghurs, many of which have no support or very limited support of right to left text or whatever the Uyghurs need. Thus, in these stuations, in order to produce elegant documents, the ARABIC TATWEEL's role could be never overestimated.

In conclusion, the above punctuation marks are well defined in textbooks, refernces and even a national standard. But some of them are missing on the windows 7 Uyghur keyboard. For this reason, and to their dismay, the majority of the Windows 7 users need to type some kind of serious Uyghur texts have not benefited much from the Microsoft's world-class new operating system - the Windows 7. This is a pity for all parties involved.

What is more confusing is the point that, while almost everything well defined, how Microsoft has been mishandling our case constantly. There was once the famous "F (or FEH) dilemma", now is still some obvious mistakes or omissions. It is really hard for many Uyghurs, who have been loyal Windows users and have very high appreciation towards Microsoft and its products, to understand that how these couple of marks were failed to be handled correctly, while the Microsoft engineers can handle billions of lines of codes correctly.

At least one point come to many Uyghurs' mind that, it is better to know which authority or references Microsoft follow while it develops Uyghur related products, most importantly, the keyboard layout. It is extremely painful to discover the things you expected so long does not work the way you expected and you have no other choice.

(By the way, do you have the experience in your childhood that, in the early morning after Christmas day, you are much excited and delighted to find your much expected toys in the Santa's socks somewhere in your houese, and, while eager to try or play your all-time favourite gift toy, to only find out that it is broken or does not work the way it should be? May God save every and each kid from such a miserable experience! But, millions of Uyghurs, loyal Windows users and Microsoft customers, have already had such a miserable experience, and not once but twice!)

Ok. By now, I hope that the seriousness of this problem could be understood easily by the thousands of world-class Microsoft engineers, who in turn, can easily understand the importance of every bit of their billion-line-codes. Please, dear Microsoft, please do not fail us again. We can not bear any more misery, or third misery in the near future. (If Microsoft ever needs any Uyghur language related assitance during its product development and testing, I am well sure that there are many Uyghur volunteers who gladly offer free assistance, at least, I am one of them.)

God bless Microsoft engineers, decision makers, executives, and more, Microsoft itself!

Michael S. Kaplan on 31 May 2010 7:07 PM:

I will forward this information on to the owners of the keyboard layout, but as I said a hotfix for a problem that has shipped in four versions of Windows (Vista, Server 2008, Windows 7, and Server 2008 R2) is incredibly unlikely.

I was unable to find an English translation of GB 21669, official or otherwise (and I do not know if the standard has suggestions about a keyboard layout or not though this link does not make me hopeful), but no images of the characters themselves are required from you; the code points you named exist in standard fonts (they even exist in the "Microsoft Uighur" font that ships with Windows >= Vista).

For the time being, one can install MSKLC and create a keyboard layout that contains these punctuation marks. They are supported by the platform and no one is blocked by the fact that they are not in the keyboard layout itself....

Random832 on 2 Jun 2010 7:43 AM:

The "Does MS pull new keyboard layouts out of..." blog doesn't really answer the question that was asked, for the Uyghur keyboard or for any other specific keyboard, of what went into the process of making that particular keyboard. At the end of the day, _someone_, at some time, invented the layout, regardless of if it was Microsoft or IBM or someone else entirely; alone or with coordination from experts, government officials, linguists, local software developers, etc. And I don't think I need to mention that there are 19 keys with no shift state on this keyboard.

Michael S. Kaplan on 2 Jun 2010 8:20 AM:

I was not attempting to answer the specific question about origin, I was giving a general answer about the stuff that goes into the process....


referenced by

2010/11/25 One Uyghur walked into a Blog, and...

go to newer or older post, or back to index or month or day