Had I known that my last release would be *the* last release..., aka hindsight is 2020

by Michael S. Kaplan, published on 2011/01/07 07:01 -05:00, original URI: http://blogs.msdn.com/b/michkap/archive/2011/01/07/10112915.aspx

It all starts with the Win32 GetKeyboardType function. If you look at the MSDN topic, it gives the following possible keyboard types:

If you look in the WDK’s kbd.h it tells a slightly fuller story, though:

Now the primary “feature” that kbd.h provides is a Scan Code to VK mapping for each of these keyboards. The mapping is basically the same for most of VK_A to VK_Z but all over the place on the various VK_OEM_* entries.

These keyboard types are referring to hardware and nominally represent the way actual keys are laid out.

Note that they do not handle all of the alternate layout information when language keyboards are in other languages even when they do handle the VK_OEM_* keys properly.

Of course I have written many times in the past in this Blog about how the keyboard layout DLLs map hardware scan codes to VK (virtual key) values.

Some of it happens to match in these two very different mappings for individual languages, but mostly they are a huge non-overlapping mess.

And this is where MSKLC comes in.

At its basic fundamental level, it is a fancy GUI wrapper around KBDUTOOL.EXE, which is itself a Unicode port of the KBDTOOL.EXE that is used by Windows itself to create keyboard layouts.

When it is building layouts, it is making use of KBD.H and thus has some interesting dependencies on one of these scan code/VK mappings even though its only real purpose is to build a DLL that supports the other mapping. And a KBD_TYPE is set by KBDUTOOL.EXE, even though one is never specified in the keyboard source files.

This strange combination of an incomplete attempt to capture keyboard hardware and inept way to build the layouts that sit atop them seems far less than ideal.

And this is where the problems can start.

It was not too long ago that ML posted a question over in the Suggestion Box (a few others have sent me mail about this before but this was the most recent one):

Hello Michael!

I'm a beginner when it comes to the complexities behind MSKLC, so if you are able to respond to this problem, I hope that you can keep your answer relatively straightforward. All that I understand so far about keyboards is that the physical keys themselves on the keyboard have codes called "scan codes". The symbol on the surface of the key, or the code for whatever appears on the screen when you press the key, is the "virtual code".

I've created my own phonetic keyboards for some different languages with MSKLC. Now I'm trying to create my own phonetic Lao keyboard, and this is the first time I've encountered this problem. The keyboard is validated, but directly thereafter I get an Error 2020 window. For example:

Error 2020 (..\.\tmpLaoPhon1.txt, line 67):
VK_OEM_1 (ba) found at scancodes 1a and 27.

Because I have a German keyboard, I then went to this page where I saw that scancodes 1a and 27 are VK_OEM_1 and VK_OEM_3. Then I looked back at those virtual key locations on the keyboard layout I was creating by clicking on those keys and selecting "All...". I couldn't figure out what could be wrong, so I just deleted everything located at those virtual keys.

When I tried to build the setup package again, I got more Error 2020's for all virtual keys VK_OEM_1 through VK_OEM_7 and the VK_OEM_PLUS and VK_OEM_MINUS keys.

These are the symbols that had been entered at those keys:

VK_OEM_1
    <Key> U+0eb6
    shift+<Key> U+0eb7

VK_OEM_PLUS
    <Key> U+002b
    shift+<Key> U+002a

VK_OEM_2
    shift+<Key> U+005f

VK_OEM_3
    Nothing

VK_OEM_4
    shift+<Key> U+003f
    ctrl+Alt+<Key> U+005c

VK_OEM_6
    Nothing

VK_OEM_7
    <Key> U+0023
    shift+<Key> U+0027

VK_OEM_MINUS
    <Key> U+002d
    shift+<Key> U+005f

I continued to delete all symbols that I had entered for the virtual keys mentioned in the error windows. I was only able to build the setup package when I had deleted all key symbols for VK_OEM_1 through VK_OEM_7 and the VK_OEM_PLUS and VK_OEM_MINUS keys.

The really weird thing is that if I copy all of the symbols for these keys into a MSKLC keyboard I created for Thai, there are no Error 2020's.

I found a similar blog entry of yours called "The advanced feature in MSKLC doesn't really scan", but it doesn't seem to be exactly the same problem, and there was also no solution presented that I could find.

I've just tried creating new keyboard layouts altogether, and I've found that I now get these errors for the VK_OEM keys consistently, no matter what language or symbol type I enter on these keys /:-.

I'd be grateful if you have any ideas to what is going on.

Thanks!
ML

Ah, the famous error 2020.

The internal constant for the error code is KBDTOOL_ERROR_DUPLICATE_SCANCODE, and the error text:

Error 2020 (..\.\tmpLaoPhon1.txt, line 67):
VK_OEM_1 (ba) found at scancodes 1a and 27.

lists what the duplicates were.

Note that you will get the same error if your .KLC file has two duplicate scan code entries; KBDUTOOL does not distinguish between a file having duplicates and a file duplicating its assumptions.

Now as luck (for lack of another word) would have it, the first scan code is pretty much invariably the one in the header file, and the second scan code is the one found in the .KLC file.

So if you find yourself in this state, one way to fix it would be to:

1. Open up the .KLC file in Notepad.
2. Change the VK_* value cited in the error to use the first scan code rather than the second.
3. Save the .KLC file.
4. Open the newly changed .KLC file in MSKLC.
5. Rebuild the keyboard in MSKLC.
6. If you get another error 2020, repeat steps 1-5 again until the build is successful.

Now although this solves the problem of the keyboard not building, it may leave you with either

• some punctuation being in the wrong place (an obvious potential problem with the moving around being done here), or
• some punctuation having an unexpected VK value associated with it (causing similar problem to the one I described in Y can't Z Undo, exactly?, albeit for different reasons and more likely involving punctuation than letters in most cases).

Both of those issues kind of suck, though they are fixable.

A less satisfying solution is to start over from as keyboard that does work (you can load an existing keyboard and save it, for example – it will build) and then make the keyboard be the way you want it to be. On the whole I prefer the first solution to the second, because fixing up a limited number of keys is always going to be easier than starting over.

In the long run, I would rather see this fixed in a new version of MSKLC and its KBDUTOOL, with the following changes:

1. Fix the bizarre keyboard type guessing behavior in KBDUTOOL that assumes the .KLC file is wrong and is duplicating scan codes just because the source does not match the [flawed] expectations of the tool;
2. Expose the keyboard type/subtype as settable options in MSKLC/KBDUTOOL;
3. Perform explicit punctuation “switching” based on keyboard type to better support one language’s keyboard against expected punctuation;

Note that will fix many problems noted previously in blogs like Ideas about loading existing keyboards in MSKLC and The real problems with keyboard switching and Does MS just make up these punctuation-challenged keyboards to piss people off? (not to mention the other blogs the latter points to!).

However, I do not own MSKLC anymore and since the team that does own it has never shipped an update, this seems unlikely to be a future change in the tool.

I should have done more when I did own it; had I known that my last release would be the last release, I migt have done things differently....

referenced by

go to newer or older post, or back to index or month or day