Getting all you can out of a keyboard layout, Part #9b

by Michael S. Kaplan, published on 2006/04/13 03:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2006/04/13/575500.aspx


(Previous posts in this series: Parts 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9a)

In my last post, I promised to provide the updated version of the code sample that will handle any keyboard layout that ships with Windows, including the Canadian Multilingual Standard layout....

A brief note on why it may seem like I am picking on this particular layout.... it represents to me the prototypical example of a government to try to support a mutlilingual keyboard. But as I discussed in Keyboards: Monolingual or Multilingual? such attempts, while being perhaps quite useful for a limited segment of the population, can be quite difficult to use for many in the intended market....

The other problem is of course the extensive use of the custom shift states, when there was actually room in the other, more conventional shift states for all of the assigned letters. Though I guess if they never did that, I would not have been posting this particular part of the series. :-)

At the end of this post you'll be able to see the layout and you can decide for yourself if you would swaer by the layout, or at it, were you to use it....

Anyway, here is the updated code (which fixes a few bugs exposed by this particular layout):

using System;
using System.Text;
using System.Collections;
using System.Windows.Forms;
using System.Runtime.InteropServices;

namespace KeyboardLayouts {

    //  You'll want to insert that enumeration from part #0 here!

    public enum ShiftState : int {
        Base            = 0,                    // 0
        Shft            = 1,                    // 1
        Ctrl            = 2,                    // 2
        ShftCtrl        = Shft | Ctrl,          // 3
        Menu            = 4,                    // 4 -- NOT USED
        ShftMenu        = Shft | Menu,          // 5 -- NOT USED
        MenuCtrl        = Menu | Ctrl,          // 6
        ShftMenuCtrl    = Shft | Menu | Ctrl,   // 7
        Xxxx            = 8,                    // 8
        ShftXxxx        = Shft | Xxxx,          // 9
    }

    public class DeadKey {
        private char m_deadchar;
        private ArrayList m_rgbasechar = new ArrayList();
        private ArrayList m_rgcombchar = new ArrayList();

        public DeadKey(char deadCharacter) {
            this.m_deadchar = deadCharacter;
        }

        public char DeadCharacter {
            get {
                return this.m_deadchar;
            }
        }

        public void AddDeadKeyRow(char baseCharacter, char combinedCharacter) {
            this.m_rgbasechar.Add(baseCharacter);
            this.m_rgcombchar.Add(combinedCharacter);
        }

        public int Count {
            get {
                return this.m_rgbasechar.Count;
            }
        }

        public char GetBaseCharacter(int index) {
            return (char)this.m_rgbasechar[index];
        }

        public char GetCombinedCharacter(int index) {
            return (char)this.m_rgcombchar[index];
        }

        public bool ContainsBaseCharacter(char baseCharacter) {
            return this.m_rgbasechar.Contains(baseCharacter);
        }
    }

    public class VirtualKey {
        [DllImport("user32.dll", CharSet=CharSet.Unicode, EntryPoint="MapVirtualKeyExW", ExactSpelling=true)]
        internal static extern uint MapVirtualKeyEx(
            uint uCode,
            uint uMapType,
            IntPtr dwhkl);

        private IntPtr m_hkl;
        private uint m_vk;
        private uint m_sc;
        private bool[,] m_rgfDeadKey = new bool[(int)ShiftState.ShftXxxx + 1,2];
        private string[,] m_rgss = new string[(int)ShiftState.ShftXxxx + 1,2];

        public VirtualKey(IntPtr hkl, KeysEx virtualKey) {
            this.m_sc = MapVirtualKeyEx((uint)virtualKey, 0, hkl);
            this.m_hkl = hkl;
            this.m_vk = (uint)virtualKey;
        }

        public VirtualKey(IntPtr hkl, uint scanCode) {
            this.m_vk = MapVirtualKeyEx(scanCode, 1, hkl);
            this.m_hkl = hkl;
            this.m_sc = scanCode;
        }

        public KeysEx VK {
            get { return (KeysEx)this.m_vk; }
        }

        public uint SC {
            get { return this.m_sc; }
        }

        public string GetShiftState(ShiftState shiftState, bool capsLock) {
            if(this.m_rgss[(uint)shiftState, (capsLock ? 1 : 0)] == null) {
                return("");
            }
           
            return(this.m_rgss[(uint)shiftState, (capsLock ? 1 : 0)]);
        }

        public void SetShiftState(ShiftState shiftState, string value, bool isDeadKey, bool capsLock) {
            this.m_rgfDeadKey[(uint)shiftState, (capsLock ? 1 : 0)] = isDeadKey;
            this.m_rgss[(uint)shiftState, (capsLock ? 1 : 0)] = value;
        }

        public bool IsSGCAPS {
            get {
                string stBase = this.GetShiftState(ShiftState.Base, false);
                string stShift = this.GetShiftState(ShiftState.Shft, false);
                string stCaps = this.GetShiftState(ShiftState.Base, true);
                string stShiftCaps = this.GetShiftState(ShiftState.Shft, true);
                return(
                    ((stCaps.Length > 0) &&
                    (! stBase.Equals(stCaps)) &&
                    (! stShift.Equals(stCaps))) ||
                    ((stShiftCaps.Length > 0) &&
                    (! stBase.Equals(stShiftCaps)) &&
                    (! stShift.Equals(stShiftCaps))));
            }
        }

        public bool IsCapsEqualToShift {
            get {
                string stBase = this.GetShiftState(ShiftState.Base, false);
                string stShift = this.GetShiftState(ShiftState.Shft, false);
                string stCaps = this.GetShiftState(ShiftState.Base, true);
                return(
                    (stBase.Length > 0) &&
                    (stShift.Length > 0) &&
                    (! stBase.Equals(stShift)) &&
                    (stShift.Equals(stCaps)));
            }
        }

        public bool IsAltGrCapsEqualToAltGrShift {
            get {
                string stBase = this.GetShiftState(ShiftState.MenuCtrl, false);
                string stShift = this.GetShiftState(ShiftState.ShftMenuCtrl, false);
                string stCaps = this.GetShiftState(ShiftState.MenuCtrl, true);
                return(
                    (stBase.Length > 0) &&
                    (stShift.Length > 0) &&
                    (! stBase.Equals(stShift)) &&
                    (stShift.Equals(stCaps)));
            }
        }

        public bool IsXxxxGrCapsEqualToXxxxShift {
            get {
                string stBase = this.GetShiftState(ShiftState.Xxxx, false);
                string stShift = this.GetShiftState(ShiftState.ShftXxxx, false);
                string stCaps = this.GetShiftState(ShiftState.Xxxx, true);
                return(
                    (stBase.Length > 0) &&
                    (stShift.Length > 0) &&
                    (! stBase.Equals(stShift)) &&
                    (stShift.Equals(stCaps)));
            }
        }

        public bool IsEmpty {
            get {
                for(int i = 0; i < this.m_rgss.GetUpperBound(0); i++) {
                    for(int j = 0; j <= 1; j++) {
                        if(this.GetShiftState((ShiftState)i, (j == 1)).Length > 0) {
                            return(false);
                        }
                    }
                }
                return true;
            }
        }

        public string LayoutRow {
            get {
                StringBuilder sbRow = new StringBuilder();

                // First, get the SC/VK info stored
                sbRow.Append(string.Format("{0:x2}\t{1:x2} - {2}", this.SC, (byte)this.VK, ((KeysEx)this.VK).ToString().PadRight(13)));

                // Now the CAPSLOCK value
                int capslock =
                    0 |
                    (this.IsCapsEqualToShift ? 1 : 0) |
                    (this.IsSGCAPS ? 2 : 0) |
                    (this.IsAltGrCapsEqualToAltGrShift ? 4 : 0) |
                    (this.IsXxxxGrCapsEqualToXxxxShift ? 8 : 0);
                sbRow.Append(string.Format("\t{0}", capslock));


                for(ShiftState ss = 0; ss <= Loader.MaxShiftState; ss++) {
                    if(ss == ShiftState.Menu || ss == ShiftState.ShftMenu) {
                        // Alt and Shift+Alt don't work, so skip them
                        continue;
                    }
                    for(int caps = 0; caps <= 1; caps++) {
                        string st = this.GetShiftState(ss, (caps == 1));

                        if(st.Length == 0) {
                            // No character assigned here, put in -1.
                            sbRow.Append("\t  -1");
                        }
                        else if((caps == 1) && st == (this.GetShiftState(ss, (caps == 0)))) {
                            // Its a CAPS LOCK state and the assigned character(s) are
                            // identical to the non-CAPS LOCK state. Put in a MIDDLE DOT.
                            sbRow.Append("\t   \u00b7");
                        }
                        else if(this.m_rgfDeadKey[(int)ss, caps]) {
                            // It's a dead key, append an @ sign.
                            sbRow.Append(string.Format("\t{0:x4}@", ((ushort)st[0])));
                        }
                        else {
                            // It's some characters; put 'em in there.
                            StringBuilder sbChar = new StringBuilder((5 * st.Length) + 1);
                            for(int ich = 0; ich < st.Length; ich++) {
                                sbChar.Append(((ushort)st[ich]).ToString("x4"));
                                sbChar.Append(' ');
                            }
                            sbRow.Append(string.Format("\t{0}", sbChar.ToString(0, sbChar.Length - 1)));
                        }
                    }
                }

                return sbRow.ToString();
            }
        }
    }

    public class Loader {

        private const uint KLF_NOTELLSHELL  = 0x00000080;

        internal static KeysEx[] lpKeyStateNull = new KeysEx[256];

        [DllImport("user32.dll", CharSet=CharSet.Unicode, EntryPoint="LoadKeyboardLayoutW", ExactSpelling=true)]
        private static extern IntPtr LoadKeyboardLayout(string pwszKLID, uint Flags);

        [DllImport("user32.dll", ExactSpelling=true)]
        private static extern bool UnloadKeyboardLayout(IntPtr hkl);

        [DllImport("user32.dll", CharSet=CharSet.Unicode, ExactSpelling=true)]
        private static extern int ToUnicodeEx(
            uint wVirtKey,
            uint wScanCode,
            KeysEx[] lpKeyState,
            StringBuilder pwszBuff,
            int cchBuff,
            uint wFlags,
            IntPtr dwhkl);

        [DllImport("user32.dll", CharSet=CharSet.Unicode, EntryPoint="VkKeyScanExW", ExactSpelling=true)]
        private static extern ushort VkKeyScanEx(char ch, IntPtr dwhkl);

        [DllImport("user32.dll", ExactSpelling=true)]
        private static extern int GetKeyboardLayoutList(int nBuff, [Out, MarshalAs(UnmanagedType.LPArray)] IntPtr[] lpList);

        private static KeysEx m_XxxxVk = KeysEx.None;
        public static KeysEx XxxxVk {
            get {
                return m_XxxxVk;
            }
            set {
                m_XxxxVk = value;
            }
        }

        public static ShiftState MaxShiftState {
            get {
                return (Loader.XxxxVk == KeysEx.None ? ShiftState.ShftMenuCtrl : ShiftState.ShftXxxx);
            }
        }

        private static void FillKeyState(KeysEx[] lpKeyState, ShiftState ss, bool fCapsLock) {
            lpKeyState[(int)KeysEx.VK_SHIFT]    = (((ss & ShiftState.Shft) != 0) ? (KeysEx)0x80 : (KeysEx)0x00);
            lpKeyState[(int)KeysEx.VK_CONTROL]  = (((ss & ShiftState.Ctrl) != 0) ? (KeysEx)0x80 : (KeysEx)0x00);
            lpKeyState[(int)KeysEx.VK_MENU]     = (((ss & ShiftState.Menu) != 0) ? (KeysEx)0x80 : (KeysEx)0x00);
            if(Loader.XxxxVk != KeysEx.None) {
                // The Xxxx key has been assigned, so let's include it
                lpKeyState[(int)Loader.XxxxVk]       = (((ss & ShiftState.Xxxx) != 0) ? (KeysEx)0x80 : (KeysEx)0x00);
            }
            lpKeyState[(int)KeysEx.VK_CAPITAL]  = (fCapsLock ? (KeysEx)0x01 : (KeysEx)0x00);
        }

        private static DeadKey ProcessDeadKey(
            uint iKeyDead,              // The index into the VirtualKey of the dead key
            ShiftState shiftStateDead,  // The shiftstate that contains the dead key
            KeysEx[] lpKeyStateDead,    // The key state for the dead key
            VirtualKey[] rgKey,         // Our array of dead keys
            bool fCapsLock,             // Was the caps lock key pressed?
            IntPtr hkl) {               // The keyboard layout

            KeysEx[] lpKeyState = new KeysEx[256];
            DeadKey deadKey = new DeadKey(rgKey[iKeyDead].GetShiftState(shiftStateDead, fCapsLock)[0]);

            for(uint iKey = 0; iKey < rgKey.Length; iKey++) {
                if(rgKey[iKey] != null) {
                    StringBuilder sbBuffer = new StringBuilder(10);     // Scratchpad we use many places

                    for(ShiftState ss = ShiftState.Base; ss <= Loader.MaxShiftState; ss++) {
                        int rc = 0;
                        if(ss == ShiftState.Menu || ss == ShiftState.ShftMenu) {
                            // Alt and Shift+Alt don't work, so skip them
                            continue;
                        }

                        for(int caps = 0; caps <=1; caps++) {
                            // First the dead key
                            while(rc >= 0) {
                                // We know that this is a dead key coming up, otherwise
                                // this function would never have been called. If we do
                                // *not* get a dead key then that means the state is
                                // messed up so we run again and again to clear it up.
                                // Risk is technically an infinite loop but per Hiroyama
                                // that should be impossible here.
                                rc = ToUnicodeEx((uint)rgKey[iKeyDead].VK, rgKey[iKeyDead].SC, lpKeyStateDead, sbBuffer, sbBuffer.Capacity, 0, hkl);
                            }

                            // Now fill the key state for the potential base character
                            FillKeyState(lpKeyState, ss, (caps != 0));

                            sbBuffer = new StringBuilder(10);
                            rc = ToUnicodeEx((uint)rgKey[iKey].VK, rgKey[iKey].SC, lpKeyState, sbBuffer, sbBuffer.Capacity, 0, hkl);
                            if(rc == 1) {
                                // That was indeed a base character for our dead key.
                                // And we now have a composite character. Let's run
                                // through one more time to get the actual base
                                // character that made it all possible?
                                char combchar = sbBuffer[0];
                                sbBuffer = new StringBuilder(10);
                                rc = ToUnicodeEx((uint)rgKey[iKey].VK, rgKey[iKey].SC, lpKeyState, sbBuffer, sbBuffer.Capacity, 0, hkl);

                                char basechar = sbBuffer[0];

                                if(deadKey.DeadCharacter == combchar) {
                                    // Since the combined character is the same as the dead key,
                                    // we must clear out the keyboard buffer.
                                    ClearKeyboardBuffer((uint)KeysEx.VK_DECIMAL, rgKey[(uint)KeysEx.VK_DECIMAL].SC, hkl);
                                }

                                if((((ss == ShiftState.Ctrl) || (ss == ShiftState.ShftCtrl)) &&
                                    (char.IsControl(basechar))) ||
                                    (basechar.Equals(combchar))) {
                                    // ToUnicodeEx has an internal knowledge about those
                                    // VK_A ~ VK_Z keys to produce the control characters,
                                    // when the conversion rule is not provided in keyboard
                                    // layout files

                                    // Additionally, dead key state is lost for some of these
                                    // character combinations, for unknown reasons.

                                    // Therefore, if the base character and combining are equal,
                                    // and its a CTRL or CTRL+SHIFT state, and a control character
                                    // is returned, then we do not add this "dead key" (which
                                    // is not really a dead key).
                                    continue;
                                }

                                if(! deadKey.ContainsBaseCharacter(basechar)) {
                                    deadKey.AddDeadKeyRow(basechar, combchar);
                                }
                            }
                            else if(rc > 1) {
                                // Not a valid dead key combination, sorry! We just ignore it.
                            }
                            else if(rc < 0) {
                                // It's another dead key, so we ignore it (other than to flush it from the state)
                                ClearKeyboardBuffer((uint)KeysEx.VK_DECIMAL, rgKey[(uint)KeysEx.VK_DECIMAL].SC, hkl);
                            }
                        }
                    }
                }
            }
            return deadKey;
        }

        private static void ClearKeyboardBuffer(uint vk, uint sc, IntPtr hkl) {
            StringBuilder sb = new StringBuilder(10);
            int rc = 0;
            while(rc != 1) {
                rc = ToUnicodeEx(vk, sc, lpKeyStateNull, sb, sb.Capacity, 0, hkl);
            }
        }

        [STAThread]
        static void Main(string[] args) {
            int cKeyboards = GetKeyboardLayoutList(0, null);
            IntPtr[] rghkl = new IntPtr[cKeyboards];
            GetKeyboardLayoutList(cKeyboards, rghkl);           
            IntPtr hkl = LoadKeyboardLayout(args[0], KLF_NOTELLSHELL);
            if(hkl == IntPtr.Zero) {
                Console.WriteLine("Sorry, that keyboard does not seem to be valid.");
            }
            else {
                KeysEx[] lpKeyState = new KeysEx[256];
                VirtualKey[] rgKey = new VirtualKey[256];
                ArrayList alDead = new ArrayList();

                // Scroll through the Scan Code (SC) values and get the valid Virtual Key (VK)
                // values in it. Then, store the SC in each valid VK so it can act as both a
                // flag that the VK is valid, and it can store the SC value.
                for(uint sc = 0x01; sc <= 0x7f; sc++) {
                    VirtualKey key = new VirtualKey(hkl, sc);
                    if(key.VK != 0) {
                        rgKey[(uint)key.VK] = key;
                    }
                }

                // add the special keys that do not get added from the code above
                for(KeysEx ke = KeysEx.VK_NUMPAD0; ke <= KeysEx.VK_NUMPAD9; ke++) {
                    rgKey[(uint)ke] = new VirtualKey(hkl, ke);
                }
                rgKey[(uint)KeysEx.VK_DIVIDE] = new VirtualKey(hkl, KeysEx.VK_DIVIDE);
                rgKey[(uint)KeysEx.VK_CANCEL] = new VirtualKey(hkl, KeysEx.VK_CANCEL);
                rgKey[(uint)KeysEx.VK_DECIMAL] = new VirtualKey(hkl, KeysEx.VK_DECIMAL);

                // See if there is a special shift state added
                for(KeysEx vk = KeysEx.None; vk <= KeysEx.VK_OEM_CLEAR; vk++) {
                    uint sc = VirtualKey.MapVirtualKeyEx((uint)vk, 0, hkl);
                    uint vkL = VirtualKey.MapVirtualKeyEx(sc, 1, hkl);
                    uint vkR = VirtualKey.MapVirtualKeyEx(sc, 3, hkl);
                    if((vkL != vkR) &&
                        ((uint)vk != vkL)) {
                        switch(vk) {
                            case KeysEx.VK_LCONTROL:
                            case KeysEx.VK_RCONTROL:
                            case KeysEx.VK_LSHIFT:
                            case KeysEx.VK_RSHIFT:
                            case KeysEx.VK_LMENU:
                            case KeysEx.VK_RMENU:

                                break;

                            default:
                                Loader.XxxxVk = vk;
                                break;
                        }
                    }
                }

                for(uint iKey = 0; iKey < rgKey.Length; iKey++) {
                    if(rgKey[iKey] != null) {
                        StringBuilder sbBuffer;     // Scratchpad we use many places

                        for(ShiftState ss = ShiftState.Base; ss <= Loader.MaxShiftState; ss++) {
                            if(ss == ShiftState.Menu || ss == ShiftState.ShftMenu) {
                                // Alt and Shift+Alt don't work, so skip them
                                continue;
                            }

                            for(int caps = 0; caps <= 1; caps++) {
                                ClearKeyboardBuffer((uint)KeysEx.VK_DECIMAL, rgKey[(uint)KeysEx.VK_DECIMAL].SC, hkl);
                                FillKeyState(lpKeyState, ss, (caps != 0));
                                sbBuffer = new StringBuilder(10)
                                int rc = ToUnicodeEx((uint)rgKey[iKey].VK, rgKey[iKey].SC, lpKeyState, sbBuffer, sbBuffer.Capacity, 0, hkl);
                                if(rc > 0) {
                                    if(sbBuffer.Length == 0) {
                                        // Someone defined NULL on the keyboard; let's coddle them
                                        rgKey[iKey].SetShiftState(ss, "\u0000", false, (caps != 0));
                                    }
                                    else {
                                        if((rc == 1) &&
                                            (ss == ShiftState.Ctrl || ss == ShiftState.ShftCtrl) &&
                                            ((int)rgKey[iKey].VK == ((uint)sbBuffer[0] + 0x40))) {
                                            // ToUnicodeEx has an internal knowledge about those
                                            // VK_A ~ VK_Z keys to produce the control characters,
                                            // when the conversion rule is not provided in keyboard
                                            // layout files
                                            continue;
                                        }
                                        rgKey[iKey].SetShiftState(ss, sbBuffer.ToString().Substring(0, rc), false, (caps != 0));
                                    }
                                }
                                else if(rc < 0) {
                                    rgKey[iKey].SetShiftState(ss, sbBuffer.ToString().Substring(0, 1), true, (caps != 0));

                                    // It's a dead key; let's flush out whats stored in the keyboard state.
                                    ClearKeyboardBuffer((uint)KeysEx.VK_DECIMAL, rgKey[(uint)KeysEx.VK_DECIMAL].SC, hkl);
                                    DeadKey dk = null;
                                    for(int iDead = 0; iDead < alDead.Count; iDead++) {
                                        dk = (DeadKey)alDead[iDead];
                                        if(dk.DeadCharacter == rgKey[iKey].GetShiftState(ss, caps != 0)[0]) {
                                            break;
                                        }
                                        dk = null;
                                    }
                                    if(dk == null) {
                                        alDead.Add(ProcessDeadKey(iKey, ss, lpKeyState, rgKey, caps == 1, hkl));
                                    }
                                }
                            }
                        }
                    }
                }

                foreach(IntPtr i in rghkl) {
                    if(hkl == i) {
                        hkl = IntPtr.Zero;
                        break;
                    }
                }

                if(hkl != IntPtr.Zero) {
                    UnloadKeyboardLayout(hkl);
                }

                // Okay, now we can dump the layout
                Console.Write("\nSC\tVK  \t\t\tCAPS\t_\t_C\ts\tsC\tc\tcC\tsc\tscC\t\tca\tcaC\tsca\tscaC");
                if(Loader.XxxxVk != KeysEx.None) {
                    Console.Write("\tx\txC\tsx\tsxC");
                }
                Console.WriteLine();
                Console.Write("==\t==========\t\t====\t====\t====\t====\t====\t====\t====\t====\t====\t====\t====\t====\t====");
                if(Loader.XxxxVk != KeysEx.None) {
                    Console.Write("\t====\t====\t====\t====");
                }
                Console.WriteLine();

                for(uint iKey = 0; iKey < rgKey.Length; iKey++) {
                    if((rgKey[iKey] != null) &&
                        ( !rgKey[iKey].IsEmpty)) {
                        Console.WriteLine(rgKey[iKey].LayoutRow);
                    }
                }

                foreach(DeadKey dk in alDead) {
                    Console.WriteLine();
                    Console.WriteLine("0x{0:x4}\t{1}", ((ushort)dk.DeadCharacter).ToString("x4"), dk.Count);
                    for(int id = 0; id < dk.Count; id++) {
                        Console.WriteLine("\t0x{0:x4}\t0x{1:x4}",
                            ((ushort)dk.GetBaseCharacter(id)).ToString("x4"),
                            ((ushort)dk.GetCombinedCharacter(id)).ToString("x4"));
                    }
                }
                Console.WriteLine();

            }
        }
    }
}

The code above fixes a bug not found with previous versions that would cause dead key tables to show up more than once if the dead key is defined multiple times....

And now, here is that keyboard with the 00011009 KLID value (you'll want a console window with 170 columns in it to print it out yourself!):

C:\KeyboardLayouts\bin\Debug>KeyboardLayouts.exe 00011009

SC      VK                      CAPS    _       _C      s       sC      c       cC      sc      scC     ca      caC     sca     scaC    x       xC      sx      sxC
==      ==========              ====    ====    ====    ====    ====    ====    ====    ====    ====    ====    ====    ====    ====    ====    ====    ====    ====
46      03 - VK_CANCEL          0       0003       ·    0003       ·    0003       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
0e      08 - VK_BACK            0       0008       ·    0008       ·    007f       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
7c      09 - VK_TAB             0       0009       ·    0009       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
1c      0d - VK_RETURN          0       000d       ·    000d       ·    000a       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
01      1b - VK_ESCAPE          0       001b       ·    001b       ·    001b       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
39      20 - VK_SPACE           0       0020       ·    0020       ·    0020       ·      -1      -1    00a0       ·      -1      -1    0020       ·    0020       ·
0b      30 - VK_0               0       0030       ·    0029       ·      -1      -1      -1      -1    005d       ·      -1      -1      -1      -1      -1      -1
02      31 - VK_1               0       0031       ·    0021       ·      -1      -1      -1      -1      -1      -1      -1      -1    00b9       ·    00a1       ·
03      32 - VK_2               0       0032       ·    0040       ·      -1      -1      -1      -1      -1      -1      -1      -1    00b2       ·      -1      -1
04      33 - VK_3               0       0033       ·    0023       ·      -1      -1      -1      -1      -1      -1      -1      -1    00b3       ·    00a3       ·
05      34 - VK_4               0       0034       ·    0024       ·      -1      -1      -1      -1    00a4       ·      -1      -1    00bc       ·    20ac       ·
06      35 - VK_5               0       0035       ·    0025       ·      -1      -1      -1      -1      -1      -1      -1      -1    00bd       ·    215c       ·
07      36 - VK_6               0       0036       ·    003f       ·      -1      -1      -1      -1      -1      -1      -1      -1    00be       ·    215d       ·
08      37 - VK_7               0       0037       ·    0026       ·      -1      -1      -1      -1    007b       ·      -1      -1      -1      -1    215e       ·
09      38 - VK_8               0       0038       ·    002a       ·      -1      -1      -1      -1    007d       ·      -1      -1      -1      -1    2122       ·
0a      39 - VK_9               0       0039       ·    0028       ·      -1      -1      -1      -1    005b       ·      -1      -1      -1      -1    00b1       ·
1e      41 - VK_A               1       0061    0041    0041    0061      -1      -1      -1      -1      -1      -1      -1      -1    00e6       ·    00c6       ·
30      42 - VK_B               1       0062    0042    0042    0062      -1      -1      -1      -1      -1      -1      -1      -1    201d       ·    2019       ·
2e      43 - VK_C               1       0063    0043    0043    0063      -1      -1      -1      -1      -1      -1      -1      -1    00a2       ·    00a9       ·
20      44 - VK_D               1       0064    0044    0044    0064      -1      -1      -1      -1      -1      -1      -1      -1    00f0       ·    00d0       ·
12      45 - VK_E               1       0065    0045    0045    0065      -1      -1      -1      -1    20ac       ·      -1      -1    0153       ·    0152       ·
21      46 - VK_F               1       0066    0046    0046    0066      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1    00aa       ·
22      47 - VK_G               1       0067    0047    0047    0067      -1      -1      -1      -1      -1      -1      -1      -1    014b       ·    014a       ·
23      48 - VK_H               1       0068    0048    0048    0068      -1      -1      -1      -1      -1      -1      -1      -1    0127       ·    0126       ·
17      49 - VK_I               1       0069    0049    0049    0069      -1      -1      -1      -1      -1      -1      -1      -1    2192       ·    0131       ·
24      4a - VK_J               1       006a    004a    004a    006a      -1      -1      -1      -1      -1      -1      -1      -1    0133       ·    0132       ·
25      4b - VK_K               1       006b    004b    004b    006b      -1      -1      -1      -1      -1      -1      -1      -1    0138       ·      -1      -1
26      4c - VK_L               1       006c    004c    004c    006c      -1      -1      -1      -1      -1      -1      -1      -1    0140       ·    013f       ·
32      4d - VK_M               1       006d    004d    004d    006d      -1      -1      -1      -1      -1      -1      -1      -1    00b5       ·    00ba       ·
31      4e - VK_N               1       006e    004e    004e    006e      -1      -1      -1      -1      -1      -1      -1      -1    0149       ·    266a       ·
18      4f - VK_O               1       006f    004f    004f    006f      -1      -1      -1      -1      -1      -1      -1      -1    00f8       ·    00d8       ·
19      50 - VK_P               1       0070    0050    0050    0070      -1      -1      -1      -1      -1      -1      -1      -1    00fe       ·    00de       ·
10      51 - VK_Q               1       0071    0051    0051    0071      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1    2126       ·
13      52 - VK_R               1       0072    0052    0052    0072      -1      -1      -1      -1      -1      -1      -1      -1    00b6       ·    00ae       ·
1f      53 - VK_S               1       0073    0053    0053    0073      -1      -1      -1      -1      -1      -1      -1      -1    00df       ·    00a7       ·
14      54 - VK_T               1       0074    0054    0054    0074      -1      -1      -1      -1      -1      -1      -1      -1    0167       ·    0166       ·
16      55 - VK_U               1       0075    0055    0055    0075      -1      -1      -1      -1      -1      -1      -1      -1    2193       ·    2191       ·
2f      56 - VK_V               1       0076    0056    0056    0076      -1      -1      -1      -1      -1      -1      -1      -1    201c       ·    2018       ·
11      57 - VK_W               1       0077    0057    0057    0077      -1      -1      -1      -1      -1      -1      -1      -1    0142       ·    0141       ·
2d      58 - VK_X               1       0078    0058    0058    0078      -1      -1      -1      -1    00bb       ·      -1      -1      -1      -1      -1      -1
15      59 - VK_Y               1       0079    0059    0059    0079      -1      -1      -1      -1      -1      -1      -1      -1    2190       ·    00a5       ·
2c      5a - VK_Z               1       007a    005a    005a    007a      -1      -1      -1      -1    00ab       ·      -1      -1      -1      -1      -1      -1
52      60 - VK_NUMPAD0         0       0030       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
4f      61 - VK_NUMPAD1         0       0031       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
50      62 - VK_NUMPAD2         0       0032       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
51      63 - VK_NUMPAD3         0       0033       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
4b      64 - VK_NUMPAD4         0       0034       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
4c      65 - VK_NUMPAD5         0       0035       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
4d      66 - VK_NUMPAD6         0       0036       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
47      67 - VK_NUMPAD7         0       0037       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
48      68 - VK_NUMPAD8         0       0038       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
49      69 - VK_NUMPAD9         0       0039       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
37      6a - VK_MULTIPLY        0       002a       ·    002a       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
4e      6b - VK_ADD             0       002b       ·    002b       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
4a      6d - VK_SUBTRACT        0       002d       ·    002d       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
53      6e - VK_DECIMAL         0       002e       ·    002e       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
35      6f - VK_DIVIDE          0       002f       ·    002f       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1
27      ba - VK_OEM_1           0       003b       ·    003a       ·      -1      -1      -1      -1    00b0       ·      -1      -1    00b4@      ·    02dd@      ·
0d      bb - VK_OEM_PLUS        0       003d       ·    002b       ·      -1      -1      -1      -1    00ac       ·      -1      -1    00b8@      ·    02db@      ·
33      bc - VK_OEM_COMMA       0       002c       ·    0027       ·      -1      -1      -1      -1    003c       ·      -1      -1    2015       ·    00d7       ·
0c      bd - VK_OEM_MINUS       0       002d       ·    005f       ·      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1    00bf       ·
34      be - VK_OEM_PERIOD      0       002e       ·    0022       ·      -1      -1      -1      -1    003e       ·      -1      -1    02d9@      ·    00f7       ·
35      bf - VK_OEM_2           1       00e9    00c9    00c9    00e9      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1    02d9@      ·
28      c0 - VK_OEM_3           1       00e8    00c8    00c8    00e8      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1    02c7@      ·
1a      db - VK_OEM_4           0       005e@      ·    00a8@      ·      -1      -1      -1      -1    0060@      ·      -1      -1      -1      -1    02da@      ·
2b      dc - VK_OEM_5           1       00e0    00c0    00c0    00e0      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1    02d8@      ·
1b      dd - VK_OEM_6           1       00e7    00c7    00c7    00e7      -1      -1      -1      -1    007e@      ·      -1      -1    007e       ·    00af@      ·
29      de - VK_OEM_7           0       002f       ·    005c       ·      -1      -1      -1      -1    007c       ·      -1      -1      -1      -1    00ad       ·
56      e2 - VK_OEM_102         1       00f9    00d9    00d9    00f9      -1      -1      -1      -1      -1      -1      -1      -1      -1      -1    00a6       ·

0x00b4  25
        0x0020  0x00b4
        0x0061  0x00e1
        0x0041  0x00c1
        0x0063  0x0107
        0x0043  0x0106
        0x0065  0x00e9
        0x0045  0x00c9
        0x0069  0x00ed
        0x0049  0x00cd
        0x006c  0x013a
        0x004c  0x0139
        0x006e  0x0144
        0x004e  0x0143
        0x006f  0x00f3
        0x004f  0x00d3
        0x0072  0x0155
        0x0052  0x0154
        0x0073  0x015b
        0x0053  0x015a
        0x0075  0x00fa
        0x0055  0x00da
        0x0079  0x00fd
        0x0059  0x00dd
        0x007a  0x017a
        0x005a  0x0179

0x02dd  5
        0x0020  0x02dd
        0x006f  0x0151
        0x004f  0x0150
        0x0075  0x0171
        0x0055  0x0170

0x00b8  17
        0x0020  0x00b8
        0x0063  0x00e7
        0x0043  0x00c7
        0x0067  0x0123
        0x0047  0x0122
        0x006b  0x0137
        0x004b  0x0136
        0x006c  0x013c
        0x004c  0x013b
        0x006e  0x0146
        0x004e  0x0145
        0x0072  0x0157
        0x0052  0x0156
        0x0073  0x015f
        0x0053  0x015e
        0x0074  0x0163
        0x0054  0x0162

0x02db  9
        0x0020  0x02db
        0x0061  0x0105
        0x0041  0x0104
        0x0065  0x0119
        0x0045  0x0118
        0x0069  0x012f
        0x0049  0x012e
        0x0075  0x0173
        0x0055  0x0172

0x02d9  10
        0x0020  0x02d9
        0x0063  0x010b
        0x0043  0x010a
        0x0065  0x0117
        0x0045  0x0116
        0x0067  0x0121
        0x0047  0x0120
        0x0049  0x0130
        0x007a  0x0017
        0x005a  0x017b

0x02c7  19
        0x0020  0x02c7
        0x0063  0x010d
        0x0043  0x010c
        0x0064  0x010f
        0x0044  0x010e
        0x0065  0x011b
        0x0045  0x011a
        0x006c  0x013e
        0x004c  0x013d
        0x006e  0x0148
        0x004e  0x0147
        0x0072  0x0159
        0x0052  0x0158
        0x0073  0x0161
        0x0053  0x0160
        0x0074  0x0165
        0x0054  0x0164
        0x007a  0x017e
        0x005a  0x017d

0x005e  25
        0x0020  0x005e
        0x0061  0x00e2
        0x0041  0x00c2
        0x0063  0x0109
        0x0043  0x0108
        0x0065  0x00ea
        0x0045  0x00ca
        0x0067  0x011d
        0x0047  0x011c
        0x0068  0x0125
        0x0048  0x0124
        0x0069  0x00ee
        0x0049  0x00ce
        0x006a  0x0135
        0x004a  0x0134
        0x006f  0x00f4
        0x004f  0x00d4
        0x0073  0x015d
        0x0053  0x015c
        0x0075  0x00fb
        0x0055  0x00db
        0x0077  0x0175
        0x0057  0x0174
        0x0079  0x0177
        0x0059  0x0176

0x00a8  13
        0x0020  0x00a8
        0x0061  0x00e4
        0x0041  0x00c4
        0x0065  0x00eb
        0x0045  0x00cb
        0x0069  0x00ef
        0x0049  0x00cf
        0x006f  0x00f6
        0x004f  0x00d6
        0x0075  0x00fc
        0x0055  0x00dc
        0x0079  0x00ff
        0x0059  0x0178

0x0060  11
        0x0020  0x0060
        0x0061  0x00e0
        0x0041  0x00c0
        0x0065  0x00e8
        0x0045  0x00c8
        0x0069  0x00ec
        0x0049  0x00cc
        0x006f  0x00f2
        0x004f  0x00d2
        0x0075  0x00f9
        0x0055  0x00d9

0x02da  5
        0x0020  0x02da
        0x0061  0x00e5
        0x0041  0x00c5
        0x0075  0x016f
        0x0055  0x016e

0x02d8  7
        0x0020  0x02d8
        0x0061  0x0103
        0x0041  0x0102
        0x0067  0x011f
        0x0047  0x011e
        0x0075  0x016d
        0x0055  0x016c

0x007e  11
        0x0020  0x007e
        0x0061  0x00e3
        0x0041  0x00c3
        0x0069  0x0129
        0x0049  0x0128
        0x006e  0x00f1
        0x004e  0x00d1
        0x006f  0x00f5
        0x004f  0x00d5
        0x0075  0x0169
        0x0055  0x0168

0x00af  11
        0x0020  0x00af
        0x0061  0x0101
        0x0041  0x0100
        0x0065  0x0113
        0x0045  0x0112
        0x0069  0x012b
        0x0049  0x012a
        0x006f  0x014d
        0x004f  0x014c
        0x0075  0x016b
        0x0055  0x016a

Now the code could have been expanded to handle more "special" shift states, though as I mentioned back in Part 9a, this does not seem as immediately important given the fact that there are no other keyboards defined at the moment that make use of this functionality.

Obviously if the code needed to stretch in that direction, it would make sense to take a few additional steps:

For now, I just can't believe how hard it is to get some of this infornation!

 

This post brought to you by "9" (U+0039, DIGIT NINE)
A Unicode character that is in the very small family of those whose VK value is the same as it's code point!


# Joku on 17 Apr 2006 5:40 PM:

Are you sure this works??

I took the code in this part 9b, added the KeysEx enum like instructed.. It always throws exception here:

DeadKey deadKey = new DeadKey(rgKey[iKeyDead].GetShiftState(shiftStateDead, fCapsLock)[0]);

The [0] is set to "" here:

           if (this.m_rgss[(uint)shiftState, (capsLock ? 1 : 0)] == null)
           {
               return ("");
           }

Changing "" to " " helps but since I have little clue to what is going on, I am not certain if that is correct fix to do.

# Michael S. Kaplan on 17 Apr 2006 5:55 PM:

Hi Joku,

It works for me and the keyboard layouts I tested with (and every layout I gave the output for came from such a run).

What layout were you testing with?

# Michael S. Kaplan on 17 Apr 2006 11:51 PM:

More details:

The code in question is called at the top of ProcessDeadKey(), using all of the parameters passed into the procedure from main() any time a dead key was detected.

So for the string to be stored in the array as "" rather than null, it means that a key assignment would have had to have been made that way, which should be impossible.

Knowing which kedyboard you are seeing this in would help me track down what the code is missing....

# Liisachan on 16 Aug 2008 5:54 AM:

The dead key U+02D9 [ ˙ ] is strange when combined with [ z ].

<quote>

0x02d9  10

       0x0020  0x02d9

       0x0063  0x010b

       0x0043  0x010a

       0x0065  0x0117

       0x0045  0x0116

       0x0067  0x0121

       0x0047  0x0120

       0x0049  0x0130

       0x007a  0x0017 <-- Here

       0x005a  0x017b

</quote>

This dead key adds a dot above, changing [ c ] to U+010B [ ċ ], [ C ] to U+010A [ Ċ ], [ e ] to U+0117 [ ė ], [ E ] to U+0116 [ Ė ], and so on.

So one would expect that [ Z ] becomes U+017B [ Ż ], and [ z ] becomes U+017C [ ż ]; which is indeed true about U+017B but not true about U+017C. You get U+0017 instead of U+017C.

I suspect this is a mistake in the KL design, where someone accidentally typed "017" when he or she had to type "017C", confusing 0 and C or dropping the trailing C or something.

Is U+0017 actually what is supposed to be there? I doubt it, because if you really need to input U+0017 for whatever reason, this KL already supports it as [Ctrl]+VK_W.

# Michael S. Kaplan on 16 Aug 2008 12:48 PM:

That looks like a typo from when I was fixing up the column... :-)

# Liisachan on 16 Aug 2008 10:43 PM:

No, it's officially wrong. Not your typo, but the KL itself, intrinsically.

You can test it by yourself on probably any keyboard, for example on the standard US keyboard, by just typing

[RCtrl]+[.] then [Shift]+[Z]

and

[RCtrl]+[.] then [Z]

after loading and activating the KL in question.

OR you can just check the source code of

http://www.microsoft.com/globaldev/keyboards/kbdcan.htm

which says

U+02D9 Dot Above [...] x0017 [...] x017B

so x0017 is "by design".

Maybe the bug itself is not very surprising. After all, humans make mistakes. Romanian KL coming with Windows XP (0x00000418) is officially wrong too, where they confused U+0163 [ ţ ] and U+021B [ ț ] for example. But I find it strange that nobody complained about this before anywhere on the Internet. Well maybe they did, but I couldn't find any page about it.

dk273 on 20 Apr 2009 3:43 PM:

After changing the 4 lines as indicated in your Post “There is a bug in Part 9b!” (i.e. changing 4 occurrences of “(caps != 0)” to “(caps == 0)”), I still get error found by Joku when processing the keyboard layout: KLID = "00000405" [Czech].

An unhandled exception of type 'System.IndexOutOfRangeException' occurred in KeyboardLayouts.exe

Additional information: Index was outside the bounds of the array.

As in Joku's case, this occurs at the second line of “DeadKey ProcessDeadKey”:

DeadKey deadKey = new DeadKey(rgKey[iKeyDead].GetShiftState(shiftStateDead, fCapsLock)[0]);

Where: iKeyDead = 48 (0x30), shiftStateDead = MenuCtrl, fCapsLock = false

All help gratefully received.

"GetShiftState" returns a empty value and hence index of [0] is out of bounds.

I am running under Windows XP SP3 using VS2008.

klaus on 24 Jan 2010 6:23 AM:

I can get the code to compile and run but  the program never finds any dead keys, no matter what Keyboard LayoutID i input.

I tried 00011009 as mentioned above and my output looks correct exspect no dead keys get listed.  

Naturally i stepped through the code tried a couple of different layout id.  Changed my keyboard on my OS (and restarted VS everytime.)  Still never finds dead keys.

My next guess would be my keyboard itself.  Its a Kinesis Advantage.  

Anyone have any idea?

JaredPar MSFT on 27 Jun 2012 5:25 PM:

I was wondering if you had any information on how to deal with the shift states that are reserved for the keyboard layout driver.  In particular when bits 0x10 and 0x20 are returned in the high byte of VkKeyScanEx.  There appears to be no documentation on how to figure out what virtual keys represent those shift states.  

I devised a fairly horrible way of detecting it via experimentation.  But my method is flimsy at best.  In short, look for any virtual keys which produce no text themselves via ToUnicodeEx, consider them as possible shift state modifiers.  Then take the original virtual key code returned from VkKeyScanEx and apply each possibility in order and see if ToUnicodeEx returns the original char.  This works somewhat OK for single shift states but wouldn't work if VkKeyScanEx return 0x30 in the high bit (wouldn't be able to detect which virtual key was which shift state).  

So far I've only encountered one keyboard layout which returns these modifiers.  It's called neo-layout and is available here.

http://neo-layout.org/

Thanks a lot for this particular blog series.  It's been absolutely invaluable in every project where I've had to do key input processing.  


Please consider a donation to keep this archive running, maintained and free of advertising.
Donate €20 or more to receive an offline copy of the whole archive including all images.

referenced by

2010/09/28 Like one of those standards that can't/won't be fully implemented

2008/01/02 All things being equal, the road on the map is much safer to use

2006/04/22 Getting all you can out of a keyboard layout, Part #10a

2006/04/18 There is a bug in Part #9b!

2006/04/17 Get off my freaking key!

go to newer or older post, or back to index or month or day