The so-called Ultimate Keyboard

by Michael S. Kaplan, published on 2005/07/20 22:14 -07:00, original URI: http://blogs.msdn.com/michkap/archive/2005/07/20/441227.aspx

It all started on Friday, July 15th, at 2:43am. I received the following mail:

Well, I'm not a celebrity or anything like that. I had not yet at that point seen the info on this keyboard.

Roughly 53 minutes later, someone had posted it to the Unicode List -- with the link and not much else.

I'll give a smattering of the feedback after that....

JC Helary asked:

And what about languages that work with syllable input ?

First letter input: the keyboard displays something like qwerty, second letter input: the keyboard changes to what is possible in combination with the first letter ?!?!?

JC Helary

ps: the concept itself is very interesting, also the fact that the keyboard was a Mac one.

And then Don Osborne stated:

... and all of this without footpedals or levers.

The implications are significant and not only for extended and non-Lestern scripts. Once you are no longer constrained by what is painted on the keys at the factory, a lot of possibilities are opened up in addition to facilitating multilingual use of any given keyboard.

Ultimately a post-QWERTY world? Well if the keyboard is not dedicated to one layout, even users of one language are not obliged to learn and stay with the legacy system. I.e., it could facilitate learning and use of alternative layouts such as Dvorak for English without requiring a hardware change away from the legacy layout.

Don Osborn
Bisharat.net

I weighed in at this point, myself, with a point I have mentioned here:

All of this implies that one cannot have a keyboard layout that does not match the letters painted on the keys. Which is of course false (although some people have trouble with the concept, even though they have no probem with shift states that may vary from those letters....),

To which Don replied:

Hi Michael. I agree the problem with the painted keys is not what's under them of course, but what one is in the user's head (or isn't, or can't be kept in it...). Remembering which key was reassigned or what such and such key means with Alt/Ctrl, etc., may be easy when it's your own setup. But if you have more than one keyboard, or one with a lot of modifications, or an unfamiliar layout, or are a person altogether new to keyboards, it may be difficult to remember or keep track of. Hence the LED keys can be a great aid.

Re Dvorak, yes one can select it easily. But if you are learning typing, it's a whole lot easier if you can check the keys (or at least don't have the "wrong" layout looking back at you). One thing going against movement to Dvorak - not that I have a thing about that but some do - is that it's not ideal to learn/use a layout different than what you see, and no one will buy a Dvorak over a QWERTY since so many others have learned the latter. The LED key keyboard in effect levels the field because the keyboard can visually represent any layout you select.

The changing visual key sequences idea that Jean-Christophe asked about is another interesting dimension.

Don

JC Helary then responded:

I was specifically thinking about Japanese input. And I am curious how that would be implemented. Of course, it is not so much the sillabic part of the input that is a problem (there are Japanese kb layouts that assign one syllable to one key, even most Japanese users type with combinations of qwerty  letters.) It is rather the fact that the qwerty input part is converted 2 times: first time when the second letter is associated to the first to produce a Japanese kana, second when the kana (on its own or associated with other kanas) is converted to a kanji.

JC Helary

If you understand how the IME works, then you will understand how at this point, the wheels have started to come off the wagon. Yes, it is true that you enter Kana and then a candidate list comes up. But the list expects you to use arrow keys or similar methods to select items. There is never a time that the changing keyboard will have an opportunity to show the candidates because the various letter keys on the keyboard are not where one selects the Kanji -- that is where one types more kana. I am not going to knock this keyboard just yet, but let's not give it powers it cannot possess!

Don then put the conversation back on track:

Another possibility that came to mind was Ethiopic.

In any event the potential to readily view - on the keys themselves - alternative layouts, extended and non-Latin characters, and to have it respond to input in cases like you describe could be a great equalizer in perceptions of scripts and their value/utility. Of course it's a practical thing, first and foremost, with specific advantages to many users, but it's also a matter of psychology (visual information being so critical in our perceptions). And it's in the latter that the impact could be revolutionary. (Of course it's also possible that other technologies will overtake it in terms of reshaping how we input text in various scripts, but I won't take that tangent...)

Anyway, this is the last I'll post on this topic until we hear more from the keyboard designers or someone who has a chance to evaluate the prototype.

Don

MJ then added an interesting tidbit but gave no explanation:

Dear all,
We are already using such keyboard for Bangla for 18 years.
Thanks
MJ

George W. Gerrity then weighed in:

I have been waiting for such a keyboard for over 20 years. I even tried to get one with LED matrices on keys, manufactured to my specification about 15 years ago, with no luck (at least, at a cost I could afford).

I am surprised that there is even some uncertainty as to how such a keyboard might be used for non-latin scripts. Consider, for instance, its use with Simplified (mainland) and Traditional (Taiwan) Chinese and Japanese: In all three cases, there exist four or five commonly-used input methods designed to be used with a standard QWERTY keyboard, that depend on remapping keys. The only alternatives are expensive, huge (largely unstandardised) keyboards originally designed for typesetters.

Users who normally enter text in Chinese and Japanese have no trouble with the remapping because usage patterns develop for touch typing just as for users of QWERTY keyboards for Latin characters. The problem arises for users like myself, who occasionally want to enter Chinese, Hebrew, Greek, etc, and simply can't remember which keys go with which method.

Having such a keyboard, whose visual indicators can be programmed to change with input method and with shift, opt and alt keys, would be a real boon.

George

Ken Whistler had some additional thoughts about problems with trying to use this keyboard, especially with IMEs:

The whole concept of moving your hands *off* the keys to *read* the keys, and then back on the keys to key your input -- particularly for complicated IME's that change the state of input as you go -- strikes me as at best un-ergonomic and at worst frustrating and inefficient.

And Raymond Mercier extended this thought:

We are more bound by the tactile experience of the keys than we imagine, even those of us who are not real touch typists. I tried a new keyboard recently (mis-designed by Belkin) that had the unfortunate characteristic that the character appeared on the screen only after the finger was lifted from the key, instead of appearing on the downstroke of the key. It was quite impossible to type at normal speed. That experience rather suggests to me that normal typing would be very frustrated by having to pay so much visual attention to the keys.

And Gregg Reynolds responded to these thoughts:

Depends on how often you need to do it.  I can touch type on an Arabic keyboard, but there are always those rarely-used keys that I can't quite remember.  In that case, it would be a big time-saver if the keyboard could change dynamically from English to Arabic.

I think it would be hugely useful for learning e.g. Vi.  Assuming that is that it can be programmed to do something like:  hold down shft-ctl-z, and little arrows pop up on the h/j/k/etc keys.  Or in emacs, hold down some key combo and some kind of mnemonics show up indicating e.g. ctl-f for forward-char.  Much easier than trying to remember where such info is in online help.

If they can keep the price point somewhere in the vicinity of ordinary keyboards they will own the corporate marketplace, or at least that segment whose workers use multiple languages.  I have the Arabic keyboard memorized, so I don't generally need the letters printed, but
most of my colleagues covet such a keyboard, even if it only uses stickers.  They could just print out an image of an Arabic keyboard and pin it next to the monitor, but they want the info on the keys.

I would think a dynamic keyboard display could do the same (in principle).

Now in the past I have mentioned the cool soft keyboard that the Tablet PC version of Windows XP has, which repaints its "keys" based on the chosen keyboard layout. It works quite well with the idea Gregg is talking about.

It really is impossible to know how a keyboard will work here, though I suspect that many of the problems related to fingers having to be lifted to see the characters are really just the tip of the iceberg. But before I get into that, Suzanne E. McCarthy covered some thoughts on this new keyboard in her blog (abecedaria):

The Optimus Keyboard

Optimus Update

A Post QWERTY World (contains many of the same comments I quoted above)

Now, where was I? Oh yes, the tip of the iceberg.

The code that one must write to interrogate a keyboard layout using Win32 keyboarding APIs in even a single shift state is non-trivial. Now add to that a need to keep running that code on a per-thread basis (since the input language is indeed a per-thread setting across the whole operating system), and the fact that this would have to be user-mode code (the kernel APIs to do all of these interrogations do not really exist). Therefore, to work reliably on Windows, one would probably require a low level keyboard hook that would grab any keystroke made (to know when to change for shift states or dead keys).

Yikes, the performance implications of having to do all of that on a per-thread basis any time one switches to different UI threads is somewhat frightening. Though the code complexity issues also seem rather formidble. I actually know a few things about the code needed here (I had to write it for MSKLC's File|Load from existing... functionality. How else to interrogate a layout? But the notion of having to do this as often as the hardware would have to (and having to integrate inyerim states like dead keys and possibly even typograophical ligatures if one were truly adventurous!) is quite a challenging one.

One day I might go through the full process here to help give some appreciation of the level of work. :-)

And this is just for Windows. The work for any supported platform would likely not match other platforms.

Perhaps there are plans to do this differently -- perhaps interrogating the keyboard layout DLL directly. Not impossible but also not trivial. And hard to manage across all of the possible keyboard layouts.

My prediction (well, my guess) about the future of the keyboard would be that it will not end up being completely dynamic. It will just show the base state (like the hardware keyboard does now), and it will switch when the regular keyboard layout does. A lot like what the Tablet PC keyboard does now, which gives a good balance of the performance versus functionality issues.

This post brought to you by "ᄧ" (U+1127, a.k.a. HANGUL CHOSEONG PIEUP-CIEUC)

# Dean Harding on Thursday, July 21, 2005 1:41 AM:

# Dean Harding on Thursday, July 21, 2005 1:45 AM:

# Rosyna on Thursday, July 21, 2005 2:05 AM:

# Michael S. Kaplan on Thursday, July 21, 2005 2:10 AM:

# Rosyna on Thursday, July 21, 2005 2:49 AM:

# Martin Bohring on Thursday, July 21, 2005 2:54 AM:

# robdoyle on Thursday, July 21, 2005 3:05 AM:

# Centaur on Thursday, July 21, 2005 11:06 AM:

# Michael S. Kaplan on Thursday, July 21, 2005 11:42 AM:

# gigi on Thursday, July 21, 2005 5:54 PM:

# Dean Harding on Thursday, July 21, 2005 8:06 PM:

# Dean Harding on Thursday, July 21, 2005 8:46 PM:

# gigi on Friday, July 22, 2005 2:49 AM:

referenced by

go to newer or older post, or back to index or month or day