What broke the input language messages?

by Michael S. Kaplan, published on 2006/05/16 11:40 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2006/05/16/598980.aspx

Yesterday, when I was talking about the role of input language change events and how Even if the message is not for you, you still need to pass it on, I dangled a small morsel at the end:

Now in fairness, the above deterministic process stopped being 100% true all the time starting with Windows XP; this is an issue that is for the most part by design but does not appear to be in any of the documentation. I'll explain what broke the model and why (as well as how to keep things working properly!) tomorrow....

It is not nice to tease the readers, so I will explain what changed in XP and when....

Basically there are two changes that happened, both of which affect Text Services Framework (TSF) members, known as Text Services (defined in the glossary as

An application type that enables text input by way of speech, multilanguage keyboard, or pen device. A text service can also enable text editing tools such as spelling and grammar checking.

What changed is some of the behavior of the Language Bar (which became a fully programmable COM object that is documented in MSDN here).

The specific changes that affect it are:

The guy in charge of TSF on the program management side mentioned to me a few years back that the reason for the change was that there were many applications that were blocking keyboards (either by not passing messages on or by not seeing them on some arbitrary list), which would actually break the support for all of the new text services when the user had specified that they wanted them working everywhere:

(There may or may not a flaw or two in the logic there, but that conversation can wait for another day....)

Now many of the Input Method Editors were converted to TSF Text Services, and of course anyone can write their own (links to several samples here). In Vista several more IMEs were converted and at least two languages that need input methods that are more complicated than a regular keyboard even if less complicated than a full IME also received TSF Text Services.

The upshot? Those two USER messages can no longer be relied on at all, for an unbounded list of possible input languages....

The replacement which will work for both regular keyboards and TSF Text Services is the ITfLanguageProfileNotifySink interface and its two methods:

Now the first method only returns a LANGID and the second returns nothing at all (an HKL is all the old USER messages returned, an HKLs are not needed in the TSF world); if you want to know what the change actually is beyond a LANGID, you can use the ITfInputProcessorProfiles interface and its GetActiveLanguageProfile method. This method will return the GUID that is used to identify the specific text service.

Where that gets you is of course an even more complicated matter -- since there is really nothing publically available other than those nine samples, and none of them cover the bulk of these language bar issues. :-(

Though if you want a nice simple way to get at this stuff (which the documentation explains but does not recommend since you have to do your own refcounting rather than letting COM do it!), you can call TF_CreateThreadManager; this will give you an ITfThreadMgr, from which you can QI an ITfSource. And you can install an advise sink via AdviseSink to get an also handy ITfActiveLanguageProfileNotifySink that will tell you when the active language or text service has changed, via its OnActivated method -- which does not give the language information (so you have to make other calls to find out what was changed to anyway!).

If there was ever an area that needed more samples, this would really be it, I think. While there are a few folks out there like Tavultesoft and Murasu who have puzzled it out, there would be a lot more if there were more samples out there.

And in the meantime, it might make sense to update the docs for WM_INPUTLANGCHANGEREQUEST and WM_INPUTLANGCHANGE to make it clear that they ain't always gonna work right....


This post brought to you by "" (U+a056, a.k.a. YI SYLLABLE BBIT)

# Gabe on 16 May 2006 1:25 PM:

Wow. That TSF is amazing! With 110 different interfaces, I can see why it could use a few more examples.

It does look like it can be used to implement cool things like a system-wide spellchecker. I would *love* to see that as an example.

# Martin Bohring on 17 May 2006 3:41 AM:

Hello Michael,
nice to see that quasi confirmed, because I have been bitten by that one in the past.

It is nice to see IME / TSF based on a more modern foundation (COM), but there are more places where the documentation left something to be desired.

There is a back compatibility layer, but sometimes it just proves that interfaces are leaky (semantical) even if not intended so.

I came to the conclusion, that every interface definition is used in leaky way.

Yuhong Bao on 11 Jul 2011 2:21 PM:

What is funny is that I think the TSF can be installed on Win9x, which pretty much relies on double secret ANSI.

Michael S. Kaplan on 11 Jul 2011 3:45 PM:

It actually relies more on ANSI PLUS than double secret ANSI. :-)

Yuhong Bao on 11 Jul 2011 6:53 PM:

Of course, the breaking of the input language messages only applies to TSF text services, and I don't think the double secret ANSI support was ever extended to IMEs, partly because in Win9x only the DBCS versions had IME support. (NT 4.0 had an Pan-Chinese version that was English with IME support for both simplified and traditional Chinese)

Yuhong Bao on 11 Jul 2011 6:59 PM:

This reminds me of a bug while the original Office 2000 FM20.DLL and older versions would crash if you try to type with an IME that do not match the system codepage. Wonder if it is related to double secret ANSI.

Please consider a donation to keep this archive running, maintained and free of advertising.
Donate €20 or more to receive an offline copy of the whole archive including all images.

referenced by

2007/10/11 Rumor as prophecy in Win32

2007/05/29 Cutting the cord while someone else is shoring it up

2007/04/26 Sprechen Sie IME?

2007/03/20 Double Secret ANSI, part 2 (the brokenest one yet, sorry 'bout that!)

2006/07/13 IME + .NET = Input Madness Editor?

go to newer or older post, or back to index or month or day