Ready... set... Reboot!

by Michael S. Kaplan, published on 2005/02/18 13:02 -05:00, original URI: http://blogs.msdn.com/b/michkap/archive/2005/02/18/376225.aspx

"But no, Michael," some say, "if I use the method described in that KB article I am not prompted to reboot!"

Well, that is true. And I was imprecise. You can change the setting at will. You can even say "no" to that reboot dialog that pops up in the user interface of Regional and Language Options. But the default system locale has not changed, either.

The problem has to do with when the data is read from the disk. It is actually read by the NT Boot Loader, in real mode. Now the boot loader does not need the data, but it has to load it early. Because there may be drivers that need to do conversions even before the disk drive is available in virtual mode.

Of course this argument becomes less interesting as time goes on since most drivers are written to use Unicode these days. Because most of the APIs they call are Unicode. No one in a low-level driver wants to take the performance hit of converting their strings to Unicode on a regular basis; its easier to just use Unicode to start with, most of the time. Driver writers are particularly sensitive to performance issues here....

Maybe one day they will cut the cord here and say no to non-Unicode components in kernel mode. But I doubt that would happen any time soon since it is hard to tell people that drivers will not work at all. Especially if they do not ever do anything with strings that need to use any of the benefits of Unicode other than this one performance benefit (in which case the benefit is unfortunately mitigated by doubling the data size requirement).

Plus, changing the default system code page of applications out from under them is pretty problematic anyway, and I would not really want to see all of the potential data corruption bugs that could cause.

It is an interesting problem, one that does its best to defy solutions and runs us head-on into battles of backwards compatibility versus best practices.

I realize that my audience may be really biased on the question of Unicodality of drivers and of kernel mode, but what do people out there think?

Brought to you by "ῷ" (U+1ff7, a.k.a. GREEK SMALL LETTER OMEGA WITH PERISPOMENI AND YPOGEGRAMMENI)
One of the over 50,000 graphic characters on the BMP (Basic Multilingual Plane of Unicode) that will not fit into ASCII)

Anything written for NT-class systems should be exclusively Unicode. Anything written for Win 9x should not be written. So you see, it's not so much about encoding but about getting rid of Win9x =)

Requireing unicode exclusively for drivers is a problem, as you cannot use e.g. DbgPrint() from DISPATCH_LEVEL with Unicode substitutions (the tables are pageable). Pageable code/data cannot be accessed at DISPATCH_LEVEL or above, so using Unicode in DPCs, timers etc gives you a blue screen....

Actually, the ACP/OEMCP are currently loaded by the executive in non-paged pool. :-)

But note that in theory a Unicode driver can still call non-Unicode functions when they want to (povided that the OS will not need to convert what they sednd, eventually).... it just means that by and large they are using Unicode.

I don't know about ACP/OEMCP stuff - I write drivers.
My comment was more that DbgPrint() and friends are not safe to use at elevated IRQL. Until that is fixed, we cannot use Unicode exclusively.
Search MSDN or the DDK for DbgPrint() for details.

The ACP/OEMCP are the code pages that will convert your call to Unicode.

I think you are referring to topics like http://msdn.microsoft.com/library/en-us/ddtools/hh/ddtools/DebugFns_5e11bbcc-adc2-46c0-b371-0e54c50bb2dc.xml.asp :

"The Format string supports all the printf-style formatting codes. However, the Unicode format codes (%C, %S, %lc, %ls, %wc, %ws, and %wZ) can only be used with IRQL = PASSIVE_LEVEL. "

But like I said, you can still call DbgPrint without being Unicode otherwise. And when I am debugging code in NTSD or KD or WinDbg it is easier to look at Unicode strings by looking at the data as WORDs anyway.

The ability to insert Unicode into DbgPrint or KdPrint calls is not nearly as important as making sure the core driver itself uses Unicode.

(received this on in email from someone else)

---------------
Of course the real fix here would be a DbgPrintW and KdPrintW -- Unicode inserts are no problem if the string being inserted into is Unicode.
---------------

This is true.

Now the reason that these functions have nothing to do with the code page tables being pageable (they are not, though some would argue they ought to be).

Ok, clarification time!

Someone from the driver team pointed out to me that although the tables are in NPP, the code itslf is paged. Since the only confirmed user of this comversion code in kernel mode is indeed DbgPrint, et. al., there is not really much of a reason to add the "W" versions.

Of course everyone *ought* be using WPP rather than DbgPrint anyway, right? :-)

"Of course everyone *ought* be using WPP rather than DbgPrint anyway, right? :-)"

Lucky there's a smiley there. To get WPP working you need to jump through a lot of hoops, put up with wildly gratuitous incompatibilities between Win2k and newer OS'es, jump through even more hoops to get it to do anything in NDIS, and then realise that the documentation is minimal to say the least...

There is the kernel of a good idea in WPP, but it's so much harder work that rolling your own from scratch that needs a fair bit of work before it's useful.

I'll stop venting now. :-)

I'm having problem whenever my system restarts or "reboot" my desktop doesn't show up. All I can do is task manager. I have to log on and off at least 10 times until everything shows up. Most of the time, when my screen boots up.. its just blank.. just my wallpaper. Theres nothing... no START.. nothing.. what should I do? sometimes I wait and leave my computer for an hour to see if it will show up.. but it doesn't. I NEED HELP!

Not much to do with international settings in general or the default system locale in particular, right? Not a whole lot I could add (well, other than maybe trying File|Run from Task Manager and running EXPLORER.EXE, I guess). Sorry....

Please consider a donation to keep this archive running, maintained and free of advertising.
Donate €20 or more to receive an offline copy of the whole archive including all images.