Challenges behind MSLU: the loader! (Part 1)

by Michael S. Kaplan, published on 2005/01/24 10:34 -05:00, original URI: http://blogs.msdn.com/b/michkap/archive/2005/01/24/359555.aspx

(This post is based on one that was sent to the microsoft.public.platformsdk.mslayerforunicode newsgroup back in June of 2001)

It all started simply enough: the Windows division decided to fund a project for a Unicode Layer for Win9x to help people be able to fully use the NT functionality that had been around all the time and was getting better each version. Obviously, anything that was going to slow down applications on WinNT/Win2K/WinXP was unacceptable. So there had to be a way to make sure this layer (MSLU) did not slow down a Unicode application running on the NT platform.

We started with the DELAYLOAD technology, introduced in Visual C++ 6.0. The purpose of delay loading is (according to MSDN):

"The Visual C++ linker now supports the delayed loading of DLLs. This relieves you of the need to use the Win32 SDK functions LoadLibrary and GetProcAddress to implement DLL delayed loading."

All well and good, right? Well, they quickly found another side effect: if you were never going to call a function at all, then you could delay the loading of the DLL till hell froze over!

And then quickly found yet another cool side effect: if you used their capability to override the loading via a hook function, you could redirect the effort and be sure on the NT platform to call the original API instead of the delayloaded entry point in your other DLL. Cool, right?

Well, we thought so. :-)

Of course, we quickly ran into problems with this approach. Developers might want to use the delayload stuff themselves, for their own DLLs. They might even have custom hooks already! It would not be too friendly to take over a technology and make people choose which functionality they preferred: delay loading or Unicode on Win9x.

Luckily, Microsoft has a lot of smart people in it. After working with one of the long-time devs on the Windows team (Bryan Tuttle) and on the linker team over in VC++ (the "father of the Microsoft linker" and original developer of delay load, Dan Spalding), we had a new solution: the MSLU loader!

(and for the record, all of the credit goes to Bryan and Dan; everything cool I did with the solution was due to the fact that they provided the solution in the first place!)

Now delay load works by dynamically creating the stubs at compile time that are needed for the program, but we did not need to duplicate *that* functionality since the APIs being wrapped are a static list -- no need for dynamic work here. So the MSLU loader is basically contained in the .LIB file and contains all of the information for every single function that is wrapped or stubbed. That is the reason why a DLL that is less than 200K ended up with a .LIB file that is over 2mb -- because it contains a lot of information (it also contains all of the failure stubs so that if for some reason the API cannot be called due to low memory, etc., the failure case would be handled gracefully).

Of course, the size might freak people out: who wants to add 2mb to their binary's size? Don't worry, thats not a problem, either. Remember that most APIs are pretty much identical as far as the linker is concerned, even in failure stubs: functions that take three parameters and return FALSE look the same whether its AddMonitorW, EnumDateFormatsW, or SetLocaleInfoW. What this means is that your debug build (no optimizations) will be a little bit bigger since you are going to have a lot of data that is identical for separate APIs, but if you do any kind of optimization (either for size or for speed) then all of these identical bits of code will be merged such that even large, complex applications only add a little bit of size.

More MSLU loader trivia -- neither 'dumpbin /imports' nor Steve Miller's Dependency Walker can detect the loader, so you have to look for it indirectly. How's that? Well, it is fairly easy to see if every single Unicode API that takes string parameters suddenly disappears from Dependency Walker's tree. You can even use such a technique to look into incorrect unicows.lib integration -- if those imports are on the list then it means you have your .LIB files out of order.

Does it all sound hideously complex? Well, a lot of work got done in the design so that all of the actual complexity is hidden, and all you have to do is link to the .LIB file, which gives you the MSLU loader free and clear (literally free, since it comes with the downloadable Platform SDK!). It made for a very interesting bit of technology (a custom delay loader) hidden away inside another interesting bit of technology (a Unicode layer for Win9x).

All of it would have been impossible without some of the really smart people in VC++ and Windows. :-)

# Serge Wautier on 24 Jan 2005 9:07 AM:

# Michael Kaplan on 24 Jan 2005 9:14 AM:

# Tim Smith on 25 Jan 2005 7:00 AM:

referenced by

go to newer or older post, or back to index or month or day