DEP is not affected by locale settings

by Michael S. Kaplan, published on 2006/06/08 03:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2006/06/08/621213.aspx

When I hear someone has a problem involving VB5/VB6 and strings that involves language issues, I can usually make a good guess at what is going wrong....

So the other day, when Christopher used the contact link to ask me a question, I had a notion of what might be happening:

Hello,

I've got an interesting regional settings situation that I thought you might have some knowledge of (there is a question somewhere in here...). I've got an app that makes use of some fancy VB6 code to dynamically execute some assembly code stored in a string (for CPU info stuff). It had worked fine until XP SP2 and all the various DEP issues. I've now fixed them. Or so I thought.

It seems that the DEP problem is fixed on English setups, but if I switch to Thai or Hebrew (the two that I tested, could be others), I get a DEP warning! By switched, I mean change the setting in both Standards and Formats and the non-Unicode Language field.

I don't run any code that is locale/language/etc specific. It should run the same on all scenarios. So my question is: What in the world goes in there that would cause DEP warnings on Thai/Hebrew, but not in English? Does VirtualAlloc/Protect/Free have any code internally that depends on the regional settings? It seems very odd. Any chance the fantastic Kaplan knows what sort of actions are taking place? :) If there is something weird here, there might be more than a few developers that would want to know about it...

Thanks,

-Christopher J. Thibeault

I don't think I'm fantastic or anything, but like I said I think I know what is going on.... :-)

Now the XPSP2 DEP (Data Execution Prevention) functionality does not itself have language-specific hooks in it. As a feature, it just keeps you from executing code that is actually sitting in a data section of a binary (a "feature" that is unique to x86 and which is used in some of security exploits that have been seen in the past related to buffer overruns on the heap).

But any time you are using strings in VB <= 6.0 and you are not taking special care to avoid it you are going to see a lot of string conversion via the default system code page (which is directly affected by the default system locale, also known as the language for non-Unicode programs.

It boils down to the same problem behind the whole Double Secret Unicode thing I have talked about before. Basically, a VB string being passed to a Win32 API, which VB will automatically convert from and to Unicode using the default system code page, andall of that conversion is basically either (a) useless or (b) destructive to the data.

In this case, where a string is being used in a Curland-esque running of assembler, any operation that is potentially destructive of the string is potentially very dangerous, and can certainly do things that were not intended.

It was a good thing DEP was there to protect the computer, when you think about it!

Of course it is hard to answer more specifically what to do here to fix the problem than what the Double Secret Unicode post talks about, but keeping that principle in mind it is usually easy enough to see where Visual Basic might be doing a bit of that evil conversion on data that ought not to be converted....

 

This post brought to you by ଏ (U+0b0f, a.k.a. ORIYA LETTER E)

# dumb programmer on 8 Jun 2006 4:38 PM:

Executing assembly code stored in a string in VB6
WTF ?!?

I've seen some s** done in VB6 but not that far!

# Michael S. Kaplan on 8 Jun 2006 7:54 PM:

Hi dp,

I try not to judge -- I have seen some really brilliant code that used such a technique and some code I was embarrassed to even look at. So who knows whether its good or not without seeing it? :-)

# Christopher J. Thibeault on 8 Jun 2006 10:49 PM:

Hi Michael,

Thanks for the reply. Having read your book (along with Matt Curland's book, one of the few that have a spot on my 'nearby' bookshelf...), this should have been the first thing I thought of, but I eventually found it...

Besides the CPU stuff, I also use this technique for some other performance related hacks, subclassing, as well as for some anti-cracking measures. In an attempt to hide what I am doing to a small degree (cracker roadblocks...), I stored encrypted copies of my ASM bytes at compile time in a string, and then decrypt them at runtime. The decryption is where things went wrong on some locales.

My DEP workarounds were indeed working. The memory I was trying to make executable was executable. But since the decryption failed to happen as planned, the code that was executing was not the code I intended. What ended up in there was just executable enough that it made a jump into the middle of nowhere. Apparently, the middle of nowhere doesn't have execute permissions... :(

Anway, I've got it worked out now, but I do appreciate you taking the time to respond (wow, I made the blog). And yes, you are indeed fantastic. And speaking of fantastic people, do you still keep in touch with Curland? He seems to have vanished...

Thanks again,
-Chris

# Michael S. Kaplan on 9 Jun 2006 1:52 AM:

Matt Curland is alive and well, working at Northface University in Utah. If you look at Matt's book site (http://powervb.com/) there is a link there to email him... :-)

Definitely still a very cool guy (and one of only four people I know of in Utah who I'd want the chance to say hi to again if I was pasing through Utah!).

# Christopher J. Thibeault on 9 Jun 2006 2:42 AM:

Hi Michael,

I knew he was at Northface, but I had seen a newsgroup posting that indicated that he was no longer at the email address. I haven't tried to contact him myself (I'm sure he is a busy guy and I'd rather not bother him unless I absolutely was deperate with a PushParamThunk mod question...), but it is good to hear that he is well.

Thanks,
-Chris
Please consider a donation to keep this archive running, maintained and free of advertising.
Donate €20 or more to receive an offline copy of the whole archive including all images.

go to newer or older post, or back to index or month or day