Q: Why do the small black squares disappear? Hint: The answer isn't complicated, it's complex...

by Michael S. Kaplan, published on 2010/03/25 07:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2010/03/25/9982164.aspx

Over in the Suggestion Box, William Krick asks:

Hi Michael,

I'm having a strange issue at work that I believe is character set and/or font related but I'm not totally sure which is to blame.

We have one XP SP2 (don't ask) system (our development system) and we use various text editors notepad/textpad to view X12 data files which contain hex characters as delimiters.  The hex characters used are: 1C, 1D, 1E, 1F.  On our development system, notepad/textpad both display the hex characters as small black squares.  This is the behavior we actually want, which is probably the opposite of everyone else on the planet.

Our other system (non-dev) also has Windows XP SP2, but it has MANY more fonts installed (due to Office 2007, I assume).  When we view the same X12 data files in notepad/textpad on this system, the hex characters do not display at all.  No black square, nothing.  The characters adjacent to the hex delimiters just runs together.

Any idea what's going on and if there's any way to fix it?

Interesting to see how people are using Windows sometimes, isn't it? :-)

Interestingly, this is the same issue discussed previously in blogs such as What happens when you involve an unenabled Uniscribe with vertical text, given that Uniscribe doesn't handle vertical text? and IsComplexEnoughForYou? and IsCrAndLfComplex or what?.

When Uniscribe is enabled for text (i.e. when the Install files for complex script and right-to-left languages (including Thai) check box in the second tab of Regional and Language Options is checked), then you will not see square boxes for these or many other characters. But that level of intelligence about Unicode is something that does not occur if the check box is checked....

In the long run, William will want to find a different way to enable these users do their work since the functionality is always on in Vista and later. As I point out in We need to be optimizing for more than just the simple cases, it is best not to put one's performance tests, or Notepad usage, in the "simple" case.

Because users may have very good reasons for the complex cases, too....

William Krick on 13 Apr 2010 3:26 PM:

Thanks for the response.  I knew it had to be something weird that I'd never be able to find on my own.

The reason that the users want to see the "little black boxes" is that they can take a data file, highlight a hex delimiter and do a search-and-replace to a printable character to transform the document into something more human readable.

We developers do have a freeware tool called EDI notepad that works great for viewing X12 EDI files, but most of the end users don't have it.

As a makeshift workaround, we figured out that you can switch the character set in TextPad from ANSI to DOS, which makes the delimiters visible again (at least in TextPad), though they now show up as little arrows and other assorted characters, which I guess is better than all little black boxes.

William Krick on 26 May 2010 3:15 PM:

I've unchecked the Uniscribe box (and rebooted) and the hex characters still aren't showing up as black boxes.  Any other ideas what might be happening?

go to newer or older post, or back to index or month or day