Code as though the Console is as it should be, to show everyone what it can be

by Michael S. Kaplan, published on 2010/12/03 07:01 -05:00, original URI:

For those who want today's technical content with no back story, you should do two things: 1) realize you are a very boring person, and 2) skip to the last section of today's blog....

Disclaimer: I have not had a beer with either Phil Lucido or Bryan Tuttle, though I believe I technically owe each of them a beer (at least), if that counts.

Now I am very well aware of the fact that I don't live in an ideal world. And yet, at each point as I go through life, I can imagine what the ideal version of the situation might be like.

Once I do this, I can, by comparing that ideal to how things actually are, do two things:

Much of my life goes this way, and after decades like this, I'm pretty comfortable with it.

There is a quote that I think of, one that inspires me to do more than I do, to be more than I am, from the TV show Angel:

But that's why there's us - champions. Doesn't matter where we come from, what we've done or suffered, or even if we make a difference. We live as though the world is as it should be, to show it what it can be.

Which is not to say I am a hero, or even a champion. Though interestingly enough one of my current roles has me as the point of contact for several people who do explicitly act as champions and are called such -- World-Readiness Champions. And when they come to talk to me about issues I like to point out what the ideal solution might look like, before moving to the one that is more likely to be possible given current resources. I like the context it can give.

If you are a regular reader and you enjoy this blog, then there may be some dimensions by which you are feeling a little of the same.

Anyway, I was thinking of all this not too long ago, when I was talking to someone about a recent blog of mine, Myth busting in the console.

It occurred to me that I had not followed my usual pattern.

So today, I am going to take a step back and fill in those blanks. In the form of an almost fantasy-like vision of what the past could have been.....

The ideal in my mind happened one Friday, fifteen years ago. Somehow I was working on Windows then (though in reality I would not be on Windows until half a decade later).

This ideal involves Phil Lucido (one of the most amazing force that Microsoft's C Runtime has ever known) who was hanging out with the Windows folks at a WIM on a Friday afternoon, having a beer with me and Bryan Tuttle (one of those few people I know who can work at 96% efficiency and who I miss in Windows every time I hear about a build break due to bad merge that no one understands).

Over that beer I am explaining these things about the console that I somehow managed to know then rather than, upset that the various console apps just recently made to work well on the NT platform weren't doing.

Suddenly, we know what to do.

Phil goes back to his office and fixes all of the console support in the CRT to support Unicode. With his interim builds as he works and using a combination of the CRT and the regular Unicode Win32 API, Bryan and I update all of the console apps to support Unicode.

The three of us work all weekend.

By Monday, we have transformed the runtime, and the platform. Every FINDSTR, every TRACERT, every PING, XCOPY, every bit of these pieces of the console use Unicode. Because that's what it should have been doing anyway.

I suspect that even in this fantasy world DaveC isn't especially thrilled that so many tools that are actually still running mostly ASCII text are going to emit text that is twice the size since it will double the size of a lot fo log files. But he doesn't push us to revert the changes, which is good (we were lucky that UTF-8 didn't really exist yet on the platform or he may have pushed to have it all redone to use UTF-8 and keep the ASCII text size down!).

Ok, now this is getting silly, I suppose?

We are looking at the other end if the telescope here -- with me writing a blog titled Myth busting in the console that points out that more than a decade and a half of console applications have been created and used without taking any of that information into account.

Changing it all now (if Phil still worked for Microsoft and Bryan still worked for Windows and the company still had a way for such things to get done by people over a weekend) would pretty much break a ton of applications that depend on the current behavior that aren't expecting such a new behavior.

So okay, knowing how easy it is to support Unicode in the console is the kind of story that could have South Park's Captain Hindsight as the hero, and his power to make us feel better knowing what we could have done would allow us that meager level of empowerment.

But what can we do now?

Ah, I'm glad you asked.

For what I think every console application should do going forward (and what some percentage of the existing ones should be updated to do though of course that is a much harder sell!) is in the next section.

 How to do console apps right:

(source material in Myth busting in the console and the Converting a project to Unicode series)

  1. Within the console window itself, everything coming into the app and going out of it should be UTF-16 LE, via functions like ReadConsoleW and WriteConsoleW. There are no backcompat issues doing this with the exception of those in love with question amrks and other corrupted text, and I don't care about them. At all. There should literally be no way to deal with the console window itself that is not Unicode, Unicode, Unicode, Unicode.
  2. For input and output that uses files, the encoding should be the input and output encoding that have been set.
  3. #2 means for Managed code that it should be able to set it to use UTF-16 LE if it likes, or whatever it feels like. And since they don't do that, people should do what I do in those samples and use Win2 within managed console apps,
  4. #2 means for native code that it should the Win32 API and use those code pages to get the text into/out of the files (and add a single command line switch to tell the app to use UTF-16 since that is the one that you can't set right now).

Such a solution would mean that anyone depending on your application behaving like every other semi-broken app over the last 15+ years will get exactly what they want.

Every application that wants to do better can do exactly that -- it can do better, using your app that is better when it needs to.

Every person who follows these principles is in my eyes a HERO.


How big of a hero or champion is just to do with the quality and the quantity, the depth and the scope.

If you decide you are one of those heroes, those champions, feel free to let me know about it (in comments or via the Contacting Me link!).

no comments

Please consider a donation to keep this archive running, maintained and free of advertising.
Donate €20 or more to receive an offline copy of the whole archive including all images.

go to newer or older post, or back to index or month or day