It's ultimately your call, but your PowerShell cmdlets really don't need to SUCK this much

by Michael S. Kaplan, published on 2011/07/13 07:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2011/07/13/10185915.aspx


I remember years ago, when the groundswell of opinion that wanted a replacement for CMD.EXE found itself with budget.

And headcount.

And executive sponsorship.

And a code name.

You may heard of it.

Monad.

Now everyone involved had their own reasons for wanting this, but for me and several others like me, there was one basic feature we wanted.

We wanted the ability to support Unicode and complex scripts within Unicode (okay, maybe that's two basic features).

Now fairly early on (though I remember in talking with several old timers that we collectively realized it earlier) it became clear that only the Unicode part was possible in CMD.EXE; the complex script part was never gonna happen there.

Thus was born the Graphical PowerShell.

Now this became my PowerShell, because it supported everything that one could ever want in terms of the display and reading of Unicode text, with the only limitations coming out of scripts built targeting older, non-Unicode type support that had dominated the console for so long.

Unfortunately, this was not the PowerShell of most people.

The vast majority of PowerShell work apparently happens in the CMD.EXE-hosted version of PowerShell.

And the vast majority of PowerShell cmdlets target that least common denominator of non-Unicode, non-complexity that the CMD.EXE-hosted PowerShell has the easiest time supporting.

Have you (the reader of this blog) ever heard of Unicode support in the console, which has been there almost since the birth of CMD.EXE?

These cmdlet authors had not heard of this.

Have you (the reader of this blog) ever heard of writing light-up features that do better when running where better can be done (e.g. in the redirected console, or in the Graphical PowerShell)?

These cmdlet authors had not heard of this either.

Have you (the reader of this blog) ever heard of using terrible features like the console fallback language any time the console was deemed to be unable to support a given language?

Unfortunately, a lot of cmdlet authors who had been napping while those earlier questions came out were bright eyed and bushy tailed when this one became available.

Crap.

And this is a ledge I have been trying to convince cmdlet authors to stop jumping off for some time. Although there has been some success, it isn't nearly as much as I would like there to be.

Now I could blame this on a small team if I wanted to.

But since PowerShell was widely embraced by components needed by administrators all throughout many groups in Microsoft and by many people outside of Microsoft, the only small group of people I could ever hope to reasonably blame is the early group of people tasked with writing up the best practices for PowerShell cmdlets.

I lacked the authority to see the guilty "punished (i.e. laid off or reviews impacted) for the decades of man years worth of cmdlets written both inside and outside of Microsoft that were written incorrectly due to the piss-poor best practices produced. Though the people, through failure of due dilligence, deserved a bit of feeling the pain they inflicted....

The plan currently there in most cmdlets -- to take the fallback language for anytime either the CurrentCulture or the CurrentUICulture has the benefit of keeping them from printing out question marks to a CMD.EXE console using a bitmap font. but with a downside of it being the wrong language and the wrong cultural preferences -- a bad thing to do in ordinary circumstances, but terrible when the platform is able to support so much more (which is more often than anything people are being told in marketing materials how much people can do in the CMD.EXE based technology -- because rather than just showing question marks or square boxes there to push people to move on they'd rather dumb down everywhere.

That is, quite simply, bad engineering.

But if you are a PowerShell cmdlet author, either inside or outside of Microsoft, you have a chance to break that pattern.

Let's inject some good engineering -- and some simple "light-up" work to make the better locations produce better results!

The code, most of which comes from Cunningly conquering communicated console caveats. Comprende, mon Capitán?, over a year ago, with one small change to handle the complex script case, is code that I wish could end up either in some central module to be used by all PowerShell cmdlet authors, or in as many cmdlets as humanly possible, to allow the Graphical PowerShell that you may know of as PowerShell ISE to get the job done for the many many languages that are supported

Or if nothing else you could make sure that your own cmdlets don't suck.  Because they don't have to....

The one small change?

A small addition, an interesting utility function for the sake of the CMD.EXE based world, which still hosts the CMD.EXE-based PowerShell.

If you look at the contrived function that is Main:

    public static void Main() {
        if(IsConsoleRedirected()) {
            Console.WriteLine("You are running in a redirected console.\r\nWrite Unicode via WriteFile and be happy!");
        } else {
            if(IsPowerShellIse()) {
                Console.WriteLine("You are running in PowerShell ISE and can support complex scripts.");
            } else {
                if(IsConsoleFontTrueType()) {
                    Console.WriteLine("No PowerShell ISE, but a TrueType font is selected;\r\nyou can at least display some Unicode in CMD.");
                } else {
                    Console.WriteLine("No PowerShell ISE, no TrueType font; you are limited to one code page.");
                }
            }
        }
    }

Basically there is no built-in IsTooComplexForCmd() method in the CultureInfo class to further figure out what works in the "it's TrueType" case beyond the "some Unicode works" comment, but we can make our own -- using the same technology currently being used to dumb down everyone.

The solution can be easily derived from the following algorithm:

  1. Retrieve the CurrentUICulture.GetConsoleFallbackUICulture().TwoLetterISOLanguageName;
  2. Retrieve the CurrentUICulture.TwoLetterISOLanguageName;
  3. If the result from #1 is either "en" or "fr" and it doesn't match the result of #2, then you have a complex script;
  4. Retrieve the CurrentCulture.GetConsoleFallbackUICulture().TwoLetterISOLanguageName;
  5. Retrieve the CurrentCulture.TwoLetterISOLanguageName;
  6. If the result from #4 is either "en" or "fr" and it doesn't match the result of #5, then you have a complex script;
  7. If either #3 or #6 claims to be a complex script, just call it complex, otherwise don't.

And that's it.

I'll leave writing the most elegant version of that last function based on these seven steps as an exercise for the reader. Anyone want to give it a shot?

Braver developers might just use this code to warn people when they should be using the Graphical PowerShell entirely -- they could simply choose to NEVER fallback, and always support languages fully, but in deference to CMD.EXE they could warn users when the results may not be as good as they could be.

And this bit of code can help you single handedly create PowerShell cmdlets that don't suck for all users!


Yuhong Bao on 14 Jul 2011 12:20 AM:

Again, CMD.EXE don't host the Windows built-in console. CSRSS or CONHOST does.

Michael S. Kaplan on 14 Jul 2011 1:21 AM:

But CMD.EXE hosts the non-ISE PowerShell. Even though something hosts it....

rjcox on 14 Jul 2011 1:50 AM:

> But CMD.EXE hosts the non-ISE PowerShell

Not here it doesn't. No instance of cmd.exe running. I agree with @Yuhong: Win32 console subsystem != cmd.exe.

Of course a s/cmd.exe/Win32 console subsystem/ would still make sense and be more, pedantically, accurate.

Blake on 14 Jul 2011 7:28 AM:

Yuhong Bao & rjcox are exactly correct.   cmd.exe is not special in any fashion, it is just another executable flagged to use the console subsystem, exactly the same as powershell.exe.   No version of powershell has any direct or indirect dependancy on cmd.exe.

Michael S. Kaplan on 14 Jul 2011 8:39 AM:

Hmmm... when I run the "PowerShell" shortcut (not the ISE one) from a default Windows 7 install, it is inside CMD.EXE. That is what I'm referring to here....

Michael S. Kaplan on 14 Jul 2011 1:22 PM:

Put another way -- PowerShell hosted by the default console provider, which for 99.999% of all users in CMD.EXE.

Wow, my first official nitpickers! :-)

Blake on 14 Jul 2011 4:51 PM:

No, I'm not sure what you are seeing that makes you think this, but it is not running "inside cmd.exe".  

To confirm this for yourself, consider using Process Explorer, or tlist, or other diagnostic tool to examine the parent processes.  

When you run an instance of powershell.exe there is no new cmd.exe process involved.   What you will find (on Vista or later) is a new conhost.exe process launched as a child of csrss.exe.   That is what owns the window and manages IO for any console application.  (Pre Vista, csrss.exe did this internally, it was split out for security/stability reasons.)

This isn't a nit being picked, this is a significant misunderstanding of how the console subsystem works.

Michael S. Kaplan on 14 Jul 2011 5:19 PM:

Since the legacy PowerShell window has limitations that are completely identical to CMD.EXE in relation to fonts and complex scripts, and since both dev and test members regularly refer to these as windows as being CMD based, and further since the details you prfovide have nothing to do with the points I was making about font and complex script support, this IS nitpicking.

The diferences, even if true, are irrelevant to the point of the blog, which is about writing for the ISE in order to improve support, and getting away from the legacy PowerShell....

Blake on 14 Jul 2011 6:38 PM:

Choosing to be aggressively misinformed about how the console subsystem works makes everything you do say about Unicode and complex script support within console applications rather suspect.

It is your blog, you are free to promote inaccuracies on it, but it costs you credibility.

Michael S. Kaplan on 14 Jul 2011 6:48 PM:

"aggessively misinformed"

Yes, these two screenshots look like they are from entirely separate components that have nothing to do with each other and share no code.

  

Good point.

Blake on 14 Jul 2011 9:16 PM:

Yes, aggressively misinformed, because the true situation has now been explained several times, both by myself and by previous posters and you are avoiding actually listening.

Nothing in those property pages comes from cmd.exe.  Correlation is not causation.   All the things that are common between cmd.exe and powershell.exe are provided by the console subsystem.  The console subsystem is implemented primarily in csrss.exe and conhost.exe and has supporting code in various pieces of the windows shell and the loader.  All the painting of glyphs on the screen for both cmd.exe and powershell.exe is done by conhost.exe in modern versions of Windows.

X looking just like Y does not imply X is built on Y.  In this case, it means X & Y are both built on Z.

The reason this matters in the current blog entry, is that the owners of cmd.exe couldn't do anything about complex scripts anymore than the owners of powershell.exe could.  Those sorts of changes would need to be made by the folks who own conhost.exe.

ErikF on 14 Jul 2011 9:40 PM:

To the nitpickers: Good boys. Have a cookie and run along.

You kind of failed to see the point of the post: it's not whether PowerShell is hosted under CMD.EXE or not, but that cmdlet authors aren't bothering to make their code work with Unicode! Regarding the, I'll simply note that most cmdlets that I see in regular use don't do a lot of stuff where Unicode is really necessary (the majority of the stuff I see is basically the Registry-whacking or Exchange-modifying variety.) Anything that's remotely interactive tends to get the full non-scripted programming treatment, where proper Unicode support is still very much hit-and-miss.

Blake on 14 Jul 2011 11:17 PM:

Poor Erik, did you get called in just to play defense?  Yes, you're clearly a good boy.

For the record, there wasn't anything that merited comment in the primary point of the post.  I, and presumably the others, understood it and agreed with it.  The repeated confusion about cmd.exe is spreading misinformation.  Doing so from an authoritative blog like this should be corrected.  

I certainly know I won't be reading this blog any longer if Michael continues to show a complete unwillingness to learn.

Michael S. Kaplan on 15 Jul 2011 11:46 AM:

There are a bunch of people I wouldn't mind ever hearing from again....


referenced by

2011/07/18 Pedantic nitpickery, turned up to 11

go to newer or older post, or back to index or month or day