Pedantic nitpickery, turned up to 11

by Michael S. Kaplan, published on 2011/07/18 07:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2011/07/18/10187271.aspx

Further, I am expert on the way most things interact with the things I am expert on.

Not an expert on most of those things; just on the way they interact with the thing I am an expert on.

Up until just recently, I had been able to pretty much steer clear of a particular breed of reader -- the nitpicker.

This fairly insane need to make a point, even at the expense of missing the point.

A situation where, to quote myself from another, tangentially related email thread about a similar bug affecting the ISE itself:

“The promise of the Graphical PowerShell is moving beyond the CMD boundaries, and bugs like this betray that promise”

But, as circumstances unfolded, my contrasting of the "ISE-based PowerShell" versus the "CMD-based PowerShell" became what virtually every comment was about.

The fact that many people have the same (apparently incorrect) assumption about the legacy PowerShell being CMD-based is irrelevant.

The fact that the property sheets for the two of them look amazingly similar while the ISE's looks more like a regular app shortcut does not:

And the fact that the main app propsheet for the first two (right down to the "Defaults" and "Properties" entries) also look the same while the third did not have either option or look in any way similar did not:

The fact that the "Properties" and "Defaults" options for the first two that did not exist on the third at all was also not relevant:

And finally, and most relevant to the point of Unicode and complex script support, the fact that the first two behaved identically limitedly useful while the third did not was also, once again, irrelevant.

The only thing that was important was that I was saying that these two different things that acted just like each other, had property sheets that were the same, had right click menus that were the same and shortcuts used in consoles, or anything else, had nothing to do with CMD.EXE but instead was a CSRSS.exe/conhost.exe thing.

Of course a s/cmd.exe/Win32 console subsystem/ would still make sense and be more, pedantically, accurate.

In other words, there was a single mistake here to do with labeling that would make the entire post accurate. A single pedantic misapprehension on my part that has nothing important to do with the point in question.

Now clearly if I were billing myself as an expert on the console subsystem this would have been bad.

But I'm not, I'm just someone who points out bugs to experts in it to do with Unicode (ref: Ā was unexpected at this time).

The same way I'm not an expert in conhost.exe but have helped fix problems and bugs in its TSF input component (contsf) compiled right into it.

The same way I'm not an expert in security but know more about the localizability of account names than some experts in that.

Or that I'm not an expert in a number of things, though in explaining how they work with Unicode or with internationalization or localizability or keyboards or whatever, I am.

Funny how until I got to the PowerShell experts (who were sick of dealing this CMD.EXE mistake long before they got to me) did I become such a target for pedantic nitpickery!

And not from team members for PowerShell, either. Or any of the other internal folks I've been dealing with. Perhaps since when I am talking to them I am usually helping them with an actual bug, they see no purpose or reason in correcting me on my terminology....

Now it would be my hope that most people will suss all of this out correctly, and recognize that not only my lack of expertise on the legacy console subsystem's pieces but my lack of desire to obtain expertise on the legacy console subsystem will simply keep me from finding it important to make such fine distinctions of things that behave identically and neither of which should be used when one can help it.

All console work should happen in the PowerShell ISE, not just the cmdlet work -- since everything behaves better there.

This fact is true whether you are hosted by CSRSS, conhost.exe, CMD.EXE, or Console Fred, or anything else. This whole collection of components that I called "CMD.EXE based" is bad whether or not what I called it was the exact right name.

So, will I update the blog at some point? Maybe. Given that the software behind the Blog can send trackbacks every time, I'm not especially eager to. The comment spooge is there, it just doesn't seem that important....

If anyone has trouble with this still, then they, like commenter Blake, should likely be excited about one less Blog in their RSS reader. This Blog is clearly unsuitable.

I think that reason why all that happened was that your post seemed to be unfair and wrong attack on Powershell and team responsible.

(On second reading it seems attack is on cmdlets, whose authors may or may not be on Powershell team)

In the end it looks like misunderstanding escalating quickly.

You're probably right. One could I suppose attack the PowerShell team a little -- they do have a bug here, and they have been putting out some less than great "best practices". But they are fixing the bug and they are working on communicating the practices. So generally speaking I don't think they are the guilty parties here. Or if they are the restitution is well under way.

I was told offline that the misunderstanding (assumption that legacy PowerShell is in CMD.EXE) is pretty common, at least in part for the reasons I mention in this blog here, in part because both legacy PowerShell and CMD do have the same underlying host doing lots of the work here. And that people are pretty sensitive about trying to correct this mistake when they can (my previous PS team interactions missed this fun because no one acts that way when they're getting assistance, and previous cmdlet authors often assumed I was right anyway -- it isn't an unreasonable assumption, it just isn't true).

Not that it matters too much. As I said to someone earlier today: "since my whole point was “don’t use it”, it’s a bit like complaining that I called something horseshit when it was really from cows. Either way I don’t want to touch it, and the article was about not touching it...."

:-)

the blog keeps eating my comments... (I can tell when it happens, i don't get that nice green banner on the page after going to it, but by then i've lost the text)

"since everything behaves better there. -- A Unicode console app behaves perfectly in the ISE." Even ignoring the problem of _interactive_ console apps, it's not clear how a console application is supposed to use unicode in the ISE. Standard output is a pipe; data written to it is interpreted as OEMCP, and WriteConsole does nothing. I only tried this with a native console application - is the situation better with .NET?

Of course, having read the earlier post now, they were far too strong. While calling the console "cmd.exe" *is* wrong, it was pretty clear who you were referring to (though you *might* get the wrong team on the first call), and you semi-correctly place blame for invalid output at the cmdlet writers. Semi-correctly, because I'm not actually sure that outputting in a fallback langauge is worse than outputting intellegible ? streams or simply denying output - but obviously that would depend on the purpose of the cmdlet.

I don't know how much more simply I can put it, other than...

DWORD n; WCHAR wc; HANDLE hCI, hCO; BOOL r;

hCO = CreateFileW(L"CONOUT$", GENERIC_WRITE,FILE_SHARE_WRITE,0,OPEN_EXISTING,0,0); /* succeeds */

hCI = CreateFileW(L"CONIN$", GENERIC_READ,FILE_SHARE_READ,0,OPEN_EXISTING,0,0); /* succeeds */

r = WriteConsoleW(hCO,L"Hello, World!\n",14,&n,0); /* No visible output, TRUE, 14. */

r = ReadConsoleW(hCI,&wc,1,&n,0); /* Does not return. */

Works great on the console, not so much in ISE.

Attempting to type in the ISE command line simply complains that there is already a command running. There are no other obvious candidates for where one would type to provide input to an interactive console program. (or where one would look for the output that is not showing up in the ISE output window).

Is there something I'm doing that I shouldn't be doing, or something I'm not doing that I should be doing?

The blog ate my comment again... but it was basically a repetition of what I already said: "Standard output is a pipe; data written to it is interpreted as OEMCP, and WriteConsole does nothing." Did you just not believe me? Or is it that since I said WriteConsole (I had UNICODE defined, in fact) you assumed that I meant WriteConsoleA, ignoring the fact that _not displaying anything_ is just as strange behavior even for that?

And anyway I don't understand how other than those "old hacks" you're supposed to get a handle to console output when standard output **is redirected**! (and it definitely is redirected in the ISE). GetStdHandle returns redirected handle (a pipe, which the ISE reads from and displays as OEMCP), not the console.

I was finally able to get it working. WriteConsole[W] does *not* produce any output in the ISE under any circumstance. The only way that worked was to use WriteFile [with the data in UTF-8, UTF-16, or UTF-32] _and_ to explicitly write a BOM [so it doesn't get interpreted as OEMCP] before writing anything else. Using the CRT [e.g. _setmode(_O_U8TEXT)], at least in VS2010, also solves the issue on a real console and redirected files.

Is it possible that whatever code you got working before was taking a code path meant for redirected files when invoked from the ISE?

There doesn't seem to be any hope of interactive input working, whether through ReadConsole[W] or ReadFile (ReadFile works fine in a pipeline, for whatever that's worth)

As I pointed out in those other posts previously, there is no single way that always works -- you need a combination of the different techniques, and then it works. Since I have posted code that *does* work -- even in the ISE -- it is now your responsibility to explain what fails for you when you run that code....