Cunningly conquering communicated console caveats. Comprende, mon Capitán?

by Michael S. Kaplan, published on 2010/05/07 07:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2010/05/07/10008232.aspx

If you are someone who is generally annoyed by my blather then you may skip to the code, otherwise I find it all interesting so I don't mind if you do, as well.....

Ok, so I wrote the blog The real problem(s) with all of these console "fallback" discussions to point how backasswards most of the console code out there really is in relation to available Unicode support.

And I wrote the blog Conventional wisdom is retarded, aka What the @#%&* is _O_U16TEXT? to point out how Unicode support was in there, even though no one thought it was.

Then last month I wrote the blog Anyone who says the console can't do Unicode isn't as smart as they think they are where I pointed out that this stuff can work even in places like .Net where people assume it won't.

Now between these three blogs, I am telling a lot of people they are wrong. Including three prior versions of me, each of whom did not know the knowledge in some or all of these blogs.

Now let's pretend that I hadn't written any of these for a moment.

In fact, let's pretend I hadn't even read any of them.

And that I had never heard of Michael S. Kaplan or Sorting it all Out at all.

Just for a moment.

Now further, pretend that I am just as ornery and non-authoritarian as I actually am usually.

My response to these three blogs would be who the hell is Michael Kaplan?!?

I would look especially at that last blog and point out the flaws in the arguments made here, i.e.

There are so many caveats listed there that they could fill a whole 'nuther blog!

and

Sometimes you will still get question marks, other times you will get square boxes, and only occasionally will you get full support of text.

and

There is not a lot to help you detect which is which!

Those three points alone are all I would need to write all of this off as interesting but not useful for my console applications.

I would demand those blanks be filled in, that the job get finished, or this Michael Kaplan fellow should just quit talking so much. Or writing so much. Or whatever so much.

Today, I am going to try to satisfy that fictional version of me, with a bit of code:

using System;
using System.Text;
using System.Runtime.InteropServices;

public class Test {
    public static void Main() {
        if(IsConsoleRedirected()) {
            Console.WriteLine("You are running in a redirected console.\r\nWrite Unicode via WriteFile and be happy!");
        } else {
            if(IsPowerShellIse()) {
                Console.WriteLine("You are running in Powershell ISE and can support complex scripts.");
            } else {
                if(IsConsoleFontTrueType()) {
                    Console.WriteLine("No PowerShell ISE, but a TrueType font is selected;\r\nyou can at least display some Unicode in CMD.");
                } else {
                    Console.WriteLine("No PowerShell ISE, no TrueType font; you are limited to one code page.");
                }
            }
        }
    }

    internal static bool IsConsoleFontTrueType() {
        IntPtr stdout = GetStdHandle(STD_OUTPUT_HANDLE);
        CONSOLE_FONT_INFO_EX cfie = new CONSOLE_FONT_INFO_EX();
        cfie.cbSize = (uint)Marshal.SizeOf(cfie);
        if(GetCurrentConsoleFontEx(stdout, false, ref cfie)) {
            return((cfie.FontFamily & TMPF_TRUETYPE) == TMPF_TRUETYPE);
        }
        return false;
    }

    internal static bool IsPowerShellIse() {
        uint[] rgpl = new uint[1];
        uint siz = GetConsoleProcessList(rgpl, 1);
        if(siz > 0) {
            rgpl = new uint[siz];
            siz = GetConsoleProcessList(rgpl, (uint)rgpl.Length);
            for(int pid=0; pid < siz; pid++) {
                StringBuilder sb = new StringBuilder(260);
                uint dwSize = (uint)sb.Capacity;
                IntPtr process = OpenProcess(PROCESS_QUERY_LIMITED_INFORMATION, false, (int)rgpl[(int)pid]); 
                QueryFullProcessImageName(process, 0, sb, ref dwSize);
                // Name of EXE is PowerShell_ISE.exe
                if(sb.ToString(0, (int)dwSize).IndexOf("_ise") != -1) {
                    return(true);
                }
            }
        }
        return(false);
    }

    public static bool IsConsoleRedirected() {
        IntPtr stdout = GetStdHandle(STD_OUTPUT_HANDLE);
        if(stdout != INVALID_HANDLE_VALUE) {
            uint filetype = GetFileType(stdout);
            if(! ((filetype == FILE_TYPE_UNKNOWN) && (Marshal.GetLastWin32Error() != ERROR_SUCCESS))) {
                uint mode;
                filetype &= ~(FILE_TYPE_REMOTE);
                if (filetype == FILE_TYPE_CHAR) {
                    bool retval = GetConsoleMode(stdout, out mode);
                    if ((retval == false) && (Marshal.GetLastWin32Error() == ERROR_INVALID_HANDLE)) {
                        return true;
                    } else {
                        return false;
                    }
                } else {
                    return true;
                }
            }
        }
        // TODO: Not even a stdout so this is not even a console?
        return false;
    }

    [DllImport("kernel32.dll", ExactSpelling=true, EntryPoint="QueryFullProcessImageNameW", CharSet = CharSet.Unicode)]
    internal static extern bool QueryFullProcessImageName(IntPtr hProcess, uint dwFlags, StringBuilder lpExeName, ref uint lpdwSize);

    [DllImport("kernel32.dll", ExactSpelling=true)]
    internal static extern IntPtr OpenProcess(uint dwDesiredAccess, bool bInheritHandle, int dwProcessId);

    [DllImport("kernel32.dll", ExactSpelling=true, SetLastError=true)]
    internal static extern bool GetConsoleMode(IntPtr hConsoleHandle, out uint lpMode);

    [DllImport("kernel32.dll", ExactSpelling=true)]
    internal static extern bool GetCurrentConsoleFontEx(IntPtr hConsoleOutput, bool bMaximumWindow, ref CONSOLE_FONT_INFO_EX lpConsoleCurrentFontEx);

    [DllImport("Kernel32.DLL", ExactSpelling=true, SetLastError=true)]
    internal static extern uint GetFileType(IntPtr hFile);

    [DllImport("Kernel32.DLL", ExactSpelling=true)]
    internal static extern IntPtr GetStdHandle(int nStdHandle);

    [DllImport("kernel32.dll", SetLastError = true)]
    static extern uint GetConsoleProcessList(uint[] ProcessList, uint ProcessCount);

    [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Unicode)]
    internal struct CONSOLE_FONT_INFO_EX {
        internal uint cbSize;
        internal uint nFont;
        internal ushort dwFontSizeX;
        internal ushort dwFontSizeY;
        internal int FontFamily;
        internal int FontWeight;
        [MarshalAs(UnmanagedType.ByValTStr, SizeConst = LF_FACESIZE)]
        internal string FaceName;
    }

    internal const uint PROCESS_QUERY_LIMITED_INFORMATION = 0x1000;
    internal const int TMPF_TRUETYPE = 0x4;
    internal const int LF_FACESIZE = 32;
    internal const int STD_OUTPUT_HANDLE = -11; // Handle to the standard output device.
    internal const int ERROR_INVALID_HANDLE = 6;
    internal const int ERROR_SUCCESS = 0;
    internal const uint FILE_TYPE_UNKNOWN = 0x0000;
    internal const uint FILE_TYPE_DISK = 0x0001;
    internal const uint FILE_TYPE_CHAR = 0x0002;
    internal const uint FILE_TYPE_PIPE = 0x0003;
    internal const uint FILE_TYPE_REMOTE = 0x8000;
    internal static IntPtr INVALID_HANDLE_VALUE = new IntPtr(-1);
}

First, note that as with the previous blog I am writing Win32 code in C#, to not only let any C/C++ Win32 developer know what to do, but to allow managed developers to be able to support it right away with no delay.

Now there are three important routines here:

IsConsoleRedirected() returns true when your console has been redirected to a file, which means that you can just write everything out as Unicode and call it good.

And when that function returns false, you can move on to the next function.

Because that next function, IsPowerShellIse(), returns true if you are running in PowerShell ISE, a.k.a. The Graphical PowerShell. If you are, then you have full support for Unicode and complex scripts, and anything your application can produce or try to take in. You do not have to be writing in PowerShell ISE to get that support; a Unicode console application can do all the work here, taking advantage of running in this cool modern host, where IMEs and other input methods work irregardless of systm locale and so on.

And when that function returns false, you can move on to the last function.

Because that last function, IsConsoleFontTrueType(), returns true, it means that you are in CMD.EXE but with a TrueType font set. And that means you can display any character the font supports and that the square box that is displayed for the rest of the characters can be copied and pasted somewhere.

And if all three functions return false and you are truly limited to code pages, then you can do what you would have done if none of this support existed.

With all of this support, you can write some kickass and cool console applications that will in every case do the best that the platform can offer the user of the application. Every time....

Now there are other fancy things you (or I!) might want to do like try to change some of those answers, but for now we're going to pretend you are trying to live within the environment you are given. Perhaps some other time all of you non-J. Alfred Prufrock types like myself who would dare to disturb the universe can exercise that bit of psyche and try to change some of the answers that can be changed....

:-)

Random832 on 11 May 2010 7:36 AM:

Michael S. Kaplan on 11 May 2010 11:03 PM:

Random832 on 12 May 2010 8:13 AM:

Michael S. Kaplan on 12 May 2010 8:44 AM:

referenced by

go to newer or older post, or back to index or month or day