Social engineering in Windows Explorer....

by Michael S. Kaplan, published on 2007/12/08 14:31 -08:00, original URI: http://blogs.msdn.com/michkap/archive/2007/12/08/6708151.aspx


Regular reader Jan Kučera suggested in response to In Case you have problems that you might think are ǸȦȘȚȲ in a comment:

Although this scenario might seem quite NASTY, why does Vista the same thing when copying folders/files beginning/ending with space?

One question is whether to allow user to create such folders/files, but not be able to copy them when they are there is another one I think (which I feel should be supported).

I came to this just when I wanted to copy "Folder" and " Folder" from XP HDD to Vista's one. XP had no problems copying this by dragging in Explorer, but Vista refused. What was the reason for this change?

I am aware this is not a NTFS versioning related problem, though (sorry) (it can by workarounded by copying using the command line).

This is an interesting case, almost a bit of social engineering for end users, similar in spirit to the developer bit of social engineering I have discussed previously in posts like Keeping out the undesirables? and others I have talked about in the past.

Where things that are legal at the lower levels are disallowed at the levels above.

This particular change, I kind of like -- if you ignore the people who test edge cases and the ornery folk like me, who else would ever hit this sort of problem other than somebody making a mistake -- accidentally typing the space bar, a genuine typo?

It is interesting that if you try to create the folder " FOO" in Explorer in Vista it will be created without the space -- so the warnings that may come up are expected.

If you create the folder with the prepended space in CMD and then try to create one without the space in Explorer, the creation succeeds!

So really it is just inside Explorer (and probably most Shell-related file/folder handling functions) that you would be blocked from the extra shenanigans (most likely via a SHLWAPI StrTrim call), though of course the Unicode-savvy among us will find all kinds of spaces in the Unicode Zs (Separator, Space) general category, most of which will not be trimmed by this particular feature -- since in all likelihood it is a StrTrim call that is not listing all of those characters!).

Now I myself like to use fullwidth characters like , , and when I want to use these characters in filenames because they are in titles and I don't want to be hampered by the filesystem rules (this is also a fun way to find non-Unicode applications since they will best fit map to their regular halfwidth counterparts that are illegal in file and folder names.

And I like spotting non-Unicode applications. :-)

Would I be bothered if some future version of Explorer decided to make it harder to create files and folders like that? Probably not.

Would others? Would this keep a person from wanting to upgrade if confusable characters that looked a lot like illegal characters were also blocked? Perhaps it would!

So I tend to think of the Shell behavior as intending to center itself on operations that probably are typos, rather than the ones that are more intentional -- so maybe my strange acts are safe here.

But putting all that aside, it is an interesting philosophical question -- how much to blur the line between the character identity a function as CompareStringOrdinal would understand it as opposed to international domain names, or to a regular user or the CompareStringEx function. Do you stop at the "avoiding typos" level or would you extend it to the "avoiding confusables" level?

And when you answer, keep in mind all of the rules surrounding international domain names -- does your answer change when you think about the creation of folders and files in the non-virtualized directories and files on a web server? Would you like for Explorer to behave differently in that case, to avoid the creation of files on a web server that could not be accessed by others through the web interface?

Of course I don't speak for the Shell team so I have no idea if they would want to expand this behavior or not. Though if they do ask, you might now have some insight into what I'll tell them. :-)

 

This post brought to you by , and (U+ff0f, U+ff1a, and U+ff3c, aka FULLWIDTH SOLIDUS, FULLWIDTH COLON, nd FULLWIDTH REVERSE SOLIDUS)


# Zooba on Saturday, December 08, 2007 8:03 PM:

Am I the only one who sniggered at "fullwidth colon"?

Growing up appears to be on my to do list...

# Dean Harding on Saturday, December 08, 2007 9:24 PM:

I think the problem is not when *creating* the files -- explorer in XP will silently trim spaces before and after the name as well -- the problem is when you go to copy such files that you've created through some other means.

In Windows XP, you can copy " Foo" and "Foo" to the same folder by dragging-and-dropping. In Vista it thinks they've got the same name and asks whether you want to overwrite one with the other.

# Michael S. Kaplan on Saturday, December 08, 2007 9:34 PM:

My Vista doesn't do that at all for me in creation if the "space" version is created from CMD.

So far every behavior I have seen looks right. I do get offersto merge folders when they are the same, but never an overwrite request.

Perhaps someone could provide the reproducible steps if they say something different?

# Dean Harding on Sunday, December 09, 2007 1:30 AM:

OK, here's my steps:

From the command prompt, type:

echo test1 > "hello.txt"

echo test2 > " hello.txt"

Then, in explorer, drag those two files to a different folder. I get the "There is already a file with the same name in this location" prompt. If you choose the "Copy & keep both files" option, it copies one as "hello.txt" and the other as "hello (2).txt" -- no space.

# Dean Harding on Sunday, December 09, 2007 1:31 AM:

The point is that this is different to explorer in XP. If you drag the two files to another location in XP, it just copies/moves them without complaint.

# Michael S. Kaplan on Tuesday, December 11, 2007 1:04 PM:

Igor is unhappy about new behavior that reportedly isn't new? :-)

I can repro with Dean's steps, though I don't see this change as all that bad, myself -- the fact that you can't create them in Explorer makes this an edge case for most people....

# Dean Harding on Wednesday, December 12, 2007 6:24 AM:

I agree it's an edge-case, and to be honest, I didn't even know about it until I read your post. I was just making sure we understood what Jan was talking about :-)

I'm not sure what Igor is talking about either. The behaviour he describes has not changed since (at least) XP. Maybe it was true in 2000 but I don't have w2k to test on anymore... I prefer the stripping space thing myself, because it's usually UNintentional for me. What's wrong with using numbers to order things in a list?

# Igor Levicki on Wednesday, December 12, 2007 3:57 PM:

I thought that explorer in XP allowed spaces before the name but it seems I was wrong.

For me it is strange that the computer doesn't do what you tell it to do.

If I type spaces before the name that is because I want them there. I hate it when the software plays smart on me because usually neither the software nor the one who wrote it are smarter than me.

Software _is_ stupid, here's the most recent example of the top of my head:

Recently I had to troubleshoot bluetooth USB dongle installation problem and I lost enormous amount of time doing it. Installation was failing with the "access denied" error. No explanation what it was denied access to, nothing. setupapi.log, regmon.log, nothing. It turned out that it was failing because it couldn't create a shortcut in the Start Menu!!!

Shortcut was already there and it was read only.

Smart software would do one of the following:

  1. give more sensible error message to user
  2. ask you if it is ok to overwrite read only file
  3. check if the files are byte identical and if so continue
  4. create new shortcut by appending number to the name
  5. complain and offer to create a shortcut on the desktop instead
  6. perform this step at the very end after all hardware was installed

And yet, with so many smarter choices available, it simply died with a cryptic "access denied" message and I spent 5+ days troubleshooting it with a colleague of yours until we finally figured it out.

In my opinion, if space is a valid character in a file/folder name (which it is), then Explorer should not prevent you from using it. Period.

# Igor Levicki on Wednesday, December 12, 2007 4:12 PM:

Dean,

As for numbering, if you use numbers and you make a mistake in ordering things you have more work to do to fix it:

1jkl

2def

3abc

To put 3abc on the second position you need to rename 3abc to 2abc (providing you don't have 2abc already), then rename 2def to 3def (providing that you don't have 3def already).

On the other hand if you had (-=space so you can count them):

---jkl

--def

-abc

Then you would just have to add one space to abc.

# Michael S. Kaplan on Monday, December 17, 2007 2:07 AM:

Hi Igor,

This is a little contrived, in my opinion. It is not realistic to expect a regular user to create such a set of files in Explorer (and if it is done programatically then it will work just fine). So I really have trouble seeing that as a bug....

# Jan Kučera on Tuesday, December 18, 2007 4:19 AM:

Okay guys, my question, so I should give some explanation and thougs.

First of all, I don't mind that Vista's Explorer trimms the name upon creation/rename. You see this immediately and I expect that if you need to create a file/folder beginning/ending with space or whatever, you are experienced enough to use the command line.

Then, I did not do any special kind of trick to find out this behaviour. I was just blindly using 3rd party software as my grandmum would, which creates folders named the same as your project and guess what, it allows you to start your project name with space and does not trim that. Later, when I just wanted to copy whole application's user folder to another location - here we are. Somewhere deep in the structure, the copy failed. :(

Finally I agree with Dean. I mean it is okay for me to not allow occasional user to enter such names, but once they are there, no matter how it happened, do not break the system when manipulating them. Windows allowed me to create these, so they should also allow me to move/copy/whatever them.

Unless however, the file copy API are okay with it, which I can't test now. That way its back up to you - you created them, you have to manipulate them.

However, this topic is more for The Old New Thing then for Sorting It All Out, don't you think? ;-)

Jan

# Michael S. Kaplan on Tuesday, December 18, 2007 4:23 AM:

Yes, technically it might be. But it is still interesting to me, which means is is on-topic for here. :-)

# Jan Kučera on Tuesday, December 18, 2007 6:47 AM:

Then it's all right. :-)

Okay I checked some APIs, and they seem to work properly (both managed and unmanaged), as actually suggests the command line workaround.

So, either the Vista's Explorer does its own checking and trimming before copying/moving (as this does not come from the CreateDirectory's ERROR_ALREADY_EXISTS)(and this seems very strange to me) or uses some API I have no idea of.

Well I don't think I can do much more here, any further ideas?

# Michael S. Kaplan on Tuesday, December 18, 2007 10:34 AM:

The Shell/shlwapi keeps a consistent interface for the stuff it does itself, so it would make sense if move/copy/create all want through a consistent interface.

I sort of see the exception here, though I guess if I had ever run across it myself or somebody gave me a more convincing example a non irregular user like myself ran/runs into I'd be more swayed.... :-)

# Dean Harding on Thursday, December 20, 2007 1:34 AM:

Well, it's still possible for a regular user, depending on what programs they use. For example, say you play a game, and this game lets you type in a name for your savegames, and then it saves them as a file with whatever name you specified. It probably will not trim the spaces from the name you type in. If you then want to copy your savegame from one place to another, you might run into a problem with explorer renaming the file as you copy it.

Perhaps not entirely "regular" but not really in the realms of "irregularity" either...

From what I can see, I think it has to do with the new IFileOperation interface introduced in Vista. Since this new interface handles copy, move, delete, creating new items, renaming items and so on, my guess is that it was HERE that the trimming code was added -- and it was added to ALL operations that IFileOperation can perform. I guess I could try it out by writing a sample myself, but I'm too lazy :-)

# Igor on Friday, December 28, 2007 3:05 PM:

Any file manager (Total Commander, Directory Opus, etc) will also allow you to create files and folders with a leading space. Not being able to access them without having one of those managers installed is inconvenience for the user.

As I said, my main issue with this behavior is -- computer should do what you tell him to do, not what it feels like doing.

# Jan Kučera on Tuesday, February 12, 2008 11:51 AM:

Dean, this is exactly the repro how I came to the problem. Thank you for the IFileOperation hint, as I did not have the success with shlwapi either. Though, being not a C++ developer (yet) I cannot test IFileOperation myself. ;-)


go to newer or older post, or back to index or month or day