An interview question (by popular demand)

by Michael S. Kaplan, published on 2005/02/28 02:09 -05:00, original URI: http://blogs.msdn.com/b/michkap/archive/2005/02/28/381465.aspx

The other day in Not all interview questions are created equal I explained why I did not personally care much for the types of questions that Scott Hanselman gave in his post (What Great .NET Developers Ought To Know (More .NET Interview Questions)). I then blathered for a bit about what I try to not do in interview questions, and never said much about what I did try to do. Some people rightfully pointed out that it is not entirely cool to say I disagree with someone without at least a concrete example of something that I think is a good question....

So, I thought I would give an example of one I have used before (though probably won't use again, sorry! Staying awake while reading this blog is impressive but I don't want people to have an edge in interviews just because of they put up with my blatherings!).

I model it after a question that Dave Kaplan (no relation to me!) gave me way back in 1997 in my very first interview of the very first interview loop I ever went through at Microsoft.

You see, Dave (who is no longer at Micrsosoft) was one of the developers who helped design the Intelli-Join feature that is used in many of the Access wizards like the Form Wizard, since Access 95. It allows you to pick fields from different tables all over a database and then using the relationships between those tables would create the most efficient query it could manage, to return the expected results. It is a hard problem to solve at all, a fiendish problem to solve well, and I believe at least one patent came out of the solution that was developed. He had moved to another team at Microsoft, but was very willing to be part of an interview loop for a "wizard developer" (not the term I would have used, myself!).

Now Dave felt reasonably comfortable with his expertise in the area (with good reason), and by giving the original problem suggested above (how to let the user pick fields from different tables all over a database and then create a query to return the expected results), he was comfortable with the knowledge that no one could really "solve" the problem in an hour. It can actually be an excellent question for a candidate who has to do database development that involves UI for filtering raw resultsets, and one that they may not have previously considered. The methods used by a candidate to try to find a solution can often be illuminating.

Of course, there are risks to a 'top down' question like this one, which asks the hard question first, that can work against the person asking questions. It can make a candiate feel like they are failing pretty quickly -- if you put up a huge wall of unfamiliar materials, it is not always clear how to scale it. Definitely it can put the pressure on in an uncomfortable way.

In my case, the question kind of backfired in the other direction. I had actually just written the Partial Replica Wizard, as a proof of concept that such a wizard should be added to Microsoft Access¹. It solves a similar problem in a totally different way (I think I cowrote an article about it in Smart Access explaining the concept, too). I had no access at the time to any of the Intellijoin stuff (if I did I probably would have used it; coming up with clever solutions is not necessary when you have something on the shelf that will get the job done. All I would have lost was a publishing credit that does not even end up in Google Scholar!). But I basically had a solution in my mind that I was able to use here. Technically my solution was easier to implement, though admittedly with fewer features. I laid out my suggested solution in just a few minutes and I think I shocked him. The fact that it was a problem I had solved outside the context of the interview did not phase him; we ended up spending most of the interview walking through the algorithm I suggested as he convinced himself it would work!

(I knew it would work, since I had tested code that used it <grin>)

I am fairly certain I aced that one particular interview, though it could have gone either way, if he were less cool of a guy. There are plenty of people in the world who would take that sort of thing personally. Luckily he was much more about the technology than about ego, so a different solution was just really interesting. It was not something to feel weird about. :-)

I don't think I have ever been able to do that in an interview since then, so it is not the kind of thing I expect is common^{2, 3}. At least it is not for me -- though I'll teach would-be candidates a cool trick in a future column that can be used favorably in interviews, if there is interest....

Anyway, taking that idea of asking a question in an area in which I have some expertise, I instead try more of a 'bottom up' approach -- start the question simply in an area and then make it progressively more complex:

So for example, I tell them about what I do and about how one my foci is about collation. I then have them write a simple string comparison function. Take two strings and return -1, 0, or 1 depending on whether the first string comes before, in the same place, or after the second string.

Easy, right?

Well, yes. But here is where it gets tricky. The question morphs depending on how well people are doing, such that any of the following can then be added:

• Of course all of the important generic stuff like parameter checking and code that compiles, etc.;
• Maybe I'll add the concept of case -- how do you optionally make "A" be equivalent to "a"?
• Or maybe even harder, how do you make sure that even if they are not the same, that they will sort one after the other?
• Perhaps I will suggest adding support for the AE ligature (Æ) -- how can you make it optionally equivalent to "AE"?
• What about the Spanish "ch", that optionally sorts before "d" depending in language?
• How about diacritics -- how best to handle them, both when they are considered to be variants of the base letter and when they are not?
• Other variables like width or kana type?
• String sort vs. word sort?
• What about if you add Unicode to the mix?
• And what happens if you add Unicode equivalency?

Some of these questions lead to code changes, others are just more in the realm of conceptual questions about how they might change the algorithm or what data would be needed to support things, or people to explain how they would approach the problem. And I am sure you can imagine even more beyond these few....

Things that are cool about this type of question, in my mind:

• It is probably one of the more relevant questions I could ask in that it directly applies to how I do a part of my job;
• It can be paced to how well the candidate is doing -- which features I ask for will vary depending on their strengths and weaknesses;
• When I move to a new step they get the positive feedback from the prior step being completed and know they accomplished something;
• Their first function they write will seldom resemble what they finally come up with -- when they modify vs. when they start over is interesting from a design standpoiny;
• Sometimes they will think ahead on certain things to try to design for them (not required but cool to see);
• Candidates will often try to think behind the question (e.g. to underlying data structures, to understand how they would like the data structured to return answers). I definitely encourage this;
• No one can ever really "finish" the question -- if it were a six hour interview there would always be new complications I could add. So it is an open question with many individual closings that can comfortably stop at almost any point.

I have asked similar questions with Unicode, encodings, parsing, formatting, calendars, normalization, casing, string iteration, and more. They seem to work well, and I can usually get a good idea of how a candidate approaches a new technical problem. Folks definitely come in and have to think about the problem, and sometimes come up with quite novel solutions. There has not yet been a case that has impacted actual product development or caused a design to change, but when you talk to smart people, you never know what may come out of the conversation. :-)

I guess that kind of gave you two questions that I consider to be good ones, rather than just one, huh? I have learned about intelli-Join since then, and even owned that code for a few years. Ah well, I have only ever asked a question like Dave's once, usually we are not seeing database developers in this end of the world. And it is not just about having a cool question, it is about having it be relevant to something.

So, would I recommend these questions (or ones like them)? Yes, if (a) you feel you can speak intelligently to what is behind them and (b) they are relevant to either what you do or what the candidate might do if they are hired. But the principle can be applied to anything you are good at (or that applies to the job in question). The principle? RELEVANCY.

1-I demo'ed it to a PM on the Access team as proof of concept and said "if you gave a it a litte time you could even do something lame like this" and they turned around and bought it. I was released on the web for Access 97 and was added to the core product in Access 2000. Only minor changes have ever been made to it since that original release, though there was one cool internationalization bug that I will talk about another time....

2- Another time, I may tell the story about a time when I asked a "top down" sort of question about collation and someone managed to shock me just as I did Dave eight years ago. That person now works down the hall from me, on MUI. :-)

3 - For what it is worth, the interview after that was with a guy who is also no longer at Microsoft. He had a bunch of those questions I love so much (not!) like XORing strings in clever ways that if I ever had received a CS degree or had gone to a bunch of interview web sites I might have done well at. But I hadn't, I hadn't, and so I am pretty sure that I didn't. But he was a tester so I asked him questions about test (a subject about which I knew little at the time) and that made them think I should maybe be trying to be a tester -- a notion I had some difficulty shaking them of, back then. They did eventually make a dev offer a few months later (no hurry since I was under contract anyway), but by then I decided to hang back and perhaps find a group that would have me do something more than VB/VBA code (that is a hard reputation to shake at Microsoft, believe me!). I get a little wistful when I think about what I would have in the bank now had I started fulltime with Microsoft when the stock was where it was in 1997.... but then I realize that I may not have stayed this long had things not happened as they did, and then I get over myself. Even oldtimers at MS get wistful about what they did with stock options during the 90's. :-)

This post brought to you by "ส" (U+0e2a, a.k.a. THAI CHARACTER SO SUA)

# Sriram on 28 Feb 2005 4:01 AM:

# Michael Kaplan on 28 Feb 2005 7:15 AM:

referenced by

go to newer or older post, or back to index or month or day