by Michael S. Kaplan, published on 2010/05/14 07:01 -04:00, original URI: http://blogs.msdn.com/b/michkap/archive/2010/05/14/10004255.aspx
An interesting hypothetical came up the other day.
Let's say that the Windows 7 install base is around 100 million users.
It isn't exactly that, but it makes the percentages easier so we'll go with it since I am going to throw some percents out for you to consider.
And I am not trying to give you overly complicated math problems.
Get it? I am doing ths FOR YOU.
Anyway....
Let's say you found a bug in a specific calendar. You know, in one of these calendars:
Calendar identifier Meaning 1 CAL_GREGORIAN Gregorian (localized) 2 CAL_GREGORIAN_US Gregorian (English strings always) 3 CAL_JAPAN Japanese Emperor Era 4 CAL_TAIWAN Taiwan calendar 5 CAL_KOREA Korean Tangun Era 6 CAL_HIJRI Hijri (Arabic Lunar) 7 CAL_THAI Thai 8 CAL_HEBREW Hebrew (Lunar) 9 CAL_GREGORIAN_ME_FRENCH Gregorian Middle East French 10 CAL_GREGORIAN_ARABIC Gregorian Arabic 11 CAL_GREGORIAN_XLIT_ENGLISH Gregorian transliterated English 12 CAL_GREGORIAN_XLIT_FRENCH Gregorian transliterated French 23 CAL_UMALQURA Windows Vista and later: Um Al Qura (Arabic lunar) calendar
We will assume for the sake of argument that we are not talking about the first, second, or third entries in the table. Since we aren't..
It is not an algorithm problem, just something displaying kind of wrong. Not exactly a misspelling. And not a typo. But if it had been something like that, the results would be similar.
Just so you can screw your head on straight about the problem. I'm not asking you to fix the oil spill here.
So if you wanted to ship an update to fix the bug but you figured only a specific (small) percentage (I won't give the actual number, assume it is less than 5%) of the 100 million were going to be affected, how would you detect what users are most likely to be impacted?
Bear in mind all of the following data points:
So now for the hypothetical:
What would you recommend here?
1) Would you recommend deploying the fix?
2) If the answer to #1 is YES, would you limit the number of machines who would be offered the fix?
3) If the answers to #1 and #2 are YES, how would you craft the limit, exactly? (be sure to properly balance accuracy of hitting the target market and simplicity of the fix to minimize flaws in the design of the filter!)
4) Would any of the above answers change if the impact were not 5% but instead 10%? 20% How about 1%?
5) Can you name both sources of the quote I modified for the title, without looking them up anywhere?
I have obfuscated enough details in the above to make it a proper hypothetical (plus everything is already figured out, so no opinion could weigh any actual decision to be made!) and although I am asking you what you would do as if you are the lone person deciding, in reality there is no one person who makes this call and even the most expert person would be a single voice that would have to have any opinion weighed in terms of how well that person could back up the opinion to a triaging group. And how well they could get people to pay attention, of course.
Anyone want to share how they would answer those five questions, especially the first four? :-)
Of course then after you answer that I have to point out that, as it turns out, architecturally the update process cannot detect all of the possible detection vectors listed above, which lead to a sixth question:
6) If the best possible fix is not available, is a less-good fix that will get some hits acceptable? Metaphorically, you could imagine it like testing for H1N1 by testing for other varieties of seasonal flu; you will get some hits, but you will miss some people who definitely have the bug....
Your thoughts on questions 1-4 and question 6 would be appreciated. The answer to question 5 (if you didn't cheat to get it!) would tickle, at least....
Azarien on 14 May 2010 8:01 AM:
If the system's built-in language is affected, or there's any MUI/LIP pack or even a keyboard layout for the language installed, or the current/default/system locale is affected (that includes current custom settings), then make the fix Recommended (if we talk about Windows Update).
Make the fix Optional for everyone else. That would include a number of interested but undetectable users, and all non interested ones.
Make the fix non-optional in subsequent Service Pack, if one is ever released.
And it doesn't matter if this affects 0.01% or 10% of the users.
Mat on 14 May 2010 10:08 AM:
That one is easy enough to answer: If you feel you can ignore the needs of 5 million potentially affected customers, you are obviously not behaving according to your own best interest.
Aaron on 14 May 2010 1:14 PM:
I would not release the fix, at least at first.
Instead, I would release a universally recommended hotfix that can be installed without forcing a reboot, whose purpose is to detect if the flawed calendar is used on the system and set a flag that the update process can check in the future.
Then, when the update process detects the flag, reccommend the actual fix.
Allan on 14 May 2010 9:37 PM:
1) Would you recommend deploying the fix?
As a software developer, yes. Bugs are to be fixed, regardless of the situation.
(As a manager, then probably no. But that is another matter.)
2) If the answer to #1 is YES, would you limit the number of machines who would be offered the fix?
No. Even if the bug does not affect me, I would want it fixed on my systems.
3) If the answers to #1 and #2 are YES, how would you craft the limit, exactly? (be sure to properly balance accuracy of hitting the target market and simplicity of the fix to minimize flaws in the design of the filter!)
N/A.
4) Would any of the above answers change if the impact were not 5% but instead 10%? 20% How about 1%?
No.
5) Can you name both sources of the quote I modified for the title, without looking them up anywhere?
Honestly, no.
Mihai on 25 May 2010 11:44 AM:
1. Yes
2. No
3. No limit. All or nothing. This is the locale support policy in Win 7, so why break it? If there is a worry about deploying to all, make it "pull" for a while (people can go and download), before you "push" it with a regular update (one that would require a reboot, if that's the worry).
4. No difference
5. Don't know :-)
6. I don't understand the need to restrict to some. Is is big? Unsafe?
Random832 on 28 May 2010 6:04 AM:
Missed posting this before comments closed for the upgrade:
All of the entries in the table except for the first two have a higher probability of being used in specific regions of the world than everywhere else in the world (i.e. the countries which the locales supporting them belong to). What do you suppose the percentage in that part of the world is, and the percentage outside that part of the world? They also probably have a higher probability of being used on some localized versions of windows than others.
Also... does the bug apply to dates that are at a specific time of year, and/or does the bug apply to dates on or after a specific point in time in the future? If so, how far in the future is this date (relative to a hypothetical service pack release)?
"I have obfuscated enough details in the above to make it a proper hypothetical" - I can only find one calendar [other than the first in the list, which you've said it's not] that is the default calendar in more than one locale.
Is there a way to apply the patch in such a way as to not require an immediate restart, but to take effect the next time the system is restarted (either by a more important update or by normally restarting the system)?
Question 5... I will assume for the sake of this answer that the two sources can be named (or approximated, or otherwise identified by [say] an IMDB url, if they are too long to fit) with at least one string of no more than 512 characters. There is no such string that the procedure of choosing a random number between 0 and 512 for the length, and then choosing a random character for each element, cannot result in. So I can say, without cheating, that the answer to question 5 is "yes". Now, as to what the sources in fact are? Well, that wasn't the question - in fact, I have no idea what they might be. There's a metaphor about the axiom of choice somewhere in here, I think. Or maybe about quantum computing.
Michael S. Kaplan on 28 May 2010 8:14 PM:
I would tend to count that as "looking them up" and disqualify the entry. :-)
referenced by