The bizarre variation of a skeleton that is iBot + me, to a Kinect

by Michael S. Kaplan, published on 2010/11/15 07:01 -05:00, original URI: http://blogs.msdn.com/b/michkap/archive/2010/11/15/10090877.aspx


I'll start by saying I think Kinect is, generally speaking, very cool.

Just so you know where I stand, generally. :-)

Others have written about the Kinect and wheelchairs before, like in I Connect with Kinect from August 18th.

I have only interacted with the Kinect twice:

Now when I read reviews and see really cool displays like this one, and the possibilities here are very exciting.

I was not given the opportunity to take pictures or video on either occasion, but maybe if I get a chance I'll get some another day!

That meeting in August was a half day talking about the device, and a half day with us trying it out. Most of the time I was "trying it out" various people on the Kinect team were having me try things out on different devices (that were running at least three different builds of the Kinect firmware/software).

I guess they had already tried out the Kinect with wheelchairs, but the iBot really confused the Kinect in every build I tried. The primary reason for this is that the detection has two core 'tracks" or "modes" that it uses -- one standing, and one sitting.

For the iBot, it couldn't figure out which one was right. Because I seemed to be sitting, yet I also seemed to have legs far enough down to the ground that I appeared to be standing. Not to mention my iBot joystick, especially when it was pushed off to one side a bit, seemed a lot like my right arm.

Skeletal tracking was obviously a little confused in this bizarre variation of a skeleton that is iBot + me.

Now the October build I tried a few months later was pretty much the final or close to it. They had actually done some work to fix a lot of the problems with skeletal tracking as it applies to a guy in an iBot.

I guess I should say they "fixed" it, actually. Sincw the fix was to make sure I was always in the sitting mode (they call it the sitting "pipe") and never the standing mode.

This "fix" actually makes a lot of games not work, since many of require standing mode -- even if you are doing something that clearly implies sitting like driving a car.

I've talked to enough people on the team at this point that  I know this could be considered a limitation with two clear architecting causes:

In my mind, these are the immediate problems that will make the Kinect more generally useful for me (in its current state and the state of games most of it is unavailable to me).

So for now, the Kinect took a person who generally does not feel very disabled (me) due to my iBot and made me feel excluded. The same way that I might be if I were not allowed in certain buildings or to certain events.

But in the long term there is more to it.

If you look at the community of people who are in wheelchairs and those who have problems such as missing limbs as one big group for a moment, they fall into two distinct categories:

  1. Those who see themselves as whole people with four fully active limbs, and
  2. Those who see themselves with the particular limitation their disability gives them.

This can be thought of the way Morpheus described it to Neo, the residual self image, also described here:

Residual self image is the concept that individuals tend to think of themselves as projecting a certain physical appearance. The term was popularized in fiction by the Matrix series, where persons who existed in a digitally created world would subconsciously maintain the physical appearance that they had become accustomed to projecting.

Now I think it is fair to say (once these issues related to best practices and game developers are more widely addressed) that every person who is either missing limb or limbs or who is in a wheelchair see themselves in one of the two ways I mention above, and they would ideally want their Kinect avatar to match that, in large part. Furthermore, I think that if they (by which I also mean me) are asked to have an avatar that denies them their residual self image then they will not enjoy the experience as much.

Once this is addressed, there is remarkable potential to give people back the life they want if the ability to virtually extend themselves into a digital world lets them be who they feel they really are, with their own residual self image, even if circumstances have worked to interfere with that image a bit.

I could spitball (and have spitballed) thoughts on how this might work, i.e. configuration options that would alter the way the Kinect projects the user to the games based on their images of themselves. But I have no idea how all of that will go, how much they will do.

Some of these issues are issues I hinted at in a blog back in February (this blog), though at the time I did not give the option of a digital way out of things -- or a way to move beyond things.

But I think the Kinect has some very real possibilities here, and I look forward to seeing what they come up with....

If I find myself in front of a Kinect again, I'll try to get some pictures and/or video. It really is pretty cool to see it work with something it is clearly not entirely comfortable with!


J on 15 Nov 2010 12:29 PM:

Making people stand up in games seems to be the flaw.  

The number of people with iBots can't be a meaningful design point. You're very lucky to have been involved so directly as is. I think it's cool that they spent so much effort on it, actually. Did they spend time for those without arms? That would be cool too.

Lobbying the game developers (if Kinect is even a success) to support sitting 'pipe' seems the best course of action. That could be enabling.

Michael S. Kaplan on 15 Nov 2010 5:32 PM:

iBots themselves aren't useful as a group (not enough of a market) but by dealing with the general issue of people sitting (which the iBot helped show some real flaws with) they were able to deal with a much larger circumstance (one that they can now push game developers to do better with and that will enable couch potatoes as much as those who can't get up).

Exciting times! :-)

fourboyzmom on 17 Nov 2010 10:16 AM:

I am the mom of a 12 yr old boy who has CP and uses a powerchair. He is a master at crazy taxi, but I would love to give him more choices game wise. We have a PS2 and an xbox at the moment. I looked into the Wii a while back, but my husband and other sons thought the graphics were lousy. I was about to order the Kinect, when I stumbled upon your article. Would you be able to offer some advice before I do my holiday shopping?

Michael S. Kaplan on 17 Nov 2010 5:28 PM:

I think some games will be exciting, but others will be frustrating. It may be better to see how well game developers work to support the seated pipe, for both your son's sake and mine getting maximinal benefit....

Michael S. Kaplan on 25 Nov 2010 11:24 AM:

PingBack: blog.fawny.org/.../badtastehasacost


referenced by

2011/04/25 How I found myself feeling morally superior to iPad and iPhone owners

2010/11/26 They refer to the quality of existing, not the quality of living (or the quality of gaming)

go to newer or older post, or back to index or month or day