Saturday, August 06, 2011

Hello Kinect!

So, finally I gave up all the resistance to avoid buying this cool new stuff and ordered a piece for myself from http://www.flipkart.com/. The unit arrived last week, but had to spent the weekend rearranging my living room so that I could get enough space to use my Kinect. Plugging it to my PC and using the drivers provided with Microsoft Kinect SDK was straightforward. I did initially face a problem with the driver not installing properly, but quickly figured it out that this was because the Kinect was not plugged into a root USB port.
My intention of getting Kinect was not to play games, but to play with programming it. Although I would happily take an XBox, if you gift me one ;-)
This post and subsequent posts on Kindle on this blog will tell my experience of programming on Kindle. As a first step, I ensured that the samples provided with the SDK work well. Next, I installed Visual Studio Express 2010 (available for free here: http://www.microsoft.com/express).
I chose C# (C-Sharp, may be they should call it C-Dumb :P) as my programming language for Kinect. Jokes apart, I have very little experience using C#, mostly using Java or C++ (left using Fortran on day-to-day basis 2 years ago!). Any how, I found C# to be quite neatly designed language and easy to learn particularly if you come from Java or C++ background. If you come from C++, you are sure to enjoy some freshness that Java brought to object oriented programming.
So considering you have some idea to program in C#, writing a 'Hello Kinect' is relatively easy. First ensure that you have added Microsoft.Research.Kinect.dll as an external dependency to the Visual Studio project you create.
image
Next is to import the Kinect APIs:
using Microsoft.Research.Kinect.Nui;
All the initialization of Kinect NUI (Natural User Interface) is handled using the Runtime class.
Runtime nui = new Runtime();
nui.Initialize(RuntimeOptions.UseDepthAndPlayerIndex | 
               RuntimeOptions.UseSkeletalTracking |
               RuntimeOptions.UseColor);


Next we open the Video and Depth streams of Kinect:
nui.VideoStream.Open(ImageStreamType.Video, 2,
                     ImageResolution.Resolution640x480,
                     ImageType.Color);
nui.DepthStream.Open(ImageStreamType.Depth, 2,
                     ImageResolution.Resolution320x240, 
                     ImageType.DepthAndPlayerIndex);

Note that the current Kinect hardware only support VGA resolution (max) for video stream.
When a data frame is available for processing on Kinect, the driver sends a notification to the application. In C# this is handled by registering an appropriate even handler as follows:
nui.DepthFrameReady += 
new EventHandler<ImageFrameReadyEventArgs>(nui_DepthFrameReady);
nui.SkeletonFrameReady += 
new EventHandler<SkeletonFrameReadyEventArgs>(nui_SkeletonFrameReady);


The signatures of the event handlers look as below:
void nui_DepthFrameReady(object sender, ImageFrameReadyEventArgs e)
void nui_SkeletonFrameReady(object sender, 
                            SkeletonFrameReadyEventArgs e)


Now, I have managed to see how to get the Skeletal data easily:
SkeletonFrame skeletonFrame = e.SkeletonFrame;
foreach (SkeletonData data in skeletonFrame.Skeletons)
{                
       foreach (Joint joint in data.Joints)
       {

              // transform and plot joint.Position
       }
}
In the end uninitialize:
nui.Uninitialize();

So here is a video of my ‘dot avatar’) in action Smile
My ‘dot avatar’, the other ‘dot avatar’ is my dad in background. Kinect SDK for Windows 7 at the moment allows tracking of only two people.

Next up, I need to figure out how exactly to use depth data as well as handle audio. Hopefully a post for next weekend Smile

Have fun!

No comments: