News

There seems to be a lot of interesting things happening around hacking the Kinect recently. Here's a quick roundup of what people are making.

What's Kinect?

Kinect is an accessory for the xbox games console that allows Wii and Playstation Eye style games that are more physically active. The games involve a lot of jumping around and looking like a fool in front of friends. To make these new games possible the Kinect has a camera for vision, microphone array for voice recognition and a 3D depth sensor.

What makes the Kinect interesting to hackers is that it's a cheap structured-light 3D scanner. It fires out an array of IR light that is reflected by objects in front of it and picked up by an IR camera. The below shows this IR grid as seen through a night vision camera.

This grid of dots can be assembled together to make a 3D model of the room in front of the kinect, in a similar way to the music video for Radiohead's song House of Cards.

Within a few days of release the kinect was reverse engineered and open source drivers for allowing it to work with windows, mac os X and linux were created. It's under a month since the Kinect was released and there is already a growing number of interesting projects being created.

What are people making?

Here we can see what happens what we take the 3D model and overlay the video from the camera in the kinect on top. It has a nice aesthetic similar to Scanner Darkly, letting us see what the machine sees.

Inevitably one of the first hacks to be made is virtual lightsabre tracking.

These wonderful hand puppets are one of the few kinect hacks so far that go beyond a tech demo and create something with more character. It's also a fantastic example of mapping the captured data into a model of a human skeleton.

This combination of the roomba vacuum cleaner and the kinect also inadvertently has character. Giving gestural instructions to the Roomba is a lot like giving commands to a puppy, only hopefully more obedient.

Finally we have a more practical use case, assembling molecular structures to build nanotech.

Hopefully someone will make a cheap 3D object scanner using the kinect that could be hooked up to a Makerbot, producing the 3D equivalent of a photocopier.

How to develop for kinect

The drivers needed to interface the kinect with a computer are available on github, with documentation and getting started guides at openkinect.org. There are also libraries for Processing and OpenFrameworks available to make tinkering easier.

A useful library to use when hacking with the Kinect is OpenCV. OpenCV make image recognition task easier, such as recognising faces or gestures and can work with webcams such as that within the kinect.

Elsewhere Matt Cutts has a $2,000 competition going for the most interesting use of kinect whilst there's a writeup on ladyada on how the kinect was reverse engineered and how you can do it yourself.