After a long wait- Microsoft finally released the beta SDK for Kinect on PC earlyer this week.
Being one of the first to ever get an XBOX 360, waited for two years for Kinect to arrive (it was called "NATAL" back then) and been waiting for the ability to fiddle with it ever since, I gladly downloaded and installed the SDK.
Well, the first thing to say about is that it's fun!
But, without still going into the raw detail- the main issue is the collection of joints you receive from the SDK whenever data changes.
The SDK defines a list of well-known joints, for each the engine attempts to detect and track, but may also infer from other data.
Let's assume that the Kinect camera is set up in-front of you, while the table hides your toes. Kinect will detect and track you knees but will infer the location of your toes, assuming that you have standard legs…
If Kinect is unable to infer the location of an obscured are of your body, these joints will not be tracked until available.
Here's the list of joints, as defined by the JointID enum:
public enum JointID
HipCenter = 0,
Spine = 1,
ShoulderCenter = 2,
Head = 3,
ShoulderLeft = 4,
ElbowLeft = 5,
WristLeft = 6,
HandLeft = 7,
ShoulderRight = 8,
ElbowRight = 9,
WristRight = 10,
HandRight = 11,
HipLeft = 12,
KneeLeft = 13,
AnkleLeft = 14,
FootLeft = 15,
HipRight = 16,
KneeRight = 17,
AnkleRight = 18,
FootRight = 19,
Count = 20,
Whenever a new skeleton frame is ready, the SDK fires an event containing a collection of detected skeletons (as of now- up to two). Each skeleton contains a collection of all joints, stating wheather each of them was tracked, inferred or not-tracked, and their current position.
Now, all that raw data is interesting but something is still missing- Gestures.
Think of a scroll gesture- moving one of your hands from left-to-right at a constant rate, or a clap gesture- moving both hands to the middle and creating a clap sound (remember that we do have audio capturing input that we can time together with the hand movements!).
Gestures are not currently offered by the SDK and are a major part of my little project, and so I've been developing a small gesture detection engine on-top of the SDK in the past few days.
On my next posts I plan to share some of the code I've been working on and ask for your contributions, as such an engine also requires algorythems which I'm not sure I already got hang on all of them.