In my previous post Kinect Reception–Introduction I’ve talked about Kinect Reception and What it is, in this post I’m going to talk more about How the Reception Application works.

Using Kinect events I can see when user enters the frame, using two timers I can check if the user is just passing  by or is standing in front of the camera.

The below image describe the application flow, at the beginning the application will show random videos, when the Kinect Skeleton Event will raise then the Serious Timer will begin ticking, each tick based on the FPS rate will be aggregate to variable called NumTicks, when the Serious Timer complete we check if NumTicks is big enough based on the FPS, if so then we’ll start the Idle Timer and switch to Kinect Image.

Idle Timer – Each time the Kinect Skeleton Event is raised the Idle Timer will be restart, so if there is no one in front of the Kinect camera the Idle Timer will switch back to Videos.

As you can see from the images below (or Full Video), when I moved my hands or legs the Score Bar has changed based on the rules you defined:

You reached the Goal!!!

The entire application works with 4 managers:

1. Kinect Manager
2. Configuration Manager
3. Video Manager
4. Authentic Manager

Kinect Manager

I’ve showed how to get started with Kinect many times in my previous posts but for this application I’ve created a singleton class to handle everything related to the Kinect settings, from restarting the Kinect RunTime object to changing the camera angle.

public KinectManager()
{
    try
    {

    KinectNui = new Runtime();
 
    KinectNui.Initialize(RuntimeOptions.UseColor | 
                         RuntimeOptions.UseSkeletalTracking |
                         RuntimeOptions.UseColor);
 

    KinectNui.VideoStream.Open(ImageStreamType.Video, 2,
    ImageResolution.Resolution640x480, ImageType.ColorYuv);
 
    KinectNui.SkeletonEngine.TransformSmooth = true;
    var parameters = new TransformSmoothParameters
                            {
                            Smoothing = 1.0f,
                            Correction = 0.1f,
                            Prediction = 0.1f,
                            JitterRadius = 0.05f,
                            MaxDeviationRadius = 0.05f
                            };
    KinectNui.SkeletonEngine.SmoothParameters = parameters;
 
    _lastTime = DateTime.Now;
    Camera = KinectNui.NuiCamera;
 
    IsInitialize = true;
    StatusMessage = Properties.Resources.KinectReady;
    }
    catch (InvalidOperationException ex)
    {
    IsInitialize = false;
    StatusMessage = ex.Message;
    }
}

Another important method the KinectManager has is the CameraAngle control

public void ChangeCameraAngle(ChangeDirection dir)
{
    if (!IsInitialize) return;
 
    try
    {
    if (dir == ChangeDirection.Up)
        Camera.ElevationAngle = Camera.ElevationAngle +
        Properties.Settings.Default.ElevationAngleInterval;
    else
        Camera.ElevationAngle = Camera.ElevationAngle -
        Properties.Settings.Default.ElevationAngleInterval;
 
    StatusMessage = Properties.Resources.KinectReady;
    }
    catch (InvalidOperationException ex)
    {
    StatusMessage = ex.Message;
    }
    catch (ArgumentOutOfRangeException outOfRangeException)
    {
    StatusMessage = outOfRangeException.Message;
    }
}

Video Manager

The Video Manager has two specific types of Videos to play and the main Videos folders to pick random video each time.

Specific? When the user enters the Kinect Range you can choose to play a specific video just before the Kinect Image will appear, and when the user leaves Kinect Range you can choose to play the Out Video.

Define the type of video you want to play, If you ask for out video and there isn't one return null - Stop Video and start showing Kinect Image ,If you ask for in video and there isn't one then return random video.

public static Uri GetVideo(VideoType type)
{
 if (string.IsNullOrEmpty(Properties.Settings.Default.VideosLibraryFolder) 
   || !Directory.Exists(Properties.Settings.Default.VideosLibraryFolder))
        return null;
  else
  {
     string value = null;
     switch (type)
     {
          case VideoType.In:
              value = Properties.Settings.Default.VideosLibraryInFile;
              return string.IsNullOrEmpty(value) || !File.Exists(value) ?
                  CollectRandomMovie() : new Uri(value);
          case VideoType.Out:
              value = Properties.Settings.Default.VideosLibraryOutFile;
              return string.IsNullOrEmpty(value) || !File.Exists(value) ?
                  null : new Uri(value);
          default:
              return CollectRandomMovie();
     }
  }
}

private static Uri CollectRandomMovie()  
{     
    var movies = new ArrayList(); 
  
    foreach (var filter in Properties.Settings.Default.VideoFilter)     
    movies.AddRange(Directory.GetFiles( 

    Properties.Settings.Default.VideosLibraryFolder,filter)); 
    
    if (movies.Count == 0) return null; 
    
    var rnd = new Random(); 

    return new Uri(movies[rnd.Next(movies.Count)].ToString());     
}

Configuration Manager

Kinect Reception allows you to set a range or rules to define what is considered Funny, the rule are based on Joint to Joint and each rule define the Score if the rule applies.

The RuleObject contains the Source Joint and Target Joint, Vector to check for both, the operator (Bigger or Smaller) and the Score.

So what the configuration manager does? it’s saves the rules. (Using MemoryStream to translate the Rule to string and then save them into the application resources.)

public static ObservableCollection<RuleObject> Load()
{
 var deserializer = new XmlSerializer(typeof(ObservableCollection
                                                    <RuleObject>));
 try
 {
   var xs = new XmlSerializer(typeof(ObservableCollection<RuleObject>));
   var memoryStream = new MemoryStream(Encoding.UTF8.GetBytes
   (Properties.Settings.Default.Rules));
   var xmlTextWriter = new XmlTextWriter(memoryStream, Encoding.UTF8);
   return (ObservableCollection<RuleObject>)xs.Deserialize(memoryStream);
 }
 catch (Exception)
 {
   return new ObservableCollection<RuleObject>();
 }
}

public static void Save(ObservableCollection<RuleObject> items)
{
 try
 {
  var memoryStream = new MemoryStream();
  var xs = new XmlSerializer(typeof(ObservableCollection<RuleObject>));
  var xmlTextWriter = new XmlTextWriter(memoryStream, Encoding.UTF8);
  xs.Serialize(xmlTextWriter, items);
  memoryStream = (MemoryStream)xmlTextWriter.BaseStream;
  var xmlizedString = Encoding.UTF8.GetString(memoryStream.ToArray());
  Properties.Settings.Default.Rules = xmlizedString;
 }
catch (Exception ex)
 {
   throw new ArgumentException(ex.Message);
 }
}

Authentic Manager

The Authentic Manager is the core of Kinect Reception, he will take all rules defined by you and check them against the Skeleton Joints.

The method below will extract the UnTracked joints and will make sure the joints quality are enough for calculation (We don’t want the user moving out of the picture to be considered as funny )

If the Skeleton Joints reach the Goal Score you define then a event will raise telling the main window to save the current Image and display for the user.

public void ChecksForAuthentic(JointsCollection joints)
{
  if (_rules.Count == 0) return;
 
  var fixJoints =
      joints.Cast<Joint>().Where(
          joint => joint.Position.W >= 0.6f &&
              joint.TrackingState == JointTrackingState.Tracked).ToList();
 
  var sb = new StringBuilder();
  for (var index = 0; index < _rules.Count; index++)
  {
    var rule = _rules[index];
    var s = (from j in fixJoints.Where(joint => joint.ID == rule.Source) 
       select j).DefaultIfEmpty(new Joint() 
        { TrackingState = JointTrackingState.NotTracked }).Single();
 
    var t = (from j in fixJoints.Where(joint => joint.ID == rule.Target) 
       select j).DefaultIfEmpty(new Joint() 
        { TrackingState = JointTrackingState.NotTracked }).Single();
 
    if (s.TrackingState == JointTrackingState.NotTracked ||
        t.TrackingState == JointTrackingState.NotTracked) break;
 
    var sv = s.ToFloat(rule.SourceVector);
    var tv = t.ToFloat(rule.TargetVector);
 
    if (rule.Operator == Operators.Bigger && sv > tv)
    {
      Score = Score + rule.Score;
      sb.AppendLine(string.Format("Bigger -> Source: {0}, Target:{1} ,
      Vector:{2}", rule.Source, rule.Target, rule.SourceVector));
    }
    else if (rule.Operator == Operators.Smaller && sv < tv)
    {
      Score = Score + rule.Score;
      sb.AppendLine(string.Format("Smaller -> Source: {0}, Target:{1} ,
      Vector:{2}", rule.Source, rule.Target, rule.SourceVector));
    }
  }
 
  if (Score >= _goal)
    IsAuthentic(Score, sb.ToString());
}

Enjoy