Kinect Virtual Video Frame

Kinect based augmented reality demo application

Introduction

The Microsoft Kinect motion sensor was released at the end of 2010. The infrared projector and camera of the sensor make it possible to sense the observed scene and track the movement of individuals in three dimensions. With the use of the OpenNI software framework, the device can track the human body and joint positions.
Augmented reality (AR) is a live, direct or indirect, view of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics or GPS data.

Project goals and features

The goal of the project is to create an augmented reality application based on the 3D human joint position data acquired from the Kinect sensor. The application can track the user and attach a virtual picture frame to the user's hands, so it seems like the user is holding and moving a real picture frame.


The main fetures are as follows:

  • track the user in real-time, and attach a virtual frame when the user performs a special move
  • the virtual frame can be "dropped" and "picked up" again
  • the frame can work in "stretch mode", which means the size of the frame adapts to the width of the hands (the frame cannot be "dropped")
  • user selected video files can be loaded to the frame
  • the playback stops when the frame is dropped

Technology

The software was created in .NET4 C# and the 3D virtual frame graphics are rendered with WPF 3D. This software can be a good starting point for those who want to use C# with Kinect, calculate vectors from joint positions, attach virtual objects to joints or limbs, and visualize 2D and 3D scenes.