Virtual assembling using hand tracking with Microsoft Kinect

Abstract

With recent technological advancements, more accurate and lifelike simulations have been created. However, these are often either reliant on keyboard-and-mouse, and hence not entirely realistic in its navigation, or reliant on specific custom-made hardware that can be expensive. While there have been research done into leveraging the Kinect to create applications as well as exploring the limits of the device, there have been no specific research done into using the Kinect for virtual assembly of objects via hand-tracking. Therefore, this report looks into utilizing the Kinect, a widely available motion-sensing device, to create a virtual assembly application via Unity. The end product should be a working prototype that demonstrates using gestures to control the movement of objects alongside tracking the user’s hands. It also explores the limitations of the Kinect and concludes that big body-wide gestures fare better than small hand-based gestures. The application is made using the Unity game engine, and coded with C# as the scripting language. It works with the Kinect to track the user’s hands and use it to manipulate the virtual objects into the correct position. A user study is also carried on to determine how intuitive and accurate the suggested methods are. This report covers the design and development of the application developed.