Design and development of an augmented reality (AR) application with Leap Motion

Abstract

Augmented Reality (AR) refers to technology that superimposes virtual objects over the real world. It has become more popular as more people have been able to afford smartphones that can run AR software. It has also seen many advancements over the recent years as engineers continue to develop newer AR technology with more features and greater capabilities. One of the most interesting developments for AR would be the incorporation of hand-tracking technology to AR. This technology takes immersion a step further by allowing users to interact with virtual objects using their hands, as though the virtual objects existed in the real world. Such a possibility is not only exciting, but it also has the potential to change the way we use AR. This project aims to develop an AR application for the iOS platform which uses the Leap Motion controller, a device for hand tracking, to incorporate hand interactions into AR. During the course of the project, two applications were developed: an AR Cam application and an AR Piano. The AR Cam application was developed to study how to implement AR image targeting and ground plane detection. Lessons learned during the AR Cam’s development were then further built upon with the development of the AR Piano application, which features image targeting and hand interactions.