Outdoor augmented reality enabled by android sensor measurement processing

Abstract

In this dissertation, we developed an outdoor Augmented Reality (AR) framework that utilizes sensor data collected from Android smartphones. Unlike conventional indoor AR systems, which depend on optical devices and computer vision technologies, our approach leverages raw Global Navigation Satellite System (GNSS) data and Inertial Measurement Unit (IMU) data to accurately determine the positions and orientations of smartphones. The primary goal is to superimpose virtual objects onto real-world scenes captured by an Android smartphone camera, with the spatial coordinates of these virtual objects predefined by users. To achieve this, we introduce a positioning algorithm based on Weighted Least Squares (WLS) that utilizes raw GNSS data from Android devices. We also discuss methods for processing IMU data to acquire orientation measurements. Key to our framework is the specification of transformations between various coordinate systems, enabling the alignment of the GNSS global coordinates with the smartphone’s local coordinate system. Moreover, we apply a practical technique to estimate the camera’s scale factor, facilitating the precise mapping of three-dimensional points to specific image pixels. Comprehensive experiments conducted as part of this research demonstrate the effectiveness of our outdoor AR application. We performed both offline experiments and real-time experiments based on an Android smartphone app. Results show that it can accurately align virtual objects with the physical environment under various conditions, including different viewing angles and distances. This robustness ensures a consistent and immersive user experience in outdoor AR scenarios.