Partial auto-stereoscopic display with camera-based user tracking

Abstract

There has been increasing interest in 3D stereoscopy in digital media creation and consumption, but is held back by the need for specialized hardware. This research based project investigates a way to simulate auto-stereoscopy through the use of a single webcam. An application was created that enables the user to peer around a 3D object naturally by moving his head around, as if the 3D object exists in reality. This project aims to conduct research into the feasibility of using a single standard camera to derive the coordinates of a user’s head, which can then be used to simulate virtual reality for the user through auto-stereoscopic render of the 3D world. OpenCV was used to analyse the camera video stream, and derive the corresponding 3D coordinates of the user in real world measurements. These coordinates were passed to Ogre3D for virtual camera positioning, simulating the perspective that should be seen by the user. The derived coordinates had jitter, but after undergoing data smoothing by calculating moving average, had improved stability and usability. The architecture ensured that low computational resources were sufficient, and that future work could be done to enhance the precision of the algorithm. Thus, the aim of the project was fulfilled.