Sonification of geometry

Abstract

The technology underpinning 3D computer graphics has grown increasingly sophisticated in recent years. Modern computing devices are able to render complex 3D models and simultaneously provide virtual spaces for users to interact with these models. While the more conventional method of representing 3D models is naturally through the use of a visual display, the discipline of sonification offers an alternative modality to communicate 3D geometric information using sounds. One past final year project explored the feasibility of adding sound as a secondary modality to deliver geometric information. Users were tasked to interact with 3D objects in a virtual space developed using the Unity3D game engine, with and without parameter mapping sonification. Users reported easier interaction and better understanding of these virtual objects when their geometric properties were sonified. However, these objects were limited to relatively simple shapes that can be represented with implicit functions. This project aims to tackle such a limitation by developing two modular Unity script components: first containing several algorithms able to extract the same geometric properties from 3D models of arbitrary complexity, and second containing a basic synthesiser. With these two components, this project briefly explores different parameter mapping sonification approaches to validate whether sonifying certain geometric properties of complex 3D models do provide users with a richer interactive experience.