The augmented human : seeing sounds

Abstract

Human ears can hear sound from range frequency of 20Hz to 20kHz. They discerns between ringing of a telephone to the sound of lion’s roar. It is also capable of tracking the sounds, providing the person a rough estimation where the direction is and to confirm it source, the person only needs to view it direction. However being impaired in hearing would disabled the person from such ability. In this project, the possibility of means to using readily available technology as a substitute to sourcing the direction of sounds through merging the signal and image processing. Using portable devices such Raspberry Pi as the computation power, camera module to stream the images and MATRIX Creator for its embedded microphones to capture the audio. With testing and implementation of the system using 2 methods to merge merging the signal and image processing, using Normalized Device Coordinate (NDC) provides a better result which able to highlight any given sound and track its movement.