Audio and visual tracking system in indoor environments

Abstract

An audio and visual tracking system incorporates existing speech recognition technologies and visual aids to form a seamless autonomous system using Raspberry Pi 4. This project explores existing algorithms in both audio and visual formats and uses these algorithms via Raspberry Pi. While this project can be used in various areas in real-life situations, one emphasis used throughout the project will be in conferences. Speech recognition can be used to pinpoint a speaker and have visual tracking on current audio. In that scenario, a Raspberry Pi was used as the computing power, while a microphone array was used to determine the direction of the speech and the intensity of the speech. A camera is connected to the Raspberry Pi and will change direction according to where the microphone senses the speech is coming from. To that extent, I took over a project that was in progress in the previous year. The student was able to identify a connection between the three components and connected them via a python program on Raspberry Pi. My project was to continue and improve on the progress of the project and add advancements if needed. During my project, I was able to do testing on two types of microphone arrays, do 3D design and printing, prototyping, iOS development, and further testing with all combined.