Machine learning / deep learning approach to soundscape analysis

Abstract

Visual understanding of the soundscape environment is an enabling factor for a wide range of applications in studying how humans perceive sounds. Audiovisual scene decomposition allows further understanding of soundscape. This project will be focusing on the decomposition of urban soundscapes such as parks, plazas, streets, etc. As water sounds are a prominent sound source in urban landscapes, this project will add a new waterbody class to the segmentation model which do not currently exist in most multiclass urban semantic segmentation model. This project proposes the use of the DeepLabV3+ model, with a ResNet50 backbone, trained on an improved Cityscapes dataset to perform semantic segmentation for urban scene decomposition. The training dataset will include additional waterbody images on top of the original Cityscapes images.