Pathological voice detection based on multi-vowel multi-modal model

Abstract

Pathological voice detection aims to determine whether a given voice sample originates from an individual with a voice pathology or a healthy individual. While previous studies have demonstrated that incorporating multiple voice recordings improves classification accuracy in traditional machine learning methods [1], the impact of using multiple voice recordings in deep learning-based approaches remains unexplored. This thesis investigates the effectiveness of utilizing multiple voice recordings and leveraging within-vowel and between-vowel feature relationships for pathological voice detection. Additionally, it examines the simultaneous use of multiple input types—specifically, audio waveform, Mel-spectrogram, and extended Geneva Minimalistic Acoustic Parameter set (eGeMAPs). Experiments conducted on the SVD dataset reveal that incorporating multiple vowels and input types enhances detection performance. Based on these findings, this thesis proposes a simple model that integrates multiple vowels and input types, achieving an accuracy of 74.5%. The results suggest that combining this approach with more complex model architectures that have demonstrated high detection accuracy could further improve their performance. In addition, this thesis proposes a method to assign attributes to groups of inputs of the model based on the integrated gradients method, enhancing the explainability of the model.