Applying machine learning to human speech for image interpretations

Abstract

Speech Recognition has become prevalent over the years due to its ability to do information search, communicate and transcribe faster than typing on a keyboard. It is predicted that about half of the searches would employ Speech Recognition by 2020. With the growing trend of big data, data analytic and data science in the field of Machine Learning, the accuracy and precision for the audio recognition has vastly improved. There are technologies available to support related applications such as the voice assistance in Google Assistance, Amazon Alexa, Apple Siri, and Microsoft Cortana. The open sources for the Speech to Text recognition API enabled development to wider area such as education, customer support and even to daily texting routine.

SpeechArt is a product from adapting existing Speech Recognition technology, DeepSpeech to creating artistic images for the transcribed texts. The acoustic and language model of an open source of DeepSpeech would be utilised. The transcribed text generated from DeepSpeech would be parsed by an NLP Model. The key words would be selected based on user’s audio input and then send image search model to return internet images with artistic effects in real time. The purpose of the project is to build an application that converts speech to image that is useful for visual learning in education and can be extended to artistic aspects for example, portraying a new design work.