Kubernetes: deploying speech SGDecoding for scalability

Abstract

In recent years, cloud-based deployment of web applications has become standard practice due to its cost-effectiveness and ease of modification. As costs often scale with usage, achieving both efficient resource utilization and reliable scalability under varying loads has grown increasingly important. This project focuses on the deployment and scaling of the SGDecoding Web Application, an application offering live multilingual speech-to-text transcription, using Amazon Web Services (AWS). Prior to this project, SGDecoding was operational only in a local envi- ronment and lacked a structured online deployment strategy. Before cloud deployment, fundamental elements of the application, such as the existing technology stack, was thor- oughly analyzed. This information went into informing the selection of cloud services, scaling tools, and architectural patterns used in the project. Furthermore, an in-depth analysis of cloud services, scaling tools, and architectures was conducted to identify opti- mal approaches for hosting the application and ensuring its resilience under varying user loads. Following deployment, Kubernetes Horizontal Pod Autoscaler and Cluster Au- toscaler were used to implement cloud automated scaling mechanisms. Doing so enabled the cluster to dynamically adjust resources depending on prevailing user traffic to the application. In the testing phase, the application was shown to be able to handle up to 1,000 concurrent users under varying test conditions. This presented a 90% increase in concurrent users over default scaling approaches. These tests clearly demonstrate the system’s resilience under heavy and fluctuating workloads.