Service mesh – enhancing the resiliency, security, observability and availability of the live stream ASR cloud deployment

Abstract

The aim of this project is to improve the resiliency, enhance the observability level, raise the security level and bring about better deployment workflows of an existing automated speech recognition (ASR) system on the cloud. The ASR is currently adopting a typical microservices architecture, which can lead to increased complexities in monitoring and managing the services in terms of security, observability, resiliency and availability as the system scales and evolves to be part of a more complex system. The implemented solutions involve the use of a service mesh and are mainly driven by the Istio service mesh. Firstly, traffic management strategies like circuit breaker and fault injection policies are used to help improve the ASR’s resiliency to failures. Next, mutual TLS encryption communication between services enforced by the Istio’s certificate authority helps to make data communication more secure within the Kubernetes cluster. In addition, deployment strategies like canary deployment will significantly boost the availability level by careful management of deploying new features. Lastly, metrics collected by the service mesh on the proxy, service and control plane level provide greater visibility into interactions between services. In this report, these solutions will be further illustrated through figures and details on how they work and serve to bridge the existing gaps in the ASR. Experiments will be carried out to test the efficacy and tradeoffs of these solutions.