Continuous benchmarking of serverless cloud providers

Abstract

To date, there is no standard benchmarking methodology to quantitatively compare the performance of different serverless cloud providers. This project aims to design a framework that regularly runs a set of various microbenchmarks on multiple providers, including AWS Lambda, Azure Functions, and Google Cloud Run. To achieve this, the project extends the Serverless Tail Latency Analyzer (STeLLAR) framework by introducing automated deployment capabilities for Azure Functions and supporting the execution of image size experiments. This project analyses cold start delays related to image size and other characteristics of serverless functions, including the available network bandwidth and chunk sizes used during a cold start initialisation.