Benchmarking and optimization of large-scale serverless deployments

Abstract

Efficiently executing large-scale experiments is a critical challenge in distributed systems research, often hindered by the need for extensive manual configuration and limited flexibility in traditional workload generators. This project presents the design and implementation of the multi-loader framework—a modular, scalable system that extends the capabilities of In-Vitro by enabling batch execution of experiments through a unified and dynamic configuration schema. Key innovations include support for multiple experiment definitions via a studies field, flexible trace path specification (through directory-based, format-based and direct override methods) and enhanced custom scripting capabilities at both the study and experiment levels. The framework also integrates failure management strategies, comprehensive metrics and log aggregation, a dry run mechanism for early validation and a sweep mechanism to facilitate extensive parameter exploration. Built using the Go programming language with Bash scripting for system interactions and leveraging cloud-native tools such as Kubernetes, Docker, Prometheus and CloudLab, the multi-loader framework offers a reproducible and automated solution that significantly reduces manual overhead while ensuring scalability and reliability. This report details the system design, configuration schema and architectural choices underpinning the framework, demonstrating its potential to streamline experimental workflows in distributed environments.