Automated testing of cross-language communications in rpc frameworks - part 2

Abstract

Remote Procedure Call (RPC) frameworks like Apache Thrift are essential for building scalable, distributed systems, particularly in microservice architectures where services written in different programming languages must communicate efficiently. However, ensuring consistent data representation and communication fidelity across diverse lan- guage implementations proved to be a non-trivial challenge. This paper details the integration of the Rust programming language into an automated testing framework designed to evaluate Apache Thrift’s cross-language behaviour by systematically iden- tifying inconsistencies that arise during RPC calls involving Rust, Go, Python and NodeJS implementations across various transport and protocol configurations. Leveraging automated generation of Thrift definitions and language-specific server/- client code (written during runtime), the framework executed millions of RPC calls. A key contribution is a systematic post-mortem analysis methodology that parsed client and server logs into a structured format, enabling large-scale comparison of data values at different stages of the RPC lifecycle. The results revealed that while Rust integration was largely successful, significant in- consistencies (around 11.5% of all successful calls) persist within the tests conducted on Apache Thrift’s interactions. Notable findings include discrepancies in the handling of null/optional struct fields (often resolving to falsy defaults), precision issues with large integers specific to NodeJS clients, language-specific logging artifacts misinter- preted as data changes (e.g. Go enums, Python sets), and a suspected data mutation bug affecting boolean collections in NodeJS under the compact protocol.