Semantics-aware human-computer interaction software testing

Abstract

In an age where technology and human interaction are becoming ever more entwined, human-interactive systems—encompassing web services and artificial intelligence—are integral to daily life. From web applications to automated speech recognition (ASR) systems, these technologies are transforming human-machine communication and underpinning contemporary society's functionality. However, ensuring the reliability and robustness of these systems is challenging due to the vast and unpredictable input/output spaces they operate within, making traditional testing methods, such as fuzzing, insufficient.

This thesis addresses four critical aspects of human-computer interaction (HCI) system reliability and robustness: test input generation, service behavior representation, automation of testing processes, and test oracle generation. These aspects are vital for developing robust and reliable HCI systems, each representing a key area where current methods fall short and require innovative approaches.

The first aspect, test input generation, is explored through web applications, introducing a novel methodology that employs curiosity-driven reinforcement learning to create high-quality action sequences that adhere to temporal logical relationships. This enhances test input generation, demonstrating how conventional strategies can be adapted for complex systems.

The second aspect, service behavior representation, is addressed in the context of RESTful API backend components in web services. This study presents a model-based testing approach using a RESTful-service Property Graph (RPG) to represent service behaviors, which improves the effectiveness of fuzzing tests by streamlining call sequence generation.

The third aspect, automation of testing processes, focuses on RESTful API testing in industry, a critical need given RESTful APIs' prevalence in corporate cloud services. The research introduces an automated framework that identifies and helps fix new bugs, emphasizing the necessity of human involvement for improving automated processes and updating API specifications and test cases.

The final aspect, test oracle generation, is advanced through accessibility testing for ASR systems, particularly for stuttered speech. It introduces ASTER, a tool that generates diverse stuttering test cases to identify ASR system failures effectively, underscoring the importance of robust test oracle generation for HCI software.

Through these four interconnected studies, this thesis introduces a comprehensive approach to testing within HCI systems. It underscores the need for specialized methodologies to address the unique challenges of reliability and robustness. By integrating traditional testing techniques with the dynamic characteristics of these systems, this research advances the field of HCI system reliability and robustness, particularly within human-machine interaction.