Security testing of human-interactive systems

Abstract

In an era where technology and human interaction are increasingly intertwined, human-interactive systems, such as robotics, web services, and artificial intelligence, play a pivotal role in our daily lives. From multi-robot systems managing complex tasks to large language model chatbots transforming human-machine communication, these systems are integral to modern society's functionality. However, ensuring the security of these systems poses a formidable challenge. Unlike traditional systems, human-interactive systems operate in environments with vast and unpredictable input/output spaces, making conventional security testing methods like fuzzing insufficient.

This thesis addresses the critical and complex issue of conducting effective security testing on human-interactive systems. It tackles the unique challenges posed by the extensive and dynamic nature of these systems' interaction with both their environment and users. The research encapsulates four comprehensive studies, each targeting a different facet of human-interactive system security, yet collectively contributing to a broader understanding and enhancement of these systems' security.

The first study delves into the Byzantine threats in Multi-Robot Systems (MRSs), revealing the intricate and expanded attack surface that arises from their collaborative nature. A novel methodology specific to the Robot Operating System (ROS) is introduced, demonstrating how traditional security approaches can be adapted and applied to these complex systems.

In the realm of robotic operating systems, the second study focuses on ROS2, highlighting the vulnerabilities inherent in its security module, Secure ROS2 (SROS2). This research not only identifies critical security flaws but also proposes an innovative defense mechanism, showcasing the need for and application of advanced security measures in these systems.

The third study shifts the focus to RESTful APIs, which are fundamental to web services yet are prone to overlooked vulnerabilities. The introduction of NAUTILUS, an advanced tool for detecting API vulnerabilities, underscores the importance of specialized security approaches in dealing with the nuanced and diverse nature of human-interactive systems.

Finally, the thesis addresses security concerns in Large Language Model (LLM) chatbots. Through the development of Jailbreaker, a comprehensive framework, the research provides insights into the complex nature of security threats in AI-driven human interaction systems, highlighting the need for robust and adaptive security strategies.

Overall, this thesis presents a novel and holistic approach to security testing in human-interactive systems, emphasizing the need for specialized methods to address their unique security challenges. By bridging the gap between traditional security testing methods and the dynamic nature of these systems, this research significantly advances the field of system security in the context of human-machine interaction.