Early goal-detection for black-box environment poisoning attacks

Abstract

This project aimed to develop a Goal Recognition (GR) system to enhance the applicability of existing Environment Poisoning Attacks (EPAs). Specifically, it focused on increasing the early accuracy and robustness against environmental changes for an existing GR solution. An architecture based on the Machine Theory of Mind (ToMnet) was implemented and trained on a large dataset of an EPA-specific environment. Two evaluation techniques were proposed to test the quality of character embeddings and predictions. The model was evaluated on 10,000 trajectories of Q-learning agents of varying optimality on randomly generated environments. The model provided weak evaluation results when trained on a basic algorithm. Hence, two improved training algorithms were proposed to combat local overfitting and improve the model's ability to differentiate between goals. These training algorithms increased the model's evaluation accuracy by over 10\% and successfully provided its top predictions as early as 200 victim epochs. Evaluation performances were analyzed to investigate ties between model performance and train data, trajectory quality, and environment patterns that may be points of failure for the current model. Overall, the project successfully sets a baseline score for early goal detection without any knowledge of environment dynamics and agent policy provided to the model.