Optimal persistent monitoring using reinforcement learning

Abstract

When monitoring a dynamically changing environment where a stationary group of agents cannot fully cover, a persistent monitoring problem (PMP) arises. In contrast to constantly monitoring, where every target must be monitored simultaneously, persistent monitoring requires a smaller number of agents and provides an effective and reliable prediction with a minimized uncertainty metric. This project aims to implement Reinforcement Learning (RL) in the multiple targets monitoring simulation with a single agent. This paper presents a comparative analysis of five implementations in Reinforcement Learning: Deep Q Network (DQN), Double Deep Q Network (DDQN), Dueling Deep Q Network (Dueling DQN), Multi-Objective Deep Reinforcement Learning (MODRL) and Hierarchical Deep Q Network (HDQN). Different designs of the reward function and stop condition are tested and evaluated to improve models’ decision capability. This paper presents experiences in applying the goal decomposition, a new approach to feature extension to solve the persistent monitoring problem without modifying images, and an improved method for a highly dynamic environment. These proposed approaches significantly enhance the model’s performance and stability.