Automatic voltage control of differential power grids based on transfer learning and deep reinforcement learning

Abstract

In terms of model-free voltage control methods, when the device or topology of the system changes, the model's accuracy often decreases, so an adaptive model is needed to coordinate the changes of input. To overcome the defects of a model-free control method, this paper proposes an automatic voltage control (AVC) method for differential power grids based on transfer learning and deep reinforcement learning. First, when constructing the Markov game of AVC, both the magnitude and number of voltage deviations are taken into account in the reward. Then, an AVC method based on constrained multi-agent deep reinforcement learning (DRL) is developed. To further improve learning efficiency, domain knowledge is used to reduce action space. Next, distribution adaptation transfer learning is introduced for the AVC transfer circumstance of systems with the same structure but distinct topological relations/parameters, which can perform well without any further training even if the structure changes. Moreover, for the AVC transfer circumstance of various power grids, parameter-based transfer learning is created, which enhances the target system's training speed and effect. Finally, the method's efficacy is tested using two IEEE systems and two real-world power grids.