Please use this identifier to cite or link to this item:
Title: A soft actor-critic deep reinforcement learning method for multi-timescale coordinated operation of microgrids
Authors: Hu, Chunchao
Cai, Zexiang
Zhang, Yanxu
Yan, Rudai
Cai, Yu
Cen, Bowei
Keywords: Engineering::Electrical and electronic engineering
Issue Date: 2022
Source: Hu, C., Cai, Z., Zhang, Y., Yan, R., Cai, Y. & Cen, B. (2022). A soft actor-critic deep reinforcement learning method for multi-timescale coordinated operation of microgrids. Protection and Control of Modern Power Systems, 7(1).
Journal: Protection and Control of Modern Power Systems
Abstract: This paper develops a multi-timescale coordinated operation method for microgrids based on modern deep reinforcement learning. Considering the complementary characteristics of different storage devices, the proposed approach achieves multi-timescale coordination of battery and supercapacitor by introducing a hierarchical two-stage dispatch model. The first stage makes an initial decision irrespective of the uncertainties using the hourly predicted data to minimize the operational cost. For the second stage, it aims to generate corrective actions for the first-stage decisions to compensate for real-time renewable generation fluctuations. The first stage is formulated as a non-convex deterministic optimization problem, while the second stage is modeled as a Markov decision process solved by an entropy-regularized deep reinforcement learning method, i.e., the Soft Actor-Critic. The Soft Actor-Critic method can efficiently address the exploration–exploitation dilemma and suppress variations. This improves the robustness of decisions. Simulation results demonstrate that different types of energy storage devices can be used at two stages to achieve the multi-timescale coordinated operation. This proves the effectiveness of the proposed method.
ISSN: 2367-0983
DOI: 10.1186/s41601-022-00252-z
Schools: School of Electrical and Electronic Engineering 
Rights: © The Author(s) 2022. Open Access. This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:EEE Journal Articles

Files in This Item:
File Description SizeFormat 
s41601-022-00252-z.pdf2.45 MBAdobe PDFThumbnail

Citations 20

Updated on Nov 30, 2023

Web of ScienceTM
Citations 20

Updated on Oct 31, 2023

Page view(s)

Updated on Nov 30, 2023


Updated on Nov 30, 2023

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.