Please use this identifier to cite or link to this item:
|Title:||Deep reinforcement learning based path stretch vector resolution in dense traffic with uncertainties||Authors:||Pham, Duc-Thinh
Tran, Phu N.
Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
|Issue Date:||2022||Source:||Pham, D., Tran, P. N., Alam, S., Duong, V. & Delahaye, D. (2022). Deep reinforcement learning based path stretch vector resolution in dense traffic with uncertainties. Transportation Research Part C: Emerging Technologies, 135, 103463-. https://dx.doi.org/10.1016/j.trc.2021.103463||Journal:||Transportation Research Part C: Emerging Technologies||Abstract:||With the continuous growth in the air transportation demand, air traffic controllers will have to handle increased traffic and consequently, more potential conflicts. This gives rise to the need for conflict resolution advisory tools that can perform well in high-density traffic scenarios given a noisy environment. Unlike model-based approaches, learning-based approaches can take advantage of historical traffic data and flexibly encapsulate environmental uncertainty. In this study, we propose a reinforcement learning approach that is capable of resolving conflicts, in the presence of traffic and inherent uncertainties in conflict resolution maneuvers, without the need for prior knowledge about a set of rules mapping from conflict scenarios to expected actions. The conflict resolution task is formulated as a decision-making problem in large and complex action space. The research also includes the development of a learning environment, scenario state representation, reward function, and a reinforcement learning algorithm inspired from Q-learning and Deep Deterministic Policy Gradient algorithms. The proposed algorithm, with two stages decision-making process, is used to train an agent that can serve as an advisory tool for air traffic controllers in resolving air traffic conflicts where it can learn from historical data by evolving over time. Our findings show that the proposed model gives the agent the capability to suggest high-quality conflict resolutions under different environmental conditions. It outperforms two baseline algorithms. The trained model has high performance under low uncertainty level (success rate >= 95% ) and medium uncertainty level (success rate >= 87%) with high traffic density. The detailed analysis of different impact factors such as the environment's uncertainty and traffic density on learning performance are investigated and discussed. The environment's uncertainty is the most important factor which affects the performance. Moreover, the combination of high-density traffic and high uncertainty will be a challenge for any learning model.||URI:||https://hdl.handle.net/10356/153396||ISSN:||0968-090X||DOI:||10.1016/j.trc.2021.103463||Rights:||© 2021 Elsevier Ltd. A. All rights reserved. This paper was published in Transportation Research Part C: Emerging Technologies and is made available with permission of Elsevier Ltd.||Fulltext Permission:||embargo_20231216||Fulltext Availability:||With Fulltext|
|Appears in Collections:||ATMRI Journal Articles|
MAE Journal Articles
Files in This Item:
|2.74 MB||Adobe PDF||Under embargo until Dec 16, 2023|
Updated on Jan 17, 2022
Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.