Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/153396
Title: Deep reinforcement learning based path stretch vector resolution in dense traffic with uncertainties
Authors: Pham, Duc-Thinh
Tran, Phu N.
Alam, Sameer
Duong, Vu
Delahaye, Daniel
Keywords: Engineering::Aeronautical engineering::Aviation
Engineering::Computer science and engineering::Computing methodologies::Artificial intelligence
Issue Date: 2022
Source: Pham, D., Tran, P. N., Alam, S., Duong, V. & Delahaye, D. (2022). Deep reinforcement learning based path stretch vector resolution in dense traffic with uncertainties. Transportation Research Part C: Emerging Technologies, 135, 103463-. https://dx.doi.org/10.1016/j.trc.2021.103463
Journal: Transportation Research Part C: Emerging Technologies 
Abstract: With the continuous growth in the air transportation demand, air traffic controllers will have to handle increased traffic and consequently, more potential conflicts. This gives rise to the need for conflict resolution advisory tools that can perform well in high-density traffic scenarios given a noisy environment. Unlike model-based approaches, learning-based approaches can take advantage of historical traffic data and flexibly encapsulate environmental uncertainty. In this study, we propose a reinforcement learning approach that is capable of resolving conflicts, in the presence of traffic and inherent uncertainties in conflict resolution maneuvers, without the need for prior knowledge about a set of rules mapping from conflict scenarios to expected actions. The conflict resolution task is formulated as a decision-making problem in large and complex action space. The research also includes the development of a learning environment, scenario state representation, reward function, and a reinforcement learning algorithm inspired from Q-learning and Deep Deterministic Policy Gradient algorithms. The proposed algorithm, with two stages decision-making process, is used to train an agent that can serve as an advisory tool for air traffic controllers in resolving air traffic conflicts where it can learn from historical data by evolving over time. Our findings show that the proposed model gives the agent the capability to suggest high-quality conflict resolutions under different environmental conditions. It outperforms two baseline algorithms. The trained model has high performance under low uncertainty level (success rate >= 95% ) and medium uncertainty level (success rate >= 87%) with high traffic density. The detailed analysis of different impact factors such as the environment's uncertainty and traffic density on learning performance are investigated and discussed. The environment's uncertainty is the most important factor which affects the performance. Moreover, the combination of high-density traffic and high uncertainty will be a challenge for any learning model.
URI: https://hdl.handle.net/10356/153396
ISSN: 0968-090X
DOI: 10.1016/j.trc.2021.103463
Rights: © 2021 Elsevier Ltd. A. All rights reserved. This paper was published in Transportation Research Part C: Emerging Technologies and is made available with permission of Elsevier Ltd.
Fulltext Permission: embargo_20231216
Fulltext Availability: With Fulltext
Appears in Collections:ATMRI Journal Articles
MAE Journal Articles

Files in This Item:
File Description SizeFormat 
TRC_DRL_Conflict_Resolution_Final_Submission.pdf
  Until 2023-12-16
2.74 MBAdobe PDFUnder embargo until Dec 16, 2023

Page view(s)

29
Updated on Jan 17, 2022

Google ScholarTM

Check

Altmetric


Plumx

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.