Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/155007
Title: An incremental clustering method for anomaly detection in flight data
Authors: Zhao, Weizun
Li, Lishuai
Alam, Sameer
Wang, Yanjun
Keywords: Engineering::Computer science and engineering
Engineering::Aeronautical engineering
Issue Date: 2021
Source: Zhao, W., Li, L., Alam, S. & Wang, Y. (2021). An incremental clustering method for anomaly detection in flight data. Transportation Research Part C: Emerging Technologies, 132, 103406-. https://dx.doi.org/10.1016/j.trc.2021.103406
Journal: Transportation Research Part C: Emerging Technologies 
Abstract: Safety is a top priority for civil aviation. New anomaly detection methods, primarily clustering methods, have been developed to monitor pilot operations and detect any risks from such flight data. However, all existing anomaly detection methods are offlline learning - the models are trained once using historical data and used for all future predictions. In practice, new flight data are accumulated continuously and analyzed every month at airlines. Clustering such dynamically growing data is challenging for an offlline method because it is memory and time intensive to re-train the model every time new data come in. If the model is not re-trained, false alarms or missed detections may increase since the model cannot reflect changes in data patterns. To address this problem, we propose a novel incremental anomaly detection method based on Gaussian Mixture Model (GMM) to identify common patterns and detect outliers in flight operations from digital flight data. It is a probabilistic clustering model of flight operations that can incrementally update its clusters based on new data rather than to re-cluster all data from scratch. It trains an initial GMM model based on historical offlline data. Then, it continuously adapts to new incoming data points via an expectation-maximization (EM) algorithm. To track changes in flight operation patterns, only model parameters need to be saved. The proposed method was tested on three sets of simulation data and two sets of real-world flight data. Compared with the traditional offline GMM method, the proposed method can generate similar clustering results with significantly reduced processing time (57 % - 99 % time reduction in testing sets) and memory usage (91 % - 95 % memory usage reduction in testing sets). Preliminary results indicate that the incremental learning scheme is effective in dealing with dynamically growing data in flight data analytics.
URI: https://hdl.handle.net/10356/155007
ISSN: 0968-090X
DOI: 10.1016/j.trc.2021.103406
Schools: School of Mechanical and Aerospace Engineering 
Research Centres: Air Traffic Management Research Institute 
Rights: © 2021 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:ATMRI Journal Articles
MAE Journal Articles

Files in This Item:
File Description SizeFormat 
An incremental clustering method for anomaly detection in flight data.pdf7.8 MBAdobe PDFThumbnail
View/Open

SCOPUSTM   
Citations 10

52
Updated on Mar 25, 2024

Web of ScienceTM
Citations 20

10
Updated on Oct 26, 2023

Page view(s)

161
Updated on Mar 29, 2024

Download(s) 50

120
Updated on Mar 29, 2024

Google ScholarTM

Check

Altmetric


Plumx

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.