Outlier detection based on neighborhood proximity

Abstract

Outliers, also called anomalies are data patterns that do not conform to the behavior that is expected or differ too much from the rest. In some cases, outliers could be caused by errors in data generating/collecting methods or by inherent data variability. However, in many situations, outliers are indications of interesting events that have never been known before and hence, an adaptation of the theory to capture the new events is required to explore the underlying mechanisms. The two-side effect of outliers necessitates the development of efficient methods to detect them for either (a) eliminating/minimizing their impacts on general performance of information systems or (b) capturing the underlying interesting knowledge (e.g. intrusive connections in a network). In general, outlier detection has many practical applications, especially in domains that have scope for abnormal behavior, such as fraud detection, network intrusion detection, medical diagnosis, marketing, customer segmentation, etc. There are many ways in practice to solve our problem of interest. This thesis deals specifically with outlier notions based on measures of neighborhood dissimilarity. Related works can be divided into two main categories: distance-based and density-based. In our study, we place our focus more on distance-based approaches. With considerations to the limitations of existing works, we propose two techniques, tackling separate aspects of outlier detection.