A Dynamic Rule Creation Based Anomaly Detection Method for Identifying Security Breaches in Log Records
Date of Issue2015
Evidence of security breaches can be found in log files, created by various network devices in order to provide information about their operation. Huge amount of data contained within these files usually prevents to analyze them manually, therefore it is necessary to utilize automatic methods capable of revealing potential attacks. In this paper we propose a method for anomaly detection in log files, based on data mining techniques for dynamic rule creation. To support parallel processing, we employ Apache Hadoop framework, providing distributed storage and distributed processing of data. Outcomes of our testing show potential to discover new types of breaches and plausible error rates below 10 %. Also, rule generation and anomaly detection speeds are competitive to currently used algorithms, such as FP-growth and apriori.
Wireless Personal Communications
© 2015 Springer Science+Business Media New York. This is the author created version of a work that has been peer reviewed and accepted for publication by Wireless Personal Communications, Springer Science+Business Media New York. It incorporates referee’s comments but changes resulting from the publishing process, such as copyediting, structural formatting, may not be reflected in this document. The published version is available at: [http://dx.doi.org/10.1007/s11277-015-3128-1].