A Comparative Study of Unsupervised Anomaly Detection Techniques Using Honeypot Data

Jungsuk SONG  Hiroki TAKAKURA  Yasuo OKABE  Daisuke INOUE  Masashi ETO  Koji NAKAO  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E93-D   No.9   pp.2544-2554
Publication Date: 2010/09/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.E93.D.2544
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Information Network
Keyword: 
intrusion detection system,  unsupervised machine learning techniques,  real traffic data,  various evaluation criteria,  

Full Text: PDF(1.2MB)>>
Buy this Article




Summary: 
Intrusion Detection Systems (IDS) have been received considerable attention among the network security researchers as one of the most promising countermeasures to defend our crucial computer systems or networks against attackers on the Internet. Over the past few years, many machine learning techniques have been applied to IDSs so as to improve their performance and to construct them with low cost and effort. Especially, unsupervised anomaly detection techniques have a significant advantage in their capability to identify unforeseen attacks, i.e., 0-day attacks, and to build intrusion detection models without any labeled (i.e., pre-classified) training data in an automated manner. In this paper, we conduct a set of experiments to evaluate and analyze performance of the major unsupervised anomaly detection techniques using real traffic data which are obtained at our honeypots deployed inside and outside of the campus network of Kyoto University, and using various evaluation criteria, i.e., performance evaluation by similarity measurements and the size of training data, overall performance, detection ability for unknown attacks, and time complexity. Our experimental results give some practical and useful guidelines to IDS researchers and operators, so that they can acquire insight to apply these techniques to the area of intrusion detection, and devise more effective intrusion detection models.