Online Anomaly Prediction for Real-Time Stream Processing

Yuanqiang HUANG  Zhongzhi LUAN  Depei QIAN  Zhigao DU  Ting CHEN  Yuebin BAI 

Publication
IEICE TRANSACTIONS on Communications  Vol.E95-B  No.6  pp.2034-2042
Publication Date: 2012/06/01
Online ISSN: 1745-1345
Print ISSN: 0916-8516
Type of Manuscript: PAPER
Category: Network Management/Operation
Keyword: 
real-time stream processing,  anomaly prediction,  hidden Markov model,  mixture of expert model,  support vector machine, 

Full Text: PDF

Summary: 
With the consideration of real-time stream processing technology, it's important to develop high availability mechanism to guarantee stream-based application not interfered by faults caused by potential anomalies. In this paper, we present a novel online prediction technique for predicting some anomalies which may occur in the near future. Concretely, we first present a value prediction which combines the Hidden Markov Model and the Mixture of Expert Model to predict the values of feature metrics in the near future. Then we employ the Support Vector Machine to do anomaly identification, which is a procedure to identify the kind of anomaly that we are about to alarm. The purpose of our approach is to achieve a tradeoff between fault penalty and resource cost. The experiment results show that our approach is of high accuracy for common anomaly prediction and low runtime overhead.