Improved Resilience through Extended KVS-Based Messaging System

Masafumi KINOSHITA  Osamu TAKADA  Izumi MIZUTANI  Takafumi KOIKE  Kenji LEIBNITZ  Masayuki MURATA  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E98-D   No.3   pp.578-587
Publication Date: 2015/03/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2014NTP0015
Type of Manuscript: Special Section PAPER (Special Section on the Architectures, Protocols, and Applications for the Future Internet)
Category: Internet Operation and Management
Keyword: 
KVS (key-value store),  in-memory data grid,  messaging queue,  distribution system,  

Full Text: PDF>>
Buy this Article




Summary: 
In the big data era, messaging systems are required to process large volumes of message traffic with high scalability and availability. However, conventional systems have two issues regarding availability. The first issue is that failover processing itself has a risk of failure. The second issue is to find a trade-off between consistency and availability. We propose a resilient messaging system based on a distributed in-memory key-value store (KVS). Its servers are interconnected with each other and messages are distributed to multiple servers in normal processing state. This architecture can continue messaging services wherever in the messaging system server/process failures occur without using failover processing. Furthermore, we propose two methods for improved resilience: the round-robin method with a slowdown KVS exclusion and the two logical KVS counter-rotating rings to provide short-term-availability in the messaging system. Evaluation results demonstrate that the proposed system can continue service without failover processing. Compared with the conventional method, our proposed distribution method reduced 92% of error responses to clients caused by server failures.