DiSC: A Distributed In-Storage Computing Platform Using Cost-Effective Hardware Devices

Jaehwan LEE  Joohwan KIM  Ji Sun SHIN  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E100-D   No.12   pp.3018-3021
Publication Date: 2017/12/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2017EDL8104
Type of Manuscript: LETTER
Category: Computer System
Keyword: 
distributed data processing,  in-storage computing,  Mesos,  single-board computer,  

Full Text: PDF(859.8KB)
>>Buy this Article


Summary: 
The ability to efficiently process exponentially increasing data remains a challenging issue for computer platforms. In legacy computing platforms, large amounts of data can cause performance bottlenecks at the I/O interfaces between CPUs and storage devices. To overcome this problem, the in-storage computing (ISC) technique is introduced, which offloads some of the computations from the CPUs to the storage devices. In this paper, we propose DiSC, a distributed in-storage computing platform using cost-effective hardware. First, we designed a general-purpose ISC device, a so-called DiSC endpoint, by combining an inexpensive single-board computer (SBC) and a hard disk. Second, a Mesos-based resource manager is adapted into the DiSC platform to schedule the DiSC endpoint tasks. To draw comparisons to a general CPU-based platform, a DiSC testbed is constructed and experiments are carried out using essential applications. The experimental results show that DiSC attains cost-efficient performance advantages over a desktop, particularly for searching and filtering workloads.