For Full-Text PDF, please login, if you are a member of IEICE,|
or go to Pay Per View on menu list, if you are a nonmember of IEICE.
An Efficient Block Assignment Policy in Hadoop Distributed File System for Multimedia Data Processing
Cheolgi KIM Daechul LEE Jaehyun LEE Jaehwan LEE
IEICE TRANSACTIONS on Information and Systems
Publication Date: 2019/08/01
Online ISSN: 1745-1361
Type of Manuscript: LETTER
Category: Computer System
Hadoop, Hadoop distributed file system, video processing, group of pictures,
Full Text: PDF(510.1KB)>>
Hadoop, a distributed processing framework for big-data, is now widely used for multimedia processing. However, when processing video data from a Hadoop distributed file system (HDFS), unnecessary network traffic is generated due to an inefficient HDFS block slice policy for picture frames in video files. We propose a new block replication policy to solve this problem and compare the newly proposed HDFS with the original HDFS via extensive experiments. The proposed HDFS reduces network traffic, and increases locality between processing cores and file locations.