Summarization of 3D Video by Rate-Distortion Trade-off

Jianfeng XU  Toshihiko YAMASAKI  Kiyoharu AIZAWA  

IEICE TRANSACTIONS on Information and Systems   Vol.E90-D   No.9   pp.1430-1438
Publication Date: 2007/09/01
Online ISSN: 1745-1361
DOI: 10.1093/ietisy/e90-d.9.1430
Print ISSN: 0916-8532
Type of Manuscript: PAPER
Category: Image Processing and Video Processing
3D video,  summarization,  key frame extraction,  rate-distortion optimization,  

Full Text: PDF>>
Buy this Article

3D video, which consists of a sequence of mesh models, can reproduce dynamic scenes containing 3D information. To summarize 3D video, a key frame extraction method is developed using rate-distortion (R-D) trade-off. For this purpose, an effective feature vector is extracted for each frame. Shot detection is performed using the feature vectors as a preprocessing followed by key frame extraction. Simple but reasonable definitions of rate and distortion are presented. Based on an assumption of linearity, an R-D curve is generated in each shot, where the locations of the key frames are optimized. Finally, R-D trade-off can be achieved by optimizing a cost function using a Lagrange multiplier, where the number of key frames is optimized in each shot. Therefore, our system will automatically determine the best locations and the number of key frames in the sense of R-D trade-off. Our experimental results show the extracted key frames are compact and faithful to the original 3D video.