Weighted 4D-DCT Basis for Compressively Sampled Light Fields

Yusuke MIYAGI  Keita TAKAHASHI  Toshiaki FUJII  

IEICE TRANSACTIONS on Fundamentals of Electronics, Communications and Computer Sciences   Vol.E99-A   No.9   pp.1655-1664
Publication Date: 2016/09/01
Online ISSN: 1745-1337
DOI: 10.1587/transfun.E99.A.1655
Type of Manuscript: Special Section PAPER (Special Section on Recent Advances in Image Sampling and Reconstruction)
light field photography,  compressive sensing,  coded mask,  DCT,  

Full Text: PDF(15.8MB)
>>Buy this Article

Light field data, which is composed of multi-view images, have various 3D applications. However, the cost of acquiring many images from slightly different viewpoints sometimes makes the use of light fields impractical. Here, compressive sensing is a new way to obtain the entire light field data from only a few camera shots instead of taking all the images individually. In paticular, the coded aperture/mask technique enables us to capture light field data in a compressive way through a single camera. A pixel value recorded by such a camera is a sum of the light rays that pass though different positions on the coded aperture/mask. The target light field can be reconstructed from the recorded pixel values by using prior information on the light field signal. As prior information, the current state of the art uses a dictionary (light field atoms) learned from training datasets. Meanwhile, it was reported that general bases such as those of the discrete cosine transform (DCT) are not suitable for efficiently representing prior information. In this study, however, we demonstrate that a 4D-DCT basis works surprisingly well when it is combined with a weighting scheme that considers the amplitude differences between DCT coefficients. Simulations using 18 light field datasets show the superiority of the weighted 4D-DCT basis to the learned dictionary. Furthermore, we analyzed a disparity-dependent property of the reconstructed data that is unique to light fields.