Optimal Billboard Deformation via 3D Voxel for Free-Viewpoint System

Keisuke NONAKA  Houari SABIRIN  Jun CHEN  Hiroshi SANKOH  Sei NAITO  

Publication
IEICE TRANSACTIONS on Information and Systems   Vol.E101-D   No.9   pp.2381-2391
Publication Date: 2018/09/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2018EDP7039
Type of Manuscript: PAPER
Category: Image Processing and Video Processing
Keyword: 
free-viewpoint system,  billboard model,  voxel model,  image deformation,  convex optimization,  

Full Text: PDF(3.1MB)
>>Buy this Article


Summary: 
A free-viewpoint application has been developed that yields an immersive user experience. One of the simple free-viewpoint approaches called “billboard methods” is suitable for displaying a synthesized 3D view in a mobile device, but it suffers from the limitation that a billboard should be positioned in only one position in the world. This fact gives users an unacceptable impression in the case where an object being shot is situated at multiple points. To solve this problem, we propose optimal deformation of the billboard. The deformation is designed as a mapping of grid points in the input billboard silhouette to produce an optimal silhouette from an accurate voxel model of the object. We formulate and solve this procedure as a nonlinear optimization problem based on a grid-point constraint and some a priori information. Our results show that the proposed method generates a synthesized virtual image having a natural appearance and better objective score in terms of the silhouette and structural similarity.