Point of Gaze Estimation Using Corneal Surface Reflection and Omnidirectional Camera Image

Taishi OGAWA  Atsushi NAKAZAWA  Toyoaki NISHIDA  

IEICE TRANSACTIONS on Information and Systems   Vol.E101-D   No.5   pp.1278-1287
Publication Date: 2018/05/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2017MVP0020
Type of Manuscript: Special Section PAPER (Special Section on Machine Vision and its Applications)
Category: Machine Vision and its Applications
point of gaze estimation,  eye tracking,  corneal imaging,  corneal reflection,  spherical panorama,  

Full Text: PDF(12MB)>>
Buy this Article

We present a human point of gaze estimation system using corneal surface reflection and omnidirectional image taken by spherical panorama cameras, which becomes popular recent years. Our system enables to find where a user is looking at only from an eye image in a 360° surrounding scene image, thus, does not need gaze mapping from partial scene images to a whole scene image that are necessary in conventional eye gaze tracking system. We first generate multiple perspective scene images from an omnidirectional (equirectangular) image and perform registration between the corneal reflection and perspective images using a corneal reflection-scene image registration technique. We then compute the point of gaze using a corneal imaging technique leveraged by a 3D eye model, and project the point to an omnidirectional image. The 3D eye pose is estimate by using the particle-filter-based tracking algorithm. In experiments, we evaluated the accuracy of the 3D eye pose estimation, robustness of registration and accuracy of PoG estimations using two indoor and five outdoor scenes, and found that gaze mapping error was 5.546 [deg] on average.