Imperceptible On-Screen Markers for Mobile Interaction on Public Large Displays

Goshiro YAMAMOTO  Luiz SAMPAIO  Takafumi TAKETOMI  Christian SANDOR  Hirokazu KATO  Tomohiro KURODA  

IEICE TRANSACTIONS on Information and Systems   Vol.E100-D   No.9   pp.2027-2036
Publication Date: 2017/09/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2016PCP0015
Type of Manuscript: Special Section PAPER (Special Section on Picture Coding and Image Media Processing)
imperceptible marker,  human visual perception,  unsynchronized capture,  

Full Text: PDF>>
Buy this Article

We present a novel method to enable users to experience mobile interaction with digital content on external displays by embedding markers imperceptibly on the screen. Our method consists of two parts: marker embedding on external displays and marker detection. To embed markers, similar to previous work, we display complementary colors in alternating frames, which are selected by considering L*a*b color space in order to make the markers harder for humans to detect. Our marker detection process does not require mobile devices to be synchronized with the display, while certain constraints for the relation between camera and display update rate need to be fulfilled. In this paper, we have conducted three experiments. The results show 1) selecting complementary colors in the a*b* color plane maximizes imperceptibility, 2) our method is extremely robust when used with static contents and can handle animated contents up to certain optical flow levels, and 3) our method was proved to work well in case of small movements, but large movements can lead to loss of tracking.