Using Temporal Correlation to Optimize Stereo Matching in Video Sequences

Ming LI  Li SHI  Xudong CHEN  Sidan DU  Yang LI  

IEICE TRANSACTIONS on Information and Systems   Vol.E102-D   No.6   pp.1183-1196
Publication Date: 2019/06/01
Publicized: 2019/03/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2018EDP7273
Type of Manuscript: PAPER
Category: Image Recognition, Computer Vision
stereo matching,  disparity,  computer vision,  temporal correlation,  convolutional neural network,  

Full Text: PDF(4.8MB)>>
Buy this Article

The large computational complexity makes stereo matching a big challenge in real-time application scenario. The problem of stereo matching in a video sequence is slightly different with that in a still image because there exists temporal correlation among video frames. However, no existing method considered temporal consistency of disparity for algorithm acceleration. In this work, we proposed a scheme called the dynamic disparity range (DDR) to optimize matching cost calculation and cost aggregation steps by narrowing disparity searching range, and a scheme called temporal cost aggregation path to optimize the cost aggregation step. Based on the schemes, we proposed the DDR-SGM and the DDR-MCCNN algorithms for the stereo matching in video sequences. Evaluation results showed that the proposed algorithms significantly reduced the computational complexity with only very slight loss of accuracy. We proved that the proposed optimizations for the stereo matching are effective and the temporal consistency in stereo video is highly useful for either improving accuracy or reducing computational complexity.