Pixel Selection and Intensity Directed Symmetry for High Frame Rate and Ultra-Low Delay Matching System

Tingting HU  Takeshi IKENAGA  

IEICE TRANSACTIONS on Information and Systems   Vol.E101-D   No.5   pp.1260-1269
Publication Date: 2018/05/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2017MVP0012
Type of Manuscript: Special Section PAPER (Special Section on Machine Vision and its Applications)
Category: Machine Vision and its Applications
high frame rate and ultra-low delay image processing,  local binary feature based matching,  keypoint orientation calculation,  FPGA,  

Full Text: PDF(2.9MB)>>
Buy this Article

High frame rate and ultra-low delay matching system plays an increasingly important role in human-machine interactive applications which call for higher frame rate and lower delay for a better experience. The large amount of processing data and the complex computation in a local feature based matching system, make it difficult to achieve a high process speed and ultra-low delay matching with limited resource. Aiming at a matching system with the process speed of more than 1000 fps and with the delay of less than 1 ms/frame, this paper puts forward a local binary feature based matching system with field-programmable gate array (FPGA). Pixel selection based 4-1-4 parallel matching and intensity directed symmetry are proposed for the implementation of this system. To design a basic framework with the high process speed and ultra-low delay using limited resource, pixel selection based 4-1-4 parallel matching is proposed, which makes it possible to use only one-thread resource consumption to achieve a four-thread processing. Assumes that the orientation of the keypoint will bisect the patch best and will point to the region with high intensity, intensity directed symmetry is proposed to calculate the keypoint orientation in a hardware friendly way, which is an important part for a rotation-robust matching system. Software experiment result shows that the proposed keypoint orientation calculation method achieves almost the same performance with the state-of-art intensity centroid orientation calculation method in a matching system. Hardware experiment result shows that the designed image process core supports to process VGA (640×480) videos at a process speed of 1306 fps and with a delay of 0.8083 ms/frame.