Infants' Pain Recognition Based on Facial Expression: Dynamic Hybrid Descriptions

Ruicong ZHI  Ghada ZAMZMI  Dmitry GOLDGOF  Terri ASHMEADE  Tingting LI  Yu SUN  

IEICE TRANSACTIONS on Information and Systems   Vol.E101-D   No.7   pp.1860-1869
Publication Date: 2018/07/01
Publicized: 2018/04/20
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2017EDP7272
Type of Manuscript: PAPER
Category: Artificial Intelligence, Data Mining
infants pain assessment,  temporal geometric descriptor,  LBP-TOP,  decision fusion,  video analysis,  

Full Text: PDF(1.4MB)>>
Buy this Article

The accurate assessment of infants' pain is important for understanding their medical conditions and developing suitable treatment. Pediatric studies reported that the inadequate treatment of infants' pain might cause various neuroanatomical and psychological problems. The fact that infants can not communicate verbally motivates increasing interests to develop automatic pain assessment system that provides continuous and accurate pain assessment. In this paper, we propose a new set of pain facial activity features to describe the infants' facial expression of pain. Both dynamic facial texture feature and dynamic geometric feature are extracted from video sequences and utilized to classify facial expression of infants as pain or no pain. For the dynamic analysis of facial expression, we construct spatiotemporal domain representation for texture features and time series representation (i.e. time series of frame-level features) for geometric features. Multiple facial features are combined through both feature fusion and decision fusion schemes to evaluate their effectiveness in infants' pain assessment. Experiments are conducted on the video acquired from NICU infants, and the best accuracy of the proposed pain assessment approaches is 95.6%. Moreover, we find that although decision fusion does not perform better than that of feature fusion, the False Negative Rate of decision fusion (6.2%) is much lower than that of feature fusion (25%).