Driver Drowsiness Estimation by Parallel Linked Time-Domain CNN with Novel Temporal Measures on Eye States

Kenta NISHIYUKI  Jia-Yau SHIAU  Shigenori NAGAE  Tomohiro YABUUCHI  Koichi KINOSHITA  Yuki HASEGAWA  Takayoshi YAMASHITA  Hironobu FUJIYOSHI  

IEICE TRANSACTIONS on Information and Systems   Vol.E103-D   No.6   pp.1276-1286
Publication Date: 2020/06/01
Online ISSN: 1745-1361
DOI: 10.1587/transinf.2019MVP0017
Type of Manuscript: Special Section PAPER (Special Section on Machine Vision and its Applications)
driver monitoring,  driver drowsiness estimation,  time-domain CNN,  PERCLOS,  

Full Text: PDF(1.1MB)>>
Buy this Article

Driver drowsiness estimation is one of the important tasks for preventing car accidents. Most of the approaches are binary classification that classify a driver is significantly drowsy or not. Multi-level drowsiness estimation, that detects not only significant drowsiness but also moderate drowsiness, is helpful to a safer and more comfortable car system. Existing approaches are mostly based on conventional temporal measures which extract temporal information related to eye states, and these measures mainly focus on detecting significant drowsiness for binary classification. For multi-level drowsiness estimation, we propose two temporal measures, average eye closed time (AECT) and soft percentage of eyelid closure (Soft PERCLOS). Existing approaches are also based on a time domain convolutional neural network (CNN) as deep neural network models, of which layers are linked sequentially. The network model extracts features mainly focusing on mono-temporal resolution. We found that features focusing on multi-temporal resolution are effective to multi-level drowsiness estimation, and we propose a parallel linked time-domain CNN to extract the multi-temporal features. We collected an own dataset in a real environment and evaluated the proposed methods with the dataset. Compared with existing temporal measures and network models, Our system outperforms the existing approaches on the dataset.