Prediction of Driver Drowsiness Level Using Recurrent Neural Networks and Multi-Time-Scale Fusion

There is accumulating evidence that drowsy driving is one of the leading causes of vehicle crashes and accidents worldwide. Consequently, automotive manufacturers started to develop in-vehicle drowsiness detection devices. However, due to the limited computation resources and the complexity of the vehicular environment, the existing products' performance is limited. Moreover, the vast majority of the commercialized products focus on monitoring the subject's current drowsiness level, whereas predicting drowsiness level in advance to avoid future risks is overlooked. In this research, a multi-time-scale fusion approach is proposed where prediction results from both long-term and short-term Recurrent Neural Networks (RNN) were combined to predict a person's drowsiness level. Our results indicate that the proposed fusion strategies can successfully capture both the short-term microsleep-related features and long-term sleepiness features and improve the drowsiness prediction performance. The effectiveness of our model was evaluated on the publicly available DROZY dataset with more than 252k frames, where the accuracy of long-term and short-term models outperform existing reported results, reached 97.9% and 71.4%, respectively. Experimental results showed that the implementation of parallel computation and asynchronous fusion ensured our system's processing speed is over 30 frames per second on an embedded computation platform, making it suitable for an in-vehicle application.