Car cabin surveillance using computer vision

Abstract

Driver assistance systems (DAS) in cars has become more intelligent than ever and it has enabled autonomous driving in certain conditions. Consequently, there is a need to always monitor the drivers’ attention inside the car to make sure that they are attentive enough and ready to take over when the DAS asked. As such, this research proposes a new computer vision approach of where there would be a wide-angle camera placed in the center mirror monitoring the driver and passenger and it will be connected to a computing unit inside the car which utilizes deep learning-based AI model to detect the activity of the driver and passenger and determine whether it is safe to engage level 2 DAS. In this paper, a literature review about the different approaches to the problem will be given together with the different datasets that are available to support the model training. Moreover, this paper presents the findings and the results from 2 different approaches which are classic image classification-based approach, and a novel 2-stage classifier was proposed whereby the frames are passed to pose estimation and face mesh detection model that will output key points for both the face mesh and the human body which are then passed to an algorithm that will output the driver activity.