Intelligent video proctoring for online assessments

Abstract

During the Covid-19 pandemic period, the need for the online teaching and examination shows an increasing trend. In order to ensure the validity and fairness of the examinations, a lot of manpower is required to supervise the real-time video recording of each student’s online assessment. In this case, we develop an intelligent video proctoring system based on facial recognition and object&speech detection technique, which can monitor suspicious cheating behaviors to reduce manpower requirement. There are three main parts in our system, namely facial detection system, object detection system and speech detection system. We use Histogram of Oriented Gradients (HOG) feature detector to detect the face in real-time video and constructed convolutional neural networks to perform the facial recognition and authentication. In the systems of suspicious object detection and human speech detection, we use YOLO framework and Hidden Markov Model technique to detect the emergence of suspicious cheating behavior. In addition, we use PyQt5 framework to design a simple user interface of our system. We have simulated the entire examination process and tested the performance of our system, and the results show that our system can achieve a good perform performance in the application of online assessments with more than 90% detection accuracy and high detection speed.