Gait recognition by dynamic vision sensor and deep neural network

Abstract

Gait recognition is a study about identifying an individual by the pattern of walking. In this project, we explored the application of Dynamic Vision Sensor (DVS) event- based data on capturing gait movement and the Deep Learning approach for the gait recognition task. DVS has a different approach to capturing a gait movement compared to the commonly used RGB sensors. Instead of capturing an image as a frame by RGB camera, DVS captures gait movement as asynchronous events when there are changes in intensity. DVS offers several benefits compared to RGB sensors such as the ability to capture movement in microseconds, lower consumption in resources, and larger dynamic range. On the other hand, the DVS sensor is sensitive to noise, hence reducing noise events. Moreover, the deep learning approach, specifically the convolutional neural network has been proven successful in the image recognition task. The main objective of this project is to build and evaluate the performance of a gait recognition task by a Deep Neural Network (DNN) using event-based data collected from DVS. We used an event- based DVS128-Gait dataset. The DNN is expected to identify a person’s gait using event-based data produced by DVS. In this project, we establish a data pre- processing method to reduce noise from the DVS camera and transform event-based data into a suitable data representation format for deep learning purposes. We will study the effect of noise filter for event-based data, and the difference in data representation to the performance of the deep neural network. Additionally, we will explore the effect of modification of deep neural network training configurations on its performance for the gait recognition task. Finally, we performed model performance benchmarking with different king of popular CNN architecture.