GaitSpike: event-based gait recognition with spiking neural network

Abstract

Existing vision-based gait recognition systems are mostly designed based on video footage acquired with RGB cameras. Appearance-, model- and motion-based techniques commonly used by these systems require silhouette segmentation, skeletal contour detection and optical flow patterns, respectively for features extraction. The extracted features are typically classified by convolutional neural networks to identify the person. These preprocessing steps are computationally intensive due to the high visual data redundancies and their accuracies can be influenced by background variations and non-locomotion related external factors. In this paper, we propose GaitSpike, a new gait recognition system that synergistically combines the advantages of sparsity-driven event-based camera and spiking neural network (SNN) for gait biometric classification. Specifically, a domain-specific locomotion-invariant representation (LIR) is proposed to replace the static Cartesian coordinates of the raw address event representation of the event camera to a floating polar coordinate reference to the motion center. The aim is to extract the relative motion information between the motion center and other human body parts to minimize the intra-class variance to promote the learning of inter-class features by the SNN. Experiments on a real event-based gait dataset DVS128-Gait and a synthetic event-based gait dataset EV-CASIA-B show that GaitSpike achieves comparable accuracy as RGB camera based gait recognition systems with higher computational efficiency, and outperforms the state-of-the-art event camera based gait recognition systems.