Space-time event clouds-based event processing

Abstract

Recently developed event cameras demonstrate increasing potential in computer vision applications. There have been a number of techniques to process specially formatted event data. However, to take advantage of existing frameworks for fame-based video analytics, these techniques normally apply space-time disentanglement in the pre-processing stage. This kind of disentanglement does not fully utilize the rich temporal information inherent in the event data. This thesis proposes the Space-time Event Cloud concept to model event data as 3D event clouds, which aims to extract spatiotemporal features from the entangled states directly. With this concept, this thesis solves the event-based hand gesture recognition task with state-of-the-art performance. Furthermore, this thesis proposes a dedicated network, Space-time EventNet, to emphasize the interaction between neighboring events to enhance recognition accuracy. Finally, this thesis designs an ultra-efficient detection-tracking-recognition pipeline. This pipeline is portable enough to achieve success in light-weighted embedded systems.