Shape based hand gesture recognition

Abstract

According to Siddiqi et al.[45], “Part-based representations allow for recognition that is robust in the presence of occlusion, movement, deletion, or growth of portions of an object. In the task of forming high-level object-centered models from low-level image-based features, parts serve as an intermediate representation”. Shape decomposition is a fundamental problem in part-based shape representation. I propose the Minimum Near-Convex Decomposition (MNCD) to decompose arbitrary 2D and 3D shapes into the minimum number of “near-convex” parts. Visual naturalness is important for shape representation [36]. To improve the visual naturalness of the decomposition, two perception rules are considered and the shape decomposition is formulated as a combinatorial optimization problem by minimizing the number of non-intersection cuts. With the degree of near-convexity as a user specified parameter, my decomposition is robust to local distortions and shape deformations. To justify the advantages of my shape decomposition, I show its superiority in the application of hand gesture recognition. The recently developed depth sensors, e.g., the Kinect sensor, have provided new opportunities for human-computer-interaction (HCI). Although great progress has been made by leveraging the Kinect sensor, e.g. in human body tracking and body gesture recognition, robust hand gesture recognition remains an open problem. Compared to the entire human body, the hand is a smaller object with more complex articulations and more easily affected by segmentation errors. It is thus a very challenging problem to recognize hand gestures. I aim at building a robust hand gesture recognition system from the shape feature, using the Kinect sensor. To handle the noisy hand shape obtained from the Kinect sensor, I propose a novel distance metric, called Finger-Earth Mover’s Distance (FEMD), to measure the dissimilarity between hand shapes. As it only matches fingers while not the whole hand shape, it can better distinguish hand gestures of slight differences. In order to accurately detect the fingers, the proposed near-convex shape decomposition method MNCD is employed.