Neural network structure for spatio-temporal long-term memory : theory and applications

Abstract

This thesis presents a novel spatio-temporal neural network that is inspired by the Long-Term Memory (LTM) structure of the human cortex. The proposed LTM neural network model processes real-valued multi-dimensional data sequences and is designed to addresses three critical problems in robust sequential learning, namely error tolerance, significance of sequence elements and memory forgetting. Extensive synthetic simulations were performed to study the statistical properties of the LTM model and its robustness to different types of distortions. A computational framework to align and combine multiple sequences stored by LTM cells is also proposed. This framework provides a compact representation for handling multiple training sequences belonging to the same class and enhances error tolerance by learning spatio-temporal structures or grammatical rules that may exist in these multiple sequences.

The second part of this thesis applies the proposed LTM framework to three different pattern recognition problems, each of which deals with a different data modality. These applications include hand-sign language interpretation with the UCI Australian

Sign Language (ASL) dataset, visual topological place localization for robotic navigation with the COsy Localization Dataset (COLD) and isolated phoneme recognition for speech processing with the NIST TIMIT dataset. Experimental results show that the proposed

LTM model is able to produce comparable results with the current state-of-the-art methods for the each of the chosen datasets, both in terms of recognition performance and the efficiency in memory storage. More importantly, the results show that the proposed model is sufficiently general to different application domains that require sequential data analysis. This suggests that the biologically-inspired LTM model could be a potential candidate for use as a computational unit of episodic memory in the development of cognitive machines.