Silicon FET reservoir for dynamic edge vision

Abstract

AI is driving significant advancements in computer science and various practical applications. However, its current reliance on centralized infrastruc- tures and large-scale servers results in substantial power consumption and high latency. To address these challenges, this article introduces a hybrid system that integrates a physical reservoir with a software-based readout layer into a ResNet neural network, specifically designed for dynamic edge vision applications. We utilize a DVS-based edge processor, which helps offload certain tasks from the centralized server and detect variations in object images. By leveraging the ability of oxide vacancy defects to capture and release carriers, the H f O2 bulk exhibits memory-like behavior, converting pulse signals from the edge sen- sor into analog values. These multidimensional values are then compiled into frames, capturing both the temporal and spatial features of the original video. For readout training and classification, we employ two types of neural net- works: ResNet and ResNet-LSTM. Analyzing the output frames of the physi- cal reservoir, the ResNet model achieves an average accuracy of 94.78% using only a 2-second event stream, with a maximum accuracy of 96.00%. By incor- porating the ResNet-LSTM model to analyze only the initial 3 seconds of the event stream, using 0.5-second clips, we attain an average accuracy of 97.34% and a maximum accuracy of 98.40%. These experiments are conducted on the DVS128 dataset, classifying 10 gesture categories. Compared to existing models, our approach achieves outstanding accuracy while maintaining relatively lower computational weight.