Memory structure for multi-modal data streams

Abstract

Nowadays humanoid have a variety of application, in the field of medical it has a significant role, humanoids are used for testing purposes of various treatments. In our project the humanoid is working as a schizophrenia patient on which a psychiatrist will be trained to handle schizophrenic patient. But the main part for any robot to behave like a humanoid is memory, it should be able remember things and then act likewise. In this project, a memory modal is built to store the data. A database is made where different kind of data like audio, images, text, emotions, skelton data is stored in a table like structure. Also, data analytics is performed on the data which is received from various sources of data like microphone, camera, affectiva and Kinect respectively. And when the same person appears in front of the Nadine, data is retrieved from the tables and then accordingly she is able to respond. In this way the power of remembrance is implement in the brain of the humanoid.